Mount on-premises NFS – Valohai Help

This feature is available for customers who are using their own cloud or on-premises workers.

Mounting gives you direct access to the network file system like AWS EFS or GCP Filestore, without having to download the files on your machine.

Valohai does not version or keep track of the changes made inside the mounted file system. The files that are created, edited, or deleted will not be versioned as part of a Valohai execution.

NFS mounts are not version controlled

We strongly recommend using the Valohai inputs and outputs system, as they are versioned as a part of the execution.

You can mount a filesystem to access a large dataset, run preprocessing operations, and output the processed dataset into Valohai Outputs before it’s used further in the pipeline. This way the snapshot of the preprocessed data will be versioned.

Define a mount in YAML

- step:
    name: mount-sample
    image: python:3.8
    command:
        - ls -la /my-data
    mounts:
        - destination: /my-data
          source: /path/to/directory/outside/container

Mounts are defined for each environment separately. Contact Valohai, if you see an error mounts were ignored due to environment configuration in your execution logs.

Define a mount in an API call

You can define mounts when starting executions with an API call. You do not need to define the mount in the valohai.yaml in this case.

{
"project": "<project-id>",
"commit": "<commit>",
"step": "mount-sample",
"command": "ls /mydata",
"image": "python:3.8",
"mounts": ["/mnt/point:/mydata"]
}

The information about the mount is not currently preserved in the execution details . This means that copying the execution from the Valohai UI does not include the mounting when the execution has been started with an API call.

Define a mount in YAML

Define a mount in an API call

Related articles