Mount AWS Elastic File System (NFS)

Mounting gives you direct access to the network file system like AWS EFS without having to download the files on your machine.

Valohai does not version or keep track of the changes made inside the mounted file system. The files that are created, edited, or deleted will not be versioned as part of a Valohai execution.

Define a mount in YAML

You can either use an existing or create a new AWS EFS.

• Create your EFS in the same VPC where all Valohai resources are in or setup VPC peering between the two VPCs
• Make sure the security group of your EFS has an inbound rule to accept traffic from the Valohai workers (sg-valohai-workers)
• Valohai will connect to the EFS over DNS name or IP address. Make sure the VPC has DNS hostnames and DNS resolution enabled, if you’re connecting over DNS name.

Example valohai.yaml configuration:

- step:
    name: mount-sample
    image: python:3.8
    command:
        - ls -la /my-data
    mounts:
        - destination: /my-data
          source: fs-1234aa62.efs.eu-west-1.amazonaws.com:/
          type: nfs
          readonly: false

Define a mount in an API call

You can define mounts when starting executions with an API call. You do not need to define the mount in the valohai.yaml in this case but the same prerequisites apply here as well.

• Create your EFS in the same VPC where all Valohai resources are in or setup VPC peering between the two VPCs
• Make sure the security group of your EFS has an inbound rule to accept traffic from the Valohai workers (sg-valohai-workers)
• Valohai will connect to the EFS over DNS name or IP address. Make sure the VPC has DNS hostnames and DNS resolution enabled, if you’re connecting over DNS name.

{
  "mounts": [
    {
      "destination": "/my-data",
      "source": "fs-1234aa62.efs.eu-west-1.amazonaws.com:/",
      "type": "nfs",
      "readonly": false
    }
  ]
}