Add a valohai.yaml configuration file

Valohai is completely technology agnostic. You can develop in notebooks or scripts in a language and framework of your choice.

Let’s start by bringing just a single job type to Valohai.

Using Python and the valohai-utils toolkit

Open your existing Python file and define a Valohai step inside it:

import valohai

# Prepares a new step called train in valohai.yaml
# Execute this Python file to run every time the train step is launched in Valohai
valohai.prepare(step="train")

You can now generate a valohai.yaml configuration file that includes the train step by running:

vh yaml step myfilename.py

This will create:

• A valohai.yaml file with the train step that will run:
  • pip install -r requirements.txt
  • your Python script.
• A requirements.txt with valohai-utils
  • If you have an existing requirements.txt file the valohai-utils line will be just added there.

By default, your step will use a standard Python Docker image and install everything from requirements.txt during runtime.

You could also specify a different Docker image by passing the image name to prepare.

import valohai

# Prepares a new step called train in valohai.yaml
# Execute this Python file to run every time the train step is launched in Valohai
# inside a GPU enabled Tensorflow 2.6.1 image.
valohai.prepare(step="train", image="tensorflow/tensorflow:2.6.1-gpu")

and then rerun vh yaml step to update your valohai.yaml.

vh yaml step myfilename.py

You can now upload your local code to Valohai and run the train step inside the Docker container you just defined.

vh execution run --adhoc train

Using any other language

Create a new file called valohai.yaml and define a Step inside it.

In our example below we’re defining two commands to be run every time the step called train is run. You can remove the pip install command if you don’t have a need for it.

- step:
    name: train
    image: tensorflow/tensorflow:2.6.1-gpu
    command:
      - pip install -r requirements.txt
      - python myfilename.py

You can now upload your local code to Valohai and run the train step inside the Docker container you just defined.

vh execution run --adhoc train