Bring your parameters to Valohai

• Defining parameters allows you to easily rerun, sort, and keep track of executions based on the parameter values used to run them.
• You can easily create (or copy) an execution and change the parameters in the UI, without changing your code.
• Defining parameters allows you to start creating Tasks where you run multiple parallel executions with different parameter combinations.
• Each step has its own parameter configuration in valohai.yaml.
• A parameter can be a type of a string, integer, float, and flag (=boolean).
• Parameters are passed as command-line arguments to your code.
  • Edit how parameters are passed to your code using pass-as.
  • You can also parse parameter values from YAML and JSON files inside your execution. See file-based configuration.

Define parameters using Python and the valohai-utils toolkit

import valohai

# Define inputs available for this step and their default location
# The default location can be overriden when you create a new execution (UI, API or CLI)
default_inputs = {
    'myinput': 's3://bucket/mydata.csv'
}

# Define parameters in a dictionary
default_parameters = {
    'iterations': 10,
}

# Open the CSV file from Valohai inputs
with open(valohai.inputs("myinput").path()) as csv_file:
    reader = csv.reader(csv_file, delimiter=',')

# Create a step 'train' in valohai.yaml with a set of parameters
valohai.prepare(step="train", image="tensorflow/tensorflow:2.6.1-gpu", default_inputs=default_inputs, default_parameters=default_parameters)

# Access the parameters in your code
for i in range(valohai.parameters('iterations').value):
    print("Iteration %s" % i)

Generate or update your existing YAML file by running:

vh yaml step myfile.py

Define parameters using Python without the valohai-utils toolkit

import argparse

def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('--iterations', type=int, default=10)
    return parser.parse_args()

args = parse_args()

print(args.iterations)

Create a valohai.yaml configuration file and define your step in it:

- step:
    name: Train model
    image: tensorflow/tensorflow:1.13.1
    command: python myfile.py {parameters}
    parameters:
      - name: iterations
        type: integer
        default: 100

Define parameters using R

library(optparse)

option_list <- list(
    make_option("--iterations", default = 10),
)

parser <- OptionParser(option_list = option_list)
args <- parse_args(parser)

iterations <- args$iterations
write(iterations, stdout())

Create a valohai.yaml configuration file and define your step in it:

- step:
    name: Train model
    image: tensorflow/tensorflow:1.13.1
    command: python myfile.py {parameters}
    parameters:
      - name: iterations
        type: integer
        default: 100