- Defining parameters allows you to easily rerun, sort, and keep track of executions based on the parameter values used to run them.
- You can easily create (or copy) an execution and change the parameters in the UI, without changing your code.
- Defining parameters allows you to start creating Tasks where you run multiple parallel executions with different parameter combinations.
- Each step has its own parameter configuration in
valohai.yaml
. - A parameter can be a type of a
string
,integer
,float
, andflag
(=boolean). - Parameters are passed as command-line arguments to your code.
- Edit how parameters are passed to your code using pass-as.
- You can also parse parameter values from YAML and JSON files inside your execution. See file-based configuration.
Define parameters using Python and the valohai-utils toolkit
import valohai
# Define inputs available for this step and their default location
# The default location can be overriden when you create a new execution (UI, API or CLI)
default_inputs = {
'myinput': 's3://bucket/mydata.csv'
}
# Define parameters in a dictionary
default_parameters = {
'iterations': 10,
}
# Open the CSV file from Valohai inputs
with open(valohai.inputs("myinput").path()) as csv_file:
reader = csv.reader(csv_file, delimiter=',')
# Create a step 'train' in valohai.yaml with a set of parameters
valohai.prepare(step="train", image="tensorflow/tensorflow:2.6.1-gpu", default_inputs=default_inputs, default_parameters=default_parameters)
# Access the parameters in your code
for i in range(valohai.parameters('iterations').value):
print("Iteration %s" % i)
Generate or update your existing YAML file by running:
vh yaml step myfile.py
Define parameters using Python without the valohai-utils toolkit
import argparse
def parse_args():
parser = argparse.ArgumentParser()
parser.add_argument('--iterations', type=int, default=10)
return parser.parse_args()
args = parse_args()
print(args.iterations)
Create a valohai.yaml configuration file and define your step in it:
- step:
name: Train model
image: tensorflow/tensorflow:1.13.1
command: python myfile.py {parameters}
parameters:
- name: iterations
type: integer
default: 100
Define parameters using R
library(optparse)
option_list <- list(
make_option("--iterations", default = 10),
)
parser <- OptionParser(option_list = option_list)
args <- parse_args(parser)
iterations <- args$iterations
write(iterations, stdout())
Create a valohai.yaml configuration file and define your step in it:
- step:
name: Train model
image: tensorflow/tensorflow:1.13.1
command: python myfile.py {parameters}
parameters:
- name: iterations
type: integer
default: 100
Comments
0 comments
Please sign in to leave a comment.