You can override the default parameters of a node inside a pipeline using the override attribute.
In the example below we're creating a pipeline that runs a preprocessing step and three training steps.
The three training nodes are overriding either the default input data, the parameters, or both.
When overriding parameters keep in mind that you should always specify the name, type, and default value.
- step:
name: preprocess-dataset
image: python:3.9
command:
- pip install numpy valohai-utils
- python ./preprocess_dataset.py
inputs:
- name: dataset
default: https://valohaidemo.blob.core.windows.net/mnist/mnist.npz
- step:
name: train-model
image: tensorflow/tensorflow:2.6.0
command:
- pip install valohai-utils
- python ./train_model.py {parameters}
parameters:
- name: epochs
default: 5
type: integer
- name: learning_rate
default: 0.001
type: float
inputs:
- name: dataset
default: https://valohaidemo.blob.core.windows.net/mnist/preprocessed_mnist.npz
- pipeline:
name: Three-Trainings Pipeline
nodes:
- name: preprocess
type: execution
step: preprocess-dataset
- name: train1
type: execution
step: train-model
override:
inputs:
- name: dataset
- name: train2
type: execution
step: train-model
override:
inputs:
- name: dataset
parameters:
- name: epochs
default: 10
type: integer
- name: learning_rate
default: 0.001
type: float
- name: train3
type: execution
step: train-model
override:
inputs:
- name: dataset
parameters:
- name: epochs
default: 15
type: integer
- name: learning_rate
default: 0.001
type: float
edges:
- [preprocess.output.preprocessed_mnist.npz, train1.input.dataset]
- [preprocess.output.preprocessed_mnist.npz, train2.input.dataset]
- [preprocess.output.preprocessed_mnist.npz, train3.input.dataset]
Comments
0 comments
Please sign in to leave a comment.