How can I override default parameters in a pipeline? – Valohai Help

You can override the default parameters of a node inside a pipeline using the override attribute.

In the example below we're creating a pipeline that runs a preprocessing step and three training steps.

The three training nodes are overriding either the default input data, the parameters, or both.

When overriding parameters keep in mind that:

the whole parameter section gets overridden, meaning you always need to specify all the parameters of that step. Even the ones that keep their default value.
you should always specify the name, type, and default value when overriding a parameter

- step:
    name: preprocess-dataset
    image: python:3.9
    command:
      - pip install numpy valohai-utils
      - python ./preprocess_dataset.py
    inputs:
      - name: dataset
        default: https://valohaidemo.blob.core.windows.net/mnist/mnist.npz

- step:
    name: train-model
    image: tensorflow/tensorflow:2.6.0
    command:
      - pip install valohai-utils
      - python ./train_model.py {parameters}
    parameters:
      - name: epochs
        default: 5
        type: integer
      - name: learning_rate
        default: 0.001
        type: float
    inputs:
      - name: dataset
        default: https://valohaidemo.blob.core.windows.net/mnist/preprocessed_mnist.npz
        
- pipeline:
    name: Three-Trainings Pipeline
    nodes:
      - name: preprocess
        type: execution
        step: preprocess-dataset
      - name: train1
        type: execution
        step: train-model
        override:
          inputs:
            - name: dataset
      - name: train2
        type: execution
        step: train-model
        override:
          inputs:
            - name: dataset
          parameters:
            - name: epochs
              default: 10
              type: integer
            - name: learning_rate
              default: 0.001
              type: float
      - name: train3
        type: execution
        step: train-model
        override:
          inputs:
            - name: dataset
          parameters:
            - name: epochs
              default: 15
              type: integer
            - name: learning_rate
              default: 0.001
              type: float
    edges:
      - [preprocess.output.preprocessed_mnist.npz, train1.input.dataset]
      - [preprocess.output.preprocessed_mnist.npz, train2.input.dataset]
      - [preprocess.output.preprocessed_mnist.npz, train3.input.dataset]

Related articles