As with any Valohai executions, you can choose the environments you want to use for your pipeline steps.
You can change the environments for your pipeline steps in the Valohai UI when creating the pipeline. Click on the step in the pipeline graph and then choose the environment from the dropdown menu in the Runtime section.
You can use the
environment property in the step definition in you
valohai.yaml. The value is the environment slug and you can get it by running the command
vh environments on your CLI.
When this property is not defined, the default environment set for your organization (or project) will be used.
- step: name: preprocess-dataset image: python:3.9 environment: aws-eu-west-1-g3s-xlarge command: - pip install numpy valohai-utils - python ./preprocess_dataset.py inputs: - name: dataset default: https://valohaidemo.blob.core.windows.net/mnist/mnist.npz
In some special cases you might want to override the environment for selected nodes in the pipeline definition in your valohai.yaml.
- pipeline: name: Training Pipeline nodes: - name: preprocess type: execution step: preprocess-dataset override: environment: aws-eu-west-1-g3s-xlarge - name: train type: execution step: train-model - name: evaluate type: execution step: batch-inference edges: - [preprocess.output.preprocessed_mnist.npz, train.input.dataset] - [train.output.model*, evaluate.input.model]