In this section you will:
- Add a simple pipeline to your
- Learn how to use different types of edges (optional).
Add a pipeline to
The valohai.yaml configuration file is the blueprint for your project. To make sure the indentation is correct and all the steps contain the required properties, it is recommended to run the command
in the directory where you have the valohai.yaml to check for any syntax errors.
valohai.yaml file and add the following
pipeline in it.
- pipeline: name: preprocess-and-train nodes: - name: preprocess type: execution step: preprocess-dataset - name: train type: execution step: train-model edges: - [preprocess.output.preprocessed_mnist.npz, train.input.preprocessed_dataset]
This pipeline will:
Create two nodes:
preprocesswill execute your
trainwill execute your
Create one edge to connect the nodes:
preprocess-dataset's outputs will be passed into the input called
You can now continue to the next part of the learning path. Optionally, you can keep reading to learn more about the different edge types.
The simplest syntax to define edges is [
TARGET have 3 properties separated by dots: node, type and key. Thus the shorthand syntax becomes: [
- The available edge
- outputs (only source node)
- inputs (only target node)
- metadata (only source node)
- files (only for deployment nodes)
In the example above you already saw how to pipe output from the source node into a target node input. Here are examples for using the other types of edges.
edges: # Parameter-parameter edge # Note that the parameters have to exist in the yaml for both source and target - [sourcenode.parameter.parameterkey, targetnode.parameter.parameterkey] # Metadata-parameter edge # Note that you can only pass single values and not for example list type metadata - [sourcenode.metadata.metadatakey, targetnode.parameter.parameterkey] # Output-file edge # The file type is only available for deployment nodes. # Note that the filename is the name defined in the YAML. - [sourcenode.output.outputkey, targetnode.file.endpointname.filename]
Next: Run a pipeline ⇒
Please sign in to leave a comment.