In this section, you will learn how to:
- Import Valohai in your Python scripts
- Save Valohai outputs
- Run executions in Valohai
A sample training script
- Download this mnist.npz file to your working directory
- Create a new file
valohai-quickstart/train.py
and paste the following example script:
import numpy as np
import tensorflow as tf
input_path = 'mnist.npz'
with np.load(input_path, allow_pickle=True) as f:
x_train, y_train = f['x_train'], f['y_train']
x_test, y_test = f['x_test'], f['y_test']
x_train, x_test = x_train / 255.0, x_test / 255.0
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation='softmax')
])
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer='adam',
loss=loss_fn,
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=5)
model.evaluate(x_test, y_test, verbose=2)
output_path = 'model.h5'
model.save(output_path)
Create the configuration file
Add a new file called valohai.yaml to define a new Valohai step called train. Whenever we launch this step in Valohai we want to run the train.py script inside a TensorFlow 2.6.0 Docker image.
- step:
name: train-model
command: python train.py
image: tensorflow/tensorflow:2.6.0
The valohai.yaml configuration file is the blueprint for your project. To make sure the indentation is correct and all the steps contain the required properties, it is recommended to run the command
vh lint
in the directory where you have the valohai.yaml to check for any syntax errors.
Run in Valohai
Finally, we can start a new Valohai job directly from your command-line.
vh exec run train-model --adhoc
vh exec run step-name
creates a new Valohai execution with the step that we defined.--adhoc
tells Valohai that the code for this execution is coming from our local machine instead of our code repository. Valohai will package the local directory and upload it for an execution.
View in Valohai
- Open your project on app.valohai.com
- Open the latest execution
- The details type contains information about the executions
- The logs tab contains all logs from the execution
Next: Upload output data ⇒
Comments
0 comments
Please sign in to leave a comment.