Bring your metric collection to Valohai

• Easily keep track of your execution key metrics and sort executions based on your custom metrics.
• Anything that you print as JSON from your code gets collected as potential Valohai metadata.
• Visualize and compare execution metrics as a time-series, a scatter plot, or a confusion matrix in the UI.
• You can set early stopping rules for executions and Tasks or stop a pipeline from proceeding based on metadata values.
• You can download the metadata as a CSV or JSON file from the executions metadata tab.

Collect metrics with Python and the valohai-utils toolkit

import valohai

with valohai.metadata.logger() as logger:
    logger.log("epoch", epoch)
    logger.log("accuracy", accuracy)
    logger.log("loss", loss)

Collect metrics with Python without the valohai-utils toolkit

import json

print()
print(json.dumps({
    "epoch": epoch,
    "accuracy": accuracy,
    "loss": loss
}))

Collect metrics with R

library(jsonlite)

metadataEvent <- list()
metadataEvent[["epoch"]] <- epoch
metadataEvent[["accuracy"]] <- accuracy
metadataEvent[["loss"]] <- loss

write(toJSON(metadataEvent, auto_unbox=TRUE), stdout())