Valohai fetches these details from the Docker API and
nvidia-smi. The available memory includes only the real memory, not the memory cache. Behind the scenes, Valohai uses Moby to fetch these details.
You can view the metrics during runtime and you can view the peak usage after the job is completed. Valohai doesn't store the historical data of the CPU/GPU/Memory usage throughout the execution.
Please sign in to leave a comment.