- Each Valohai execution is run inside a Docker container
- Your Docker image should contain most, if not all, of the packages and libraries needed to run your code (e.g. specific TensorFlow version, matplotlib, xgboost or something else).
- You can use public or private Docker images
- All Valohai notebook executions are based on a custom image called
valohai/pypermill
. - You can build your own Docker images on top of
valohai/pypermill
to include additional packages.
Below you’ll find list of the most commonly used Notebook images and their Dockerfiles. You can use any of these images in your executions, or build your own image on top of them.
Each of the images contains the library marked in the tag and the following libraries: seaborn
, numpy
, pandas
, matplotlib
, valohai-utils
, and statsmodels
.
The Tensorflow images have a GPU enabled version. You’ll recognize them by the -gpu
extension.
valohai/notebook:tensorflow-2.5.0
|valohai/notebook:tensorflow-2.5.0-gpu
valohai/notebook:tensorflow-1.15.5
|valohai/notebook:tensorflow-1.15.5-gpu
valohai/notebook:sklearn-0.24.2
valohai/notebook:sklearn-1.0
valohai/notebook:xgboost-1.4.2
You can change the default Docker image for your Notebook by clicking on Settings
inside your notebook.
Dockerfile
Below you’ll find the Dockerfile
that is used to publish each of those images. You can build your own Docker images in a similar manner.
Tensorflow 2.5.0
Name: valohai/notebook:tensorflow-2.5.0
Dockerfile:
FROM valohai/pypermill
RUN pip install --upgrade pip
RUN pip install seaborn numpy pandas matplotlib valohai-utils statsmodels
RUN pip install tensorflow==2.5.0
Tensorflow 2.5.0-gpu
Name: valohai/notebook:tensorflow-2.5.0-gpu
Dockerfile:
ARG PYTHON_VERSION=3.9
FROM tensorflow/tensorflow:2.5.0-gpu
ARG NBCONVERT_VERSION=5.5.0
ARG SEABORN_VERSION=0.9.0
ARG JUPYTER_VERSION=1.0.0
ARG JUPYTER_CLIENT=6.1.12
ARG PAPERMILL_VERSION=8445fb0d984af248d6946b6672b3e42633f21e51
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
RUN export LANG=C.UTF-8
RUN export LC_ALL=C.UTF-8
# jupyter_client needs to be downgraded to avoid ZMQSocketChannel error
# because our old papermill fork conflicts with latest.
RUN \
pip3 install --no-cache-dir -q nbconvert==$NBCONVERT_VERSION jupyter==$JUPYTER_VERSION seaborn==$SEABORN_VERSION && \
pip3 install --no-cache-dir -q https://github.com/valohai/papermill/archive/$PAPERMILL_VERSION.tar.gz && \
pip3 install --no-cache-dir -q jupyter_client==$JUPYTER_CLIENT
RUN pip3 install numpy pandas matplotlib valohai-utils statsmodels
Tensorflow 1.15.5
Name: valohai/notebook:tensorflow-1.15.5
Dockerfile:
FROM valohai/pypermill
RUN pip install --upgrade pip
RUN pip install seaborn numpy pandas matplotlib valohai-utils statsmodels
RUN pip install tensorflow==1.15.5
sklearn 0.24.2
Name: valohai/notebook:sklearn-0.24.2
Dockerfile:
FROM valohai/pypermill
RUN pip install --upgrade pip
RUN pip install seaborn numpy pandas matplotlib valohai-utils statsmodels
RUN pip install scikit-learn==0.24.2
xgboost 1.4.2
Name: valohai/notebook:xgboost-1.4.2
Dockerfile:
FROM valohai/pypermill
RUN apt-get update && apt-get -y install cmake
RUN pip install --upgrade pip
RUN pip install seaborn numpy pandas matplotlib valohai-utils statsmodels
RUN pip install xgboost==1.4.2
PyTorch 1.10.0 gpu
Name: valohai/notebook:pytorch-1.10.0-gpu
Dockerfile:
ARG PYTHON_VERSION=3.9
FROM pytorch/pytorch:1.10.0-cuda11.3-cudnn8-runtime
ARG NBCONVERT_VERSION=5.5.0
ARG SEABORN_VERSION=0.9.0
ARG JUPYTER_VERSION=1.0.0
ARG JUPYTER_CLIENT=6.1.12
ARG PAPERMILL_VERSION=8445fb0d984af248d6946b6672b3e42633f21e51
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
RUN export LANG=C.UTF-8
RUN export LC_ALL=C.UTF-8
# jupyter_client needs to be downgraded to avoid ZMQSocketChannel error
# because our old papermill fork conflicts with latest.
RUN \
pip3 install --no-cache-dir -q nbconvert==$NBCONVERT_VERSION jupyter==$JUPYTER_VERSION seaborn==$SEABORN_VERSION && \
pip3 install --no-cache-dir -q https://github.com/valohai/papermill/archive/$PAPERMILL_VERSION.tar.gz && \
pip3 install --no-cache-dir -q jupyter_client==$JUPYTER_CLIENT
RUN pip3 install numpy==1.21.5 pandas==1.3.5 torchtext==0.11.2 valohai-utils
Comments
0 comments
Please sign in to leave a comment.