Using an existing GKE cluster as a Deployment Target

Valohai can push deployments to an existing Kubernetes cluster.

Valohai uses standard Kubernetes APIs to communicate with your Kubernetes cluster and app.valohai.com (34.248.245.191) should be able to access your clusters API Server over HTTPS.

You cluster can be configured to serve only private deployment endpoints.

Install ingress-nginx

• Install ingress-nginx on the cluster

  • https://kubernetes.github.io/ingress-nginx/deploy/

• Get the external IP of your ingress-nginx. You’ll need to share this with Valohai.

  kubectl -n ingress-nginx get service/ingress-nginx-controller

By default all of your Valohai endpoints will be accessible from the public internet. You can patch the nginx ConfigMap to whitelist only certain IPs so only they can access the endpoints served by Valohai.

kubectl patch -n ingress-nginx configmap/ingress-nginx-controller --type merge -p '{"data":{"whitelist-source-range": "84.251.7.123/32,84.251.7.124/32"}}'

Kubernetes Service Account

• Create a Kubernetes service account that Valohai will use

  kubectl create serviceaccount valohai-deployment

• Find the token name (one secret token should be generated automatically). You’ll need to provide this token back to Valohai.

  kubectl get serviceaccounts valohai-deployment -o json

  This will return you the secret name (e.g. valohai-deployment-token-SUFFIX). Use the name to fetch the secret value

  kubectl get secret SECRET-NAME -o json

• Setup the valohai-metadata-role in Kuberenetes. If you want to limit access to specific namespace define it below, otherwise leave it empty.

  • Create a new file valohai-deployment-role.yml with the following contents:

  kind: Role
  apiVersion: rbac.authorization.k8s.io/v1
  metadata:
    name: valohai-deployment-role
    namespace: <IF THERE IS A NAMESPACE>
  rules:
    - apiGroups: [""]
      resources: ["events", "namespaces"]
      verbs: ["get", "list", "watch"]
    - apiGroups: [""]
      resources: ["pods", "services"]
      verbs: ["create", "delete", "deletecollection", "get", "list", "patch", "update", "watch"]
    - apiGroups: ["apps", "extensions"]
      resources: ["deployments", "deployments/rollback", "deployments/scale"]
      verbs: ["create", "delete", "deletecollection", "get", "list", "patch", "update", "watch"]
    - apiGroups: ["extensions"]
      resources: ["ingresses"]
      verbs: ["create", "delete", "deletecollection", "get", "list", "patch", "update", "watch"]
  

  • Apply the role with kubectl apply -f valohai-deployment-role.yml

  • Create a rolebinding. Replace the "default" with your namespace, if you defined one when creating your serviceaccount.

  kubectl create rolebinding valohai-deployment-binding \
      --role=valohai-deployment-role \
      --serviceaccount=default:valohai-deployment
  

Make sure your cluster’s nodes can pull from the repository that Valohai is pushing images to.

Google Cloud Service Account

The valohai-sa-deployments service account is used by Valohai to manage deployments and images in your Container Registry or Artifact Registry.

In your GCP Project go to IAM -> Service Accounts and create a new service account for Valohai:

• Type: Service Account
• Name: valohai-sa-deployments
• Role:

  • Service Account Token Creator

  • Storage Admin

  • Kubernetes Engine Developer

• Create Key: JSON

Download the JSON key, as you’ll need to share it with Valohai later.