Setup

These steps should be completed after receiving access to a Trainy k8s cluster via a kubeconfig file after which you can inspect your cluster using k8s tooling like kubectl and through the konduktor CLI tool. For instructions on how to get access to your cluster see Authentication.

Installation

We maintain a nightly build of our latest client on PyPi konduktor-nightly

$ pip install konduktor-nightly

(Optional) Setup Cloud Storage Credentials

Cloud storage credential setup for file_mounts is deprecated and will be removed in a future version of Konduktor. New workloads should avoid relying on bucket/credential-backed file sync.

We currently support the following cloud object stores for syncing files —> workloads with more incoming!

Google Cloud Storage gs - Supported ✅
Amazon S3 s3 - Supported ✅
Cloudflare R2 r2 - Coming soon 🚧

Google Cloud Storage Requirements

Install with pip install konduktor-nightly
Check that your GCP user or service account has the minimal GCP permissions.
Ensure that ~/.konduktor/config.yaml is configured correctly for GS:

allowed_clouds:
  - gs

Use local credentials OR set env variables: Coming soon 🚧
Confirm your GS credentials and setup with:

$ konduktor check gs

Amazon S3 Requirements

Install with pip install konduktor-nightly[s3]
Check that your AWS user has the minimal AWS user permissions.
Ensure that ~/.konduktor/config.yaml is configured correctly for S3:

allowed_clouds:
  - s3

Use local ~/.aws/ credentials OR set env variables:

~/.aws/config

[default]
region = us-east-1

~/.aws/credentials

[default]
aws_access_key_id=ASIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

$ export AWS_ACCESS_KEY_ID="ASIAIOSFODNN7EXAMPLE"
$ export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
$ export AWS_DEFAULT_REGION="us-east-1"

Confirm your S3 credentials and setup with:

$ konduktor check s3

(Optional) Setup Private Container Registry Credentials

To have your Trainy jobs use images from private container registries (k8s docs), you must create one or more Kubernetes docker-registry secrets containing your registry credentials. Each secret stores the authentication needed for a specific registry (Docker Hub or GCR). Konduktor will use these secrets when pulling your job images. After creating the secret(s), reference them in your Konduktor config (~/.konduktor/config.yaml) so they are automatically attached to your pods.

kubernetes:
  pod_config:
    spec:
      imagePullSecrets:
        - name: regcred-dockerhub
        - name: regcred-gcr

The names in imagePullSecrets must match the secrets you create below.

Create a Docker Hub Registry Secret

Create your secret

$ kubectl create secret docker-registry regcred-dockerhub \
    --docker-server=https://index.docker.io/v1/ \
    --docker-username=<your-name> \
    --docker-password=<your-pword> \
    --docker-email=<your-email>

Verify that the secret was created

$ kubectl get secrets
NAME                      TYPE                              DATA   AGE
regcred-dockerhub         kubernetes.io/dockerconfigjson    1      67s

Create a Google Container / Artifact Registry Secret

Create a GCP Service Account for registry access

$ gcloud iam service-accounts create gcr-access --display-name "GCR Access"

Grant read permissions for container images

# For GCR (legacy)
$ gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
    --member "serviceAccount:gcr-access@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
    --role "roles/storage.objectViewer"

# For Artifact Registry (recommended)
$ gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
    --member "serviceAccount:gcr-access@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
    --role "roles/artifactregistry.reader"

Generate a JSON key for that service account

$ gcloud iam service-accounts keys create gcr-key.json --iam-account gcr-access@YOUR_PROJECT_ID.iam.gserviceaccount.com

Create your secret

# For GCR (legacy)
$ kubectl create secret docker-registry regcred-gcr \
    --docker-server=gcr.io \
    --docker-username=_json_key \
    --docker-password="$(cat ~/gcr-key.json)" \
    --docker-email=<your-email> # CHANGE THIS LINE

# For Artifact Registry (recommended)
$ kubectl create secret docker-registry regcred-gcr \
    --docker-server=us-central1-docker.pkg.dev \
    --docker-username=_json_key \
    --docker-password="$(cat ~/gcr-key.json)" \
    --docker-email=<your-email> # CHANGE THIS LINE

Verify that the secret was created

$ kubectl get secrets
NAME                TYPE                              DATA   AGE
regcred-gcr         kubernetes.io/dockerconfigjson    1      67s

Get Started

CLI

User Guides

Installation

(Optional) Setup Cloud Storage Credentials

Google Cloud Storage Requirements

Amazon S3 Requirements

(Optional) Setup Private Container Registry Credentials

Create a Docker Hub Registry Secret

Create a Google Container / Artifact Registry Secret

​Installation

​(Optional) Setup Cloud Storage Credentials

​Google Cloud Storage Requirements

​Amazon S3 Requirements

​(Optional) Setup Private Container Registry Credentials

​Create a Docker Hub Registry Secret

​Create a Google Container / Artifact Registry Secret

Installation

(Optional) Setup Cloud Storage Credentials

Google Cloud Storage Requirements

Amazon S3 Requirements

(Optional) Setup Private Container Registry Credentials

Create a Docker Hub Registry Secret

Create a Google Container / Artifact Registry Secret