Setup

For Trainy Hub, follow Submit jobs through Trainy Hub. Hub is in private preview and requires an invitation and an assigned namespace. For direct cluster access, obtain a kubeconfig using Authentication. You can then inspect the cluster with kubectl and the konduktor CLI.

Installation

We maintain a nightly build of our latest client on PyPi konduktor-nightly

$ pip install konduktor-nightly

(Optional) Setup Cloud Storage Credentials

Cloud storage credential setup for file_mounts is deprecated and will be removed in a future version of Konduktor. New workloads should avoid relying on bucket/credential-backed file sync.

We currently support the following cloud object stores for syncing files —> workloads with more incoming!

Google Cloud Storage gs - Supported ✅
Amazon S3 s3 - Supported ✅
Cloudflare R2 r2 - Coming soon 🚧

For teams: by default each machine auto-creates its own konduktor-filemounts-<username>-<user_hash> bucket. Admins can instead point every engineer at a single shared bucket by adding the following to each machine’s ~/.konduktor/config.yaml (or a shared KONDUKTOR_CONFIG):

jobs:
  bucket: gs://konduktor-filemounts

Each machine then uploads under its own folder inside that bucket. See Shared bucket for details.

Google Cloud Storage Requirements

Install with pip install konduktor-nightly
Check that your GCP user or service account has the minimal GCP permissions.
Ensure that ~/.konduktor/config.yaml is configured correctly for GS:

allowed_clouds:
  - gs

Use local credentials OR set env variables: Coming soon 🚧
Confirm your GS credentials and setup with:

$ konduktor check gs

Amazon S3 Requirements

Install with pip install konduktor-nightly[s3]
Check that your AWS user has the minimal AWS user permissions.
Ensure that ~/.konduktor/config.yaml is configured correctly for S3:

allowed_clouds:
  - s3

Use local ~/.aws/ credentials OR set env variables:

~/.aws/config

[default]
region = us-east-1

~/.aws/credentials

[default]
aws_access_key_id=ASIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

$ export AWS_ACCESS_KEY_ID="ASIAIOSFODNN7EXAMPLE"
$ export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
$ export AWS_DEFAULT_REGION="us-east-1"

Confirm your S3 credentials and setup with:

$ konduktor check s3

(Optional) Setup Private Container Registry Credentials

For Hub jobs, create these secrets in the Hub context and job namespace. Secrets in a worker’s default namespace are not imported automatically. See Hub dependencies.

To have your Trainy jobs use images from private container registries (k8s docs), you must create one or more Kubernetes docker-registry secrets containing your registry credentials. Each secret stores the authentication needed for a specific registry (Docker Hub or GCR). Konduktor will use these secrets when pulling your job images. After creating the secret(s), reference them in your Konduktor config (~/.konduktor/config.yaml) so they are automatically attached to your pods.

kubernetes:
  pod_config:
    spec:
      imagePullSecrets:
        - name: regcred-dockerhub
        - name: regcred-gcr

The names in imagePullSecrets must match the secrets you create below. For Hub submission, create referenced Secrets and ConfigMaps in your Hub tenant namespace. Dependencies that exist only on a worker are not imported into the Hub. On a context created by konduktor login, launch checks required references in every rendered JobSet PodTemplate before submitting the job. This covers image-pull secrets, regular and init-container environment references, and Secret/ConfigMap volumes, including projected sources. References marked optional: true may be absent; image-pull secrets are always required. A missing dependency produces an error naming its kind, name, Hub namespace, and context, without creating a queued job. The check reads only metadata; referenced keys remain subject to Hub admission validation. A dependency can still disappear after the check. Direct-cluster submission keeps its existing behavior.

Create a Docker Hub Registry Secret

Create your secret

$ kubectl create secret docker-registry regcred-dockerhub \
    --docker-server=https://index.docker.io/v1/ \
    --docker-username=<your-name> \
    --docker-password=<your-pword> \
    --docker-email=<your-email>

Verify that the secret was created

$ kubectl get secrets
NAME                      TYPE                              DATA   AGE
regcred-dockerhub         kubernetes.io/dockerconfigjson    1      67s

Create a Google Container / Artifact Registry Secret

Create a GCP Service Account for registry access

$ gcloud iam service-accounts create gcr-access --display-name "GCR Access"

Grant read permissions for container images

# For GCR (legacy)
$ gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
    --member "serviceAccount:gcr-access@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
    --role "roles/storage.objectViewer"

# For Artifact Registry (recommended)
$ gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
    --member "serviceAccount:gcr-access@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
    --role "roles/artifactregistry.reader"

Generate a JSON key for that service account

$ gcloud iam service-accounts keys create gcr-key.json --iam-account gcr-access@YOUR_PROJECT_ID.iam.gserviceaccount.com

Create your secret

# For GCR (legacy)
$ kubectl create secret docker-registry regcred-gcr \
    --docker-server=gcr.io \
    --docker-username=_json_key \
    --docker-password="$(cat ~/gcr-key.json)" \
    --docker-email=<your-email> # CHANGE THIS LINE

# For Artifact Registry (recommended)
$ kubectl create secret docker-registry regcred-gcr \
    --docker-server=us-central1-docker.pkg.dev \
    --docker-username=_json_key \
    --docker-password="$(cat ~/gcr-key.json)" \
    --docker-email=<your-email> # CHANGE THIS LINE

Verify that the secret was created

$ kubectl get secrets
NAME                TYPE                              DATA   AGE
regcred-gcr         kubernetes.io/dockerconfigjson    1      67s

Get Started

CLI

User Guides

Cluster Operators

Installation

(Optional) Setup Cloud Storage Credentials

Google Cloud Storage Requirements

Amazon S3 Requirements

(Optional) Setup Private Container Registry Credentials

Create a Docker Hub Registry Secret

Create a Google Container / Artifact Registry Secret

​Installation

​(Optional) Setup Cloud Storage Credentials

​Google Cloud Storage Requirements

​Amazon S3 Requirements

​(Optional) Setup Private Container Registry Credentials

​Create a Docker Hub Registry Secret

​Create a Google Container / Artifact Registry Secret

Installation

(Optional) Setup Cloud Storage Credentials

Google Cloud Storage Requirements

Amazon S3 Requirements

(Optional) Setup Private Container Registry Credentials

Create a Docker Hub Registry Secret

Create a Google Container / Artifact Registry Secret