Skip to main content
Trainy Hub is in private preview. Ask your organization owner or Trainy for an invitation and an assigned Hub namespace.
Use this guide if you already submit Konduktor jobs directly to a cluster. With Hub, you submit to one Hub context and namespace. Hub dispatches the job to an eligible worker cluster in your organization. You do not need that worker’s kubeconfig to submit jobs or read their application logs.

​
Sign in and select your workspace

Install or upgrade the CLI in the Python environment you use for jobs:
Install kubectl, the Google Cloud CLI (gcloud), and gke-gcloud-auth-plugin. Then sign in:
Complete the sign-in in your browser. Login discovers the namespaces your account can access. If several are available, choose one at the prompt. The selected namespace determines the organization used for Hub credentials. Successful login prints the Hub context and namespace, makes that context current, and checks that you can submit JobSets. If no namespace is assigned, ask your organization owner or Trainy for access. Save the exact values printed by login. Replace the example values below:
Confirm them before submitting:
The Cluster: line in status identifies your Kubernetes context. Check that Namespace: matches your Hub namespace. It does not identify the job’s destination worker cluster. This guide uses explicit --context values so another tool changing your current context does not redirect your commands. Put this global option before the command. If kubernetes.allowed_contexts in ~/.konduktor/config.yaml pins an old worker context, explicit --context overrides it. Login also warns about that setting. To choose a different assigned namespace later, run login again:
To sign in without changing or creating a Kubernetes context, use konduktor login --no-hub-context.

​
Sign in from a remote VM

Run this on the VM, with its assigned namespace:
Open the printed URL on a computer with a browser and complete the device-code sign-in. Leave the command running on the VM until context setup finishes. Credentials remain on the VM. Current Hub context setup uses the signed-in session for Google Workforce credentials; it does not require a browser on the VM.

​
Check and renew your session

expires describes the cached bearer token, rather than a timer that requires hourly browser login. When a command needs credentials near or after expiry, the CLI uses its refresh token to renew the session without a browser. whoami reads the cache and does not trigger refresh. Token lifetimes depend on the identity provider. If refresh expires or is revoked, run konduktor login again, using --no-browser on a remote VM.

​
List clusters and check readiness

List workers registered in your Hub namespace:
Check REGISTERED, CONNECTED, CONFIGURED, and DISPATCHREADY. A worker ready for dispatch reports True for these conditions and is not CORDONED. Readiness does not guarantee free capacity or immediate admission for your resource request. For condition messages and recent events on a worker, replace the name below with one from the list:
Repeat the namespace-scoped list for each namespace you can access. The Hub console also lets you select an accessible workspace and inspect its clusters. Platform administrators with permission to list workers across namespaces can use:
-A and --all-namespaces go after list. This requests a cluster-wide listing; it is not a shortcut for listing only your account’s memberships. Tenant-scoped accounts may receive a permission error.

​
Adapt your task and submit

Keep your existing task’s image, resource requests, environment, and run script. Use the queue assigned by your organization, normally user-queue. Review any worker-specific node selectors, PVC mounts, host paths, and secret references before you reuse them. A PVC or path on your old cluster is not copied to another worker by Hub. For a first CPU job, save this as hello-hub.yaml:
Submit through the Hub context:
Confirm the launch prompt. --detach-run submits without waiting for log streaming. The CLI prints a generated job ID such as hello-hub-ab12. Save that full ID for later commands:
The Hub context chooses where you submit; it does not pin execution to one worker. Hub dispatches among eligible workers according to queues and resource availability. Ask your organization administrator about placement requirements for workloads that need a specific cluster’s storage or hardware.

​
View jobs and logs

status lists your jobs in the selected Hub namespace. Use status --all-users to include other users’ jobs there. logs follows by default; --no-follow prints available logs and exits. Use --node-rank 1 to read rank 1 of a multi-node job. Hub reads application logs from workers without switching your kubecontext. Open the Hub console, select your workspace, and open a job to inspect its selected worker, scheduling conditions, and admission checks. The console’s Clusters view shows connection and dispatch readiness. Your console sign-in is separate from the CLI session.

​
Make private images and other dependencies available

Create referenced Secrets and ConfigMaps in the Hub job’s namespace, even if they already exist in your old worker cluster’s default namespace. Hub does not import those worker-only objects. This applies to image-pull credentials, container environment references, and Secret or ConfigMap volumes, including projected volumes and init-container references. For a private image, obtain a Docker configuration JSON file with registry authentication entries. A file containing only a local credential-helper setting does not supply credentials to Kubernetes. Create the Hub secret from that file:
For registry-specific credential creation, see private container registry setup. Use the Hub context and namespace for those commands. Merge this into ~/.konduktor/config.yaml and keep your other settings:
Set your task’s resources.image_id to the private image. Before launch, check that each referenced dependency exists in the Hub namespace:
During dependency sync, Hub mirrors referenced Secrets and ConfigMaps into the same namespace on eligible workers before dispatch proceeds. The Hub object is the source; matching secret names on different workers do not prove that their credentials match. Treat this as workload dependency sync, not a continuous registry-secret distribution service. For credential rotation, update the Hub source and coordinate new submissions with your administrator; do not assume an edit updates every existing job or worker copy immediately.

​
Diagnose a queued job or empty logs

Start with the job’s conditions and admission checks in the console. For a CLI inspection, list the Hub Workloads, find the one whose owner is your JobSet, and inspect its details:
  • missing source dependencies: Secret/... or ConfigMap/... means the source object is missing from the Hub namespace. Create it there, then check admission again. Creating it only in a worker’s default namespace does not fix this.
  • An offline worker or a failed mirror can hold dependency sync pending. Check worker readiness and share the condition message with your administrator.
  • A dispatched job with no application logs may still be pulling its image or starting its container. Ask an administrator to inspect the worker pod’s events for ErrImagePull, ImagePullBackOff, or missing configuration. Invalid registry credentials require a valid Hub source secret, not another CLI sign-in.
  • If logs report a missing job in default, check the context and namespace. Use the same explicit Hub context used for launch and the full generated job ID.
  • If the console returns 503, refresh and retry. If it persists, share the time, workspace, and job ID with Trainy. If the console says your session ended, sign in again there.
To continue submitting directly to an existing worker, pass its context explicitly:
That command uses the worker context’s namespace and its existing credentials. See direct cluster authentication and the CLI reference for the remaining commands.