Skip to main content
Using an AI coding assistant? Install the Konduktor Skills plugin (npx skills add Trainy-ai/konduktor-skills) to let your AI construct CLI commands for you — run /konduktor-cli to get started.

konduktor status

Shows list of all the jobs with optional filtering and pagination. Usage
Options
  • --all-users, -u
    Show all jobs, including those not owned by the current user.
  • --limit, -l INTEGER
    Maximum number of jobs to display (e.g., —limit 100)
  • --after TEXT
    Show jobs created after this timestamp (e.g., —after “08/06/25 03:54PM”, —after “08/06/25”, —after “03:54PM”)
  • --before TEXT
    Show jobs created before this timestamp (e.g., —before “08/06/25 03:54PM”, —before “08/06/25”, —before “03:54PM”)
Examples
Notes
  • When using —before or —after timestamps, “08/06/25” is equivalent to “08/06/25 00:00”.
  • “03:53PM” is equivalent to “03:53:00PM”.
  • Timestamps shown in “konduktor status” are truncated and are in the local timezone. Example: “03:53:55PM” → “03:53PM” would show up in —after “03:53PM” but not in —before “03:53PM”.

konduktor logs

Retrieve/tail the log of a job. Usage
Arguments
  • JOB_ID
Options
  • --status
    [DEPRECATED] If specified, do not show logs but exit with a status code for the job’s status: 0 for succeeded, or 1 for all other statuses.
  • --follow, --no-follow
    Follow the logs of a job. If —no-follow is specified, print the log so far and exit. (default: —follow)
  • --num-lines, --num_lines, -n INTEGER
    The number of lines to display from the end of the log file. Default is -1 (no limit). (default: -1)
  • --node-rank, --node_rank, -N INTEGER
    The node rank to tail logs from. (default: 0)
  • --start-offset, --start_offset TEXT
    Choose how much time from now to look back in logs. Examples: 30s, 5m, 2h, 1d. Default is 1h. Note: currently only applies when streaming (default —follow). With —no-follow, all available logs are returned. (default: 1h)

konduktor launch

Launch a task. Usage
Arguments
  • ENTRYPOINT...
Options
  • --dryrun
    If True, do not actually run the job.
  • --detach-run, -d
    If True, as soon as a job is submitted, return from this call and do not stream execution logs.
  • --name, -n TEXT
    Task name. Overrides the “name” config in the YAML if both are supplied.
  • --workdir DIRECTORY
    If specified, sync this dir to the remote working directory, where the task will be invoked. Overrides the “workdir” config in the YAML if both are supplied.
  • --cloud TEXT
    The cloud to use. If specified, overrides the “resources.cloud” config. Passing “none” resets the config. [defunct] currently only supports a single cloud
  • --num-nodes INTEGER
    Number of nodes to execute the task on. Overrides the “num_nodes” config in the YAML if both are supplied.
  • --max-restarts INTEGER
    Maximum number of jobset restarts allowed. Overrides YAML.Overrides the “max_restarts” config in the YAML if both are supplied.
  • --completions INTEGER
    Number of successful completions required. Overrides YAML.Overrides the “completions” config in the YAML if both are supplied.
  • --cpus TEXT
    Number of vCPUs each instance must have (e.g., --cpus=4 (exactly 4) or --cpus=4+ (at least 4)). This is used to automatically select the instance type.
  • --memory TEXT
    Amount of memory each instance must have in GB (e.g., --memory=16 (exactly 16GB), --memory=16+ (at least 16GB))
  • --disk-size INTEGER
    OS disk size in GBs.
  • --image-id TEXT
    Custom image id for launching the instances. Passing “none” resets the config.
  • --env-file DOTENV_VALUES
    Path to a dotenv file with environment variables to set on the remote node. If any values from --env-file conflict with values set by --env, the --env value will be preferred.
  • --env _PARSE_ENV_VAR
    \
    Environment variable to set on the remote node. It can be specified multiple times:
    1. --env MY_ENV=1: set $MY_ENV on the cluster to be 1.
    2. --env MY_ENV2=$HOME: set $MY_ENV2 on the cluster to be the
    same value of $HOME in the local environment where the CLI command is run.
    1. --env MY_ENV3: set $MY_ENV3 on the cluster to be the
    same value of $MY_ENV3 in the local environment.
  • --gpus TEXT
    Type and number of GPUs to use. Example values: “V100:8”, “V100” (short for a count of 1) If a new cluster is being launched by this command, this is the resources to provision. If an existing cluster is being reused, this is seen as the task demand, which must fit the cluster’s total resources and is used for scheduling the task. Overrides the “accelerators” config in the YAML if both are supplied. Passing “none” resets the config.
  • --yes, -y
    Skip confirmation prompt.
  • --skip-image-check, -s
    Skip Docker image validation checks for faster startup.
Notes
  • If ENTRYPOINT points to a valid YAML file, it is read in as the task specification. Otherwise, it is interpreted as a bash command.

konduktor down

Tear down job(s). Usage
Arguments
  • JOBS...
Options
  • --all, -a
    Tear down all jobs.
  • --all-users, --all_users
    Include other users for teardown
  • --yes, -y
    Skip confirmation prompt.
Examples
Notes
  • If both JOB and --all are supplied, the latter takes precedence.
  • Tearing down a job will delete all associated containers (all billing stops), and any data on the containers disks will be lost. Accelerators (e.g., GPUs) that are part of the job will be deleted too.
  • Wildcard patterns are supported using * characters. Ex: “test-” matches all jobs starting with “test-”, “-gpu” matches all jobs ending with “-gpu”.

konduktor stop

Suspend job(s) (manual/user-initiated). Usage
Arguments
  • JOBS...
Options
  • --all, -a
    Suspend all jobs.
  • --all-users, --all_users
    Include other users for suspension
  • --yes, -y
    Skip confirmation prompt.
Examples
Notes
  • If both JOB and --all are supplied, the latter takes precedence.
  • Suspending a job will pause execution and mark the job as SUSPENDED (by user). The job can be resumed later with konduktor start.
  • If a job is suspended by the system (e.g., due to queueing), it will show as SUSPENDED (by system).
  • Wildcard patterns are supported using * characters. Ex: “test-” matches all jobs starting with “test-”, “-gpu” matches all jobs ending with “-gpu”.

konduktor start

Resume suspended job(s) (manual/user-initiated). Usage
Arguments
  • JOBS...
Options
  • --all, -a
    Resume all suspended jobs.
  • --all-users, --all_users
    Include other users for resumption
  • --yes, -y
    Skip confirmation prompt.
Examples
Notes
  • If both JOB and --all are supplied, the latter takes precedence.
  • Resuming a job will restart execution from where it was suspended. Only suspended jobs can be resumed.
  • This command works for both manually suspended jobs (SUSPENDED by user) and system-suspended jobs (SUSPENDED by system).
  • Wildcard patterns are supported using * characters. Ex: “test-” matches all jobs starting with “test-”, “-gpu” matches all jobs ending with “-gpu”.

konduktor check

Check which clouds are available to use for storage with Konduktor Usage
Arguments
  • CLOUDS...
Examples
Notes
  • This checks storage credentials for a cloud supported by konduktor. If a cloud is detected to be inaccessible, the reason and correction steps will be shown.
  • If CLOUDS are specified, checks credentials for only those clouds.
  • The enabled clouds are cached and form the “search space” to be considered for each task.

konduktor secret

Manage secrets used in Konduktor. Usage
Examples

konduktor secret create

Create a new secret. Usage
Arguments
  • NAME
Options
  • --kind CHOICE
    Type of secret being created. More kinds coming soon. (default: default)
  • --from-directory, --from_directory DIRECTORY
    Path to a directory to store as a multi-file secret.
  • --from-file, --from_file FILE
    Path to a single file to store as a secret.
  • --inline KEY=VALUE
    Key=value pair to store as an env secret (only valid with —kind env).

konduktor secret delete

Delete a secret by name. Usage
Arguments
  • NAME

konduktor secret list

List secrets in the namespace. Usage
Options
  • --all-users, --all_users, -u
    Show all secrets, including those not owned by the current user.

konduktor serve

Manage deployment serving with Konduktor. Usage
Examples

konduktor serve launch

Launch a deployment to serve. Usage
Arguments
  • ENTRYPOINT...
Options
  • --dryrun
    If True, do not actually run the job.
  • --detach-run, -d
    If True, as soon as a job is submitted, return from this call and do not stream execution logs.
  • --name, -n TEXT
    Task name. Overrides the “name” config in the YAML if both are supplied.
  • --workdir DIRECTORY
    If specified, sync this dir to the remote working directory, where the task will be invoked. Overrides the “workdir” config in the YAML if both are supplied.
  • --cloud TEXT
    The cloud to use. If specified, overrides the “resources.cloud” config. Passing “none” resets the config. [defunct] currently only supports a single cloud
  • --num-nodes INTEGER
    Number of nodes to execute the task on. Overrides the “num_nodes” config in the YAML if both are supplied.
  • --max-restarts INTEGER
    Maximum number of jobset restarts allowed. Overrides YAML.Overrides the “max_restarts” config in the YAML if both are supplied.
  • --completions INTEGER
    Number of successful completions required. Overrides YAML.Overrides the “completions” config in the YAML if both are supplied.
  • --cpus TEXT
    Number of vCPUs each instance must have (e.g., --cpus=4 (exactly 4) or --cpus=4+ (at least 4)). This is used to automatically select the instance type.
  • --memory TEXT
    Amount of memory each instance must have in GB (e.g., --memory=16 (exactly 16GB), --memory=16+ (at least 16GB))
  • --disk-size INTEGER
    OS disk size in GBs.
  • --image-id TEXT
    Custom image id for launching the instances. Passing “none” resets the config.
  • --env-file DOTENV_VALUES
    Path to a dotenv file with environment variables to set on the remote node. If any values from --env-file conflict with values set by --env, the --env value will be preferred.
  • --env _PARSE_ENV_VAR
    \
    Environment variable to set on the remote node. It can be specified multiple times:
    1. --env MY_ENV=1: set $MY_ENV on the cluster to be 1.
    2. --env MY_ENV2=$HOME: set $MY_ENV2 on the cluster to be the
    same value of $HOME in the local environment where the CLI command is run.
    1. --env MY_ENV3: set $MY_ENV3 on the cluster to be the
    same value of $MY_ENV3 in the local environment.
  • --gpus TEXT
    Type and number of GPUs to use. Example values: “V100:8”, “V100” (short for a count of 1) If a new cluster is being launched by this command, this is the resources to provision. If an existing cluster is being reused, this is seen as the task demand, which must fit the cluster’s total resources and is used for scheduling the task. Overrides the “accelerators” config in the YAML if both are supplied. Passing “none” resets the config.
  • --min-replicas INTEGER
    Minimum number of replicas to run for the service. Overrides the “min_replicas” field in the YAML if both are supplied.
  • --max-replicas INTEGER
    Maximum number of replicas to allow for the service. Overrides the “max_replicas” field in the YAML if both are supplied.
  • --ports INTEGER
    The container port on which your service will listen for HTTP traffic. Overrides the “ports” field in the YAML if both are supplied.
  • --probe TEXT
    The HTTP path to use for health checks (liveness, readiness, and startup probes). Overrides the “probe” field in the YAML if both are supplied. The service should respond with HTTP 200 on this path when healthy.
  • --yes, -y
    Skip confirmation prompt.
  • --skip-image-check, -s
    Skip Docker image validation checks for faster startup.
Notes
  • If ENTRYPOINT points to a valid YAML file, it is read in as the task specification. Otherwise, it is interpreted as a bash command.

konduktor serve down

Tear down deployments (Deployment, Service, PodAutoscaler). Usage
Arguments
  • NAMES...
Options
  • --all, -a
    Tear down all deployments.
  • --yes, -y
    Skip confirmation prompt.
Examples

konduktor serve status

Show status of deployments launched via konduktor serve launch. Usage
Options
  • --all-users, -u
    Show all deployments, including those not owned by the current user.
  • --direct, -d
    Force display of direct IP endpoints instead of trainy.us endpoints.