Run Lifecycle

A run starts with pluto.init() and ends with either run.finish() (mark it complete on the server) or run.close() (release local resources without changing the run’s server-side state). This page covers the parameters for each, plus how to resume a finished run.

Starting a Run

import pluto

run = pluto.init(
    project="my-project",
    name="experiment-1",
    config={"lr": 1e-3, "batch_size": 32},
    tags=["baseline"],
)

Common parameters:

Parameter	Type	Description
`project`	`str`	Project name (auto-created if it doesn’t exist).
`name`	`str`	Human-readable run name. Pluto also auto-generates a sequential display ID like `MMP-42`.
`config`	`dict`	Hyperparameters. Snapshot at init; for in-run edits use `run.update_config()`.
`tags`	`list[str]`	Initial tag set. Edit later with `run.add_tags()` / `run.remove_tags()`.
`run_id`	`int \| str`	Resume an existing run by numeric ID or display ID. Used with `resume=True`.
`resume`	`bool`	If `True`, re-open a previously finished run instead of creating a new one.
`fork_run_id`	`int \| str`	Fork from a parent run. See Run Forking.
`fork_step`	`int`	Required with `fork_run_id`. Step number to fork at.
`inherit_config`	`bool`	When forking, deep-merge the parent’s config into the child’s. Defaults to `True`.
`inherit_tags`	`bool`	When forking, copy the parent’s tags. Defaults to `False`.

Resuming a Finished Run

run = pluto.init(project="my-project", run_id="MMP-42", resume=True)
run.log({"eval/accuracy": 0.94}, step=10000)
run.finish()

Both numeric IDs (12345) and display IDs ("MMP-42") are accepted.

Ending a Run

The SDK exposes two teardown methods:

Method	Local cleanup	Marks the run on the server	Use when
`run.finish()`	Yes	Yes — transitions to `COMPLETED` (or `FAILED` on uncaught exception)	The training loop is done and you want the run to leave the active state.
`run.close()`	Yes	No — server status is unchanged	A short-lived process attached to an ongoing run and shouldn’t end it.

Both stop the monitor thread, drain the sync queue, and close HTTP clients. run.finish() also fires automatically on interpreter shutdown via atexit. run.close() unregisters the atexit hook, so a process that calls close() won’t accidentally complete the run when it exits.

When to Use `close()` Instead of `finish()`

close() supports multi-process workflows where one process attaches to an active run and shouldn’t take it down on exit. Two common cases:

Eval job appending to a live training run. The training loop holds the “real” finish(). The eval process opens the same run with resume=True, writes metrics, and close()s — leaving the run active.
Side process uploading artifacts while the main run is still producing data elsewhere.

# In a side process attaching to a still-running training job
run = pluto.init(project="my-project", run_id="MMP-42", resume=True)
run.log({"eval/loss": 0.32}, step=current_train_step)
run.close()  # local teardown only, run stays active on the server

close() is idempotent and thread-safe; subsequent finish() calls on a closed run are no-ops.

Run States

A run is always in exactly one of five states:

State	Meaning
`RUNNING`	The run is active and reporting.
`COMPLETED`	The run finished successfully — normally via `run.finish()`.
`FAILED`	The run raised an uncaught exception, or stopped reporting for long enough that Pluto’s stale-run monitor gave up on it.
`CANCELLED`	A configured metric threshold was crossed and the run was cancelled.
`TERMINATED`	The run was preempted — its process received a `SIGTERM` (e.g. a spot instance was reclaimed, or a Kubernetes pod was evicted). The SDK reports this the moment it’s signalled, so preempted runs don’t linger as `RUNNING`.

So the three ways a training run can die each land on a different state:

Crash — an unhandled exception → FAILED.
Preemption / eviction — the process is sent a SIGTERM (reclaimed spot, evicted pod) → TERMINATED.
Hard kill — SIGKILL, an OOM, or the node disappearing, where the run can’t report anything → FAILED, once Pluto’s monitor notices it went silent.

Which status wins

When a run ends, the highest-priority state wins — a lower-priority update can’t replace it afterward:

FAILED  >  CANCELLED  >  TERMINATED  >  COMPLETED

This matters most for multi-node training, where several workers report on the same run. If one worker fails and another later reports success, the run stays Failed — a success arriving afterward can’t paper over the failure. The one way a finished run does change state is if you deliberately resume it — that re-opens the run as RUNNING, and whatever happens next can move it on from there.

Status History

Every run’s Summary page has a Status History card that shows each status the run passed through, in order, along with when each change happened and what caused it. A healthy run’s history is short — it started, then finished:

Each change also carries a small source tag showing what triggered it:

Source	What it means
`api`	A normal update from your code — for example, `run.finish()`.
`resume`	The run was picked back up with `resume=True`.
`stale`	Pluto’s monitor marked the run failed because it stopped reporting.
`threshold`	A metric crossed a threshold you set, cancelling the run.

When a run fails, the history tells you why — which is the whole point. In the runs table, a run that failed because it went quiet (Pluto’s monitor gave up on it) looks identical to one that crashed with an error — both just say Failed. Here you can see it was the quiet kind: it hadn’t reported in longer than the allowed window, so Pluto marked it failed.

Status History of a run failed by the stale monitor, with the details showing it stopped reporting

Runs that were resumed show their full arc too, including whatever happened next — like this one, which was picked back up and then hit an out-of-memory error:

Get Started

Core

API Reference

Integrations

Migrate

Changelog

Starting a Run

Resuming a Finished Run

Ending a Run

When to Use `close()` Instead of `finish()`

Run States

Which status wins

Status History

​Starting a Run

​Resuming a Finished Run

​Ending a Run

​When to Use close() Instead of finish()

​Run States

​Which status wins

​Status History

Starting a Run

Resuming a Finished Run

Ending a Run

When to Use `close()` Instead of `finish()`

Run States

Which status wins

Status History