Exporting Weights & Biases Runs

pluto migrate wandb is in public preview. Its interface and behavior may change as we iterate.

pluto migrate wandb imports historical Weights & Biases runs — runs that already finished — into Pluto.

Which tool do I want?

Runs you're still training

Use the compatibility shim. It dual-logs every wandb.log() call to Pluto as the run happens, with no code changes.

Runs that already finished

Use pluto migrate wandb, described on this page. It reads finished runs out of the W&B cloud API and replays them into Pluto.

The two are complementary: point the shim at your ongoing work, then backfill your history with the importer.

Install

pip install 'pluto-ml[migrate]'

You need credentials for both sides:

export WANDB_API_KEY=...     # or run `wandb login`
export PLUTO_API_KEY=...     # or run `pluto login`

How it works

W&B cloud  ──export──▶  staging directory  ──load──▶  Pluto
                        (parquet + run.json
                         + media files)

Export downloads each run to disk. Load replays the staged data into Pluto through the normal client API. Keeping them separate means you can inspect exactly what was pulled before anything is written to Pluto, re-run the load without hitting the W&B API again, and resume either phase after an interruption.

Quick start

Import one project, end to end:

# --entity    wandb entity — your team or username (required)
# --project   which wandb project; omit it to take them all
# --output    where runs are staged on disk (required)
# --workers   projects at once — budget 2-4 GB of RAM each
# --cleanup   delete each run's staged files once it has loaded
pluto migrate wandb all --entity my-team --project my-project --output ./staging --workers 4 --cleanup

Or run the phases separately, which gives you a chance to inspect what was pulled before anything reaches Pluto:

# 1. export — download the runs from W&B to ./staging
pluto migrate wandb export --entity my-team --project my-project --output ./staging

# 2. check what would load, without writing anything
pluto migrate wandb load --input ./staging --dry-run

# 3. load — replay what you staged into Pluto
pluto migrate wandb load --input ./staging --cleanup

Start with one --project before pointing it at a whole entity.

Commands

`export`

Downloads runs from W&B into a staging directory.

Flag	Description
`--entity`	W&B entity (team or user). Required.
`--output`	Directory to stage exported data in. Required.
`--project`	W&B project to migrate. Repeatable. Omit to export all projects under the entity.
`--exclude`	Project to skip when exporting all projects. Repeatable.
`--run-id`	Only these W&B run ids. Repeatable.
`--after` / `--before`	Only runs whose W&B creation date falls after / before an ISO-8601 date — `2024-01-01` or `2024-01-01T00:00:00Z`. Naive dates are read as UTC.
`--wandb-api-key`	W&B API key. Defaults to `WANDB_API_KEY` or `wandb login`.
`--workers`	Parallelism, default `4`. Projects exported concurrently, and the per-run file-download budget. Each concurrent project holds its run history and upload queue in memory (~2–4 GB per worker), so raise it only if the host has the RAM.
`--no-files`	Skip media and file downloads. Also drops custom charts — see below.
`--no-console`	Skip console `output.log`.
`--no-system-metrics`	Skip GPU/CPU system metrics.
`--no-artifacts`	Skip logged artifacts.
`--artifact-max-size-mb`	Skip artifacts larger than N MB. No cap by default.
`--strict`	Exit non-zero if any run had data that couldn’t be migrated.

--after / --before filter on W&B’s created_at, and the filtering happens locally after listing the project’s runs — so a narrow window doesn’t make the listing itself faster. A date it can’t parse fails the command rather than being ignored, and a run with no usable created_at is excluded from a date-filtered export with a warning.

`load`

Replays a staging directory into Pluto.

Flag	Description
`--input`	Export directory to load from. Required.
`--project` / `--exclude`	Limit which staged projects are loaded. Repeatable.
`--run-id`	Only these W&B run ids. Repeatable.
`--dest-project`	Pluto project to load into. Defaults to the W&B project name. Only valid with a single `--project` — otherwise the command exits with an error.
`--dry-run`	Print what would be loaded without creating runs.
`--force-resume`	Re-load runs already marked loaded. May duplicate media files.
`--flush-every`	Metric batch size, and how often non-metric rows check backpressure. Default `500`.
`--max-pending`	Max queued records before the loader throttles. Default `5000`.
`--cleanup`	Delete each run’s staged files once it’s confirmed loaded.
`--workers`	Projects loaded concurrently, default `4`.

`all`

Runs both phases together, and takes the export and load flags at once, minus --input. all is usually faster than running the two commands back to back, because it overlaps them. Export runs in the background while a loader repeatedly sweeps the staging directory, picking up each run the moment it’s fully staged — so uploads to Pluto are already going while later runs are still downloading from W&B. That matters because the two phases are bound by different things: export waits on the W&B API, load waits on uploads to Pluto. Run sequentially, the wall clock is the sum of both. Overlapped, it’s closer to the slower of the two.

Pair it with --cleanup on a large migration. Each run’s staged files are deleted once it’s confirmed loaded, so peak disk stays near the size of a single run instead of the whole entity.

--dry-run is not accepted on all. Run export first, then load --dry-run.

Exit codes

Code	Meaning
`0`	Everything migrated cleanly
`1`	Some runs failed; the rest completed
`2`	Usage error, a crashed worker, a missing `migrate` extra, or a `--strict` violation

The staging directory

Each run is staged under {output}/{entity}/{project}/runs/{run_id}/:

File	Contents
`run.json`	Run manifest — name, config, tags, summary, state, original timestamps.
`part-00000.parquet`, …	Metrics, string series, system metrics, media references, console lines and artifact rows in one long schema. Parts rotate at 50 MB.
`files/`	Downloaded media and artifact files.
`_export_complete.json`	Sentinel, written last. Its presence means the run staged completely.

Runs are staged atomically — written to a temporary directory and renamed into place — so an interrupted export re-does at most one run.

Re-running export skips any run that already has its sentinel, regardless of which flags you pass. If you first exported with --no-artifacts and want the artifacts, delete those run directories — changing the flag alone won’t re-export them.

The load phase records progress in a ledger beside the export: loaded_runs.json for a single project, or loaded_runs.{project}.json when two or more projects are in scope. A load over many projects and a load over one therefore keep separate resume state.

Timestamps

The importer replays each point at its original wall-clock time rather than stamping it with the import time. Run createdAt comes from W&B’s created_at, and the historical finish time is written to statusUpdated, which is what drives Pluto’s Duration.

Pluto copies whatever dates W&B recorded — it can’t recover a date W&B doesn’t have. If a run was trained offline and synced to W&B days later, W&B stamped it with the sync date, so that’s the date your migrated run gets too.

Resuming and re-running

Every run carries the external id wandb::{entity}/{project}/{run_id}, which is how the loader recognizes a run it has already imported. Re-running load is safe. Finished runs are skipped, and a run that was created but not fully replayed — for example if the loader was killed mid-run — is resumed and completed.

Loading into a destination that was already migrated briefly reopens those runs server-side before restoring their terminal status. The end state is correct, but a run can appear as RUNNING for a moment.

Duplicates only ever affect media and files, never scalar metrics. Metrics carry their original timestamps, so replaying them replaces rather than duplicates. Use --force-resume only when you deliberately want to re-load a run that’s already marked loaded.

Large migrations

--workers controls project concurrency and, on export, the per-run file-download budget. Budget ~2–4 GB of RAM per worker.
--flush-every and --max-pending bound how much the loader queues before it throttles.
--cleanup deletes each run’s staged files as soon as it’s confirmed loaded, so peak disk stays close to the largest single run rather than the whole entity.
Set disable_system_metrics so the import machine’s own hardware stats aren’t recorded into the runs you’re importing:
import pluto pluto.settings(disable_system_metrics=True)
The runs still receive the system metrics captured on the original W&B host.

What migrates

What	Notes
Scalar metrics	Original step and timestamp; booleans as `1.0`/`0.0`; `NaN` and `±Inf` preserved
String metrics	Non-numeric series become a string metric staircase
Histograms	Pre-binned, with real bin edges reconstructed from W&B’s packed bins
System metrics	`system.` → `sys/`, unless `--no-system-metrics`
Console logs	stdout/stderr from `output.log`, unless `--no-console`
Images	Single and list-logged, order preserved, unless `--no-files`
Image annotations	Bounding boxes and segmentation masks — see Annotations
Audio & video	Including list-logged, unless `--no-files`
Tables	Plain `wandb.Table` → `pluto.Table`
Custom charts	`wandb.plot.*` panels, re-rendered as Vega
Sweep membership	Runs keep their sweep grouping as a `sweep:<id>` tag, alongside the sweep’s recorded config
Run metadata	Name, tags (plus an `import:wandb` tag), state, config and summary
Run environment	Git commit/branch, OS, Python version, CLI args, GPU inventory
Artifact files	Files of logged artifacts, unless `--no-artifacts` and subject to `--artifact-max-size-mb`

What a migrated run looks like

Migrated content renders with some limits worth knowing before you go looking for it.

Content	Where it appears	Limits
Custom charts	All-runs and individual-run Charts tabs	Up to 25 runs overlaid per chart; only the newest step of the backing table; cannot be added to a dashboard, reordered, hidden, or saved in a Charts layout
Plotly / matplotlib	Charts view, run page, Files tab	JSON over 8 MB renders as raw text instead of a chart
3D point clouds	Same	Row widths 3, 4 and 6 only; over 200,000 points are decimated
HTML	Same	Sandboxed, so external stylesheets and scripts don’t load
Tables	Charts view, run page, Files tab	Newest step only — there is no step slider; no dashboard widget
String metrics	Everywhere, including dashboards	Sampled to 500 steps

Logs named like W&B’s per-run artifacts — run-xxxxxxxx-name:v0 — are hidden from the metrics views by design. Find them on the run’s Files tab.

What does not migrate

Hand-authored Vega charts and wandb.plot_table

W&B never exports the spec body for a custom Vega panel, so there’s nothing to render. The chart’s backing table still imports — the data survives, the visualization doesn’t. Flagged as custom-chart-unsupported.

Bokeh, and 3D meshes and molecules

Bokeh has no renderer. Object3D imports only as a point cloud — mesh and molecule formats (.obj, .glb, .pdb) aren’t handled.

Rich tables

joined-table and partitioned-table are skipped. Plain tables import; a table with media in its cells imports as data, with the media cells rendered as text.

Artifact lineage and versioning

Only the files of a run’s logged artifacts are re-uploaded. Versions, aliases, the type/lineage graph and input (used) artifacts are dropped.

Reports and saved workspace views

W&B Reports, saved panel layouts and custom dashboards.

Model Registry

Registered models and their lineage.

Internal summary keys

_-prefixed summary fields such as _wandb are stripped.

Silently dropped

Most gaps are reported in the coverage flags below. These are not — nothing warns you, and --strict won’t catch them:

Custom charts, when you pass --no-files. The panel specs live in the run’s config.yaml, which --no-files never downloads. Every wandb.plot.* chart disappears with no flag raised.
History keys beginning with _ — _runtime, _wandb and similar.
History rows missing _step or _timestamp, and None values.
System metrics not prefixed system. in W&B’s events stream.

A few things migrate, but with less fidelity than you might expect:

W&B notes don’t become Pluto notes. They’re preserved under the run’s config, alongside the original URL, state and summary.
Console output loses its stream. Every line is recorded at INFO, so stdout and stderr are no longer distinguishable.
Histograms with no bin edges get synthetic ones (0..N). Counts are exact; the x-axis is generic.
String values over 200 characters are dropped — string metrics are short state labels, not free-form text. Flagged as string-series-too-long.

Coverage flags

The exporter prints a flag for each dropped item, per run and as an end-of-project total. --strict exits non-zero if any flag was raised.

Dropped item	Flag
Hand-authored Vega, or an unresolvable backing table	`custom-chart-unsupported`
String value over 200 characters	`string-series-too-long`
Media skipped via `--no-files`	`media-file(--no-files)`
Unknown media type	`unsupported(<type>)`
Non-scalar history value	`unsupported(<python type>)`
Failed downloads	`file-download-failed` / `artifact-download-failed`
Artifact over `--artifact-max-size-mb`	`artifact-over-size-cap`
Table columns containing media	`table-media-cell`
Input-artifact lineage	`artifact-input-lineage`
Artifact versions and non-latest aliases	`artifact-versioning`

--strict fires on any flag, including ones you caused deliberately. Exporting with --no-files raises media-file(--no-files) for every skipped item, so --strict will fail that run.

Get Started

Core

API Reference

Integrations

Migrate

Changelog

Exporting Weights & Biases Runs

Which tool do I want?

Runs you're still training

Runs that already finished

Install

How it works

Quick start

Commands

`export`

`load`

`all`

Exit codes

The staging directory

Timestamps

Resuming and re-running

Large migrations

What migrates

What a migrated run looks like

What does not migrate

Silently dropped

Coverage flags

​Which tool do I want?

Runs you're still training

Runs that already finished

​Install

​How it works

​Quick start

​Commands

​export

​load

​all

​Exit codes

​The staging directory

​Timestamps

​Resuming and re-running

​Large migrations

​What migrates

​What a migrated run looks like

​What does not migrate

​Silently dropped

​Coverage flags

Which tool do I want?

Install

How it works

Quick start

Commands

`export`

`load`

`all`

Exit codes

The staging directory

Timestamps

Resuming and re-running

Large migrations

What migrates

What a migrated run looks like

What does not migrate

Silently dropped

Coverage flags