> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trainy.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Exporting Weights & Biases Runs
> Export existing Weights & Biases runs to Pluto using pluto migrate wandb
`pluto migrate wandb` is in **public preview**. Its interface and behavior may change as we iterate.
`pluto migrate wandb` imports **historical** Weights & Biases runs — runs that already finished — into Pluto.
## Which tool do I want?
Use the **compatibility shim**. It dual-logs every `wandb.log()` call to Pluto as the run happens, with no code changes.
Use **`pluto migrate wandb`**, described on this page. It reads finished runs out of the W\&B cloud API and replays them into Pluto.
The two are complementary: point the shim at your ongoing work, then backfill your history with the importer.
## Install
```bash theme={null}
pip install 'pluto-ml[migrate]'
```
You need credentials for both sides:
```bash theme={null}
export WANDB_API_KEY=... # or run `wandb login`
export PLUTO_API_KEY=... # or run `pluto login`
```
## How it works
```
W&B cloud ──export──▶ staging directory ──load──▶ Pluto
(parquet + run.json
+ media files)
```
**Export** downloads each run to disk. **Load** replays the staged data into Pluto through the normal client API.
Keeping them separate means you can inspect exactly what was pulled before anything is written to Pluto, re-run the load without hitting the W\&B API again, and resume either phase after an interruption.
## Quick start
Import one project, end to end:
```bash theme={null}
# --entity wandb entity — your team or username (required)
# --project which wandb project; omit it to take them all
# --output where runs are staged on disk (required)
# --workers projects at once — budget 2-4 GB of RAM each
# --cleanup delete each run's staged files once it has loaded
pluto migrate wandb all --entity my-team --project my-project --output ./staging --workers 4 --cleanup
```
Or run the phases separately, which gives you a chance to inspect what was pulled before anything reaches Pluto:
```bash theme={null}
# 1. export — download the runs from W&B to ./staging
pluto migrate wandb export --entity my-team --project my-project --output ./staging
# 2. check what would load, without writing anything
pluto migrate wandb load --input ./staging --dry-run
# 3. load — replay what you staged into Pluto
pluto migrate wandb load --input ./staging --cleanup
```
Start with one `--project` before pointing it at a whole entity.
## Commands
### `export`
Downloads runs from W\&B into a staging directory.
| Flag | Description |
| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--entity` | W\&B entity (team or user). **Required.** |
| `--output` | Directory to stage exported data in. **Required.** |
| `--project` | W\&B project to migrate. Repeatable. Omit to export **all** projects under the entity. |
| `--exclude` | Project to skip when exporting all projects. Repeatable. |
| `--run-id` | Only these W\&B run ids. Repeatable. |
| `--after` / `--before` | Only runs whose W\&B creation date falls after / before an ISO-8601 date — `2024-01-01` or `2024-01-01T00:00:00Z`. Naive dates are read as UTC. |
| `--wandb-api-key` | W\&B API key. Defaults to `WANDB_API_KEY` or `wandb login`. |
| `--workers` | Parallelism, default `4`. Projects exported concurrently, and the per-run file-download budget. Each concurrent project holds its run history and upload queue in memory (\~2–4 GB per worker), so raise it only if the host has the RAM. |
| `--no-files` | Skip media and file downloads. **Also drops custom charts** — see below. |
| `--no-console` | Skip console `output.log`. |
| `--no-system-metrics` | Skip GPU/CPU system metrics. |
| `--no-artifacts` | Skip logged artifacts. |
| `--artifact-max-size-mb` | Skip artifacts larger than N MB. No cap by default. |
| `--strict` | Exit non-zero if any run had data that couldn't be migrated. |
`--after` / `--before` filter on W\&B's `created_at`, and the filtering happens locally after listing the project's runs — so a narrow window doesn't make the listing itself faster. A date it can't parse fails the command rather than being ignored, and a run with no usable `created_at` is excluded from a date-filtered export with a warning.
### `load`
Replays a staging directory into Pluto.
| Flag | Description |
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--input` | Export directory to load from. **Required.** |
| `--project` / `--exclude` | Limit which staged projects are loaded. Repeatable. |
| `--run-id` | Only these W\&B run ids. Repeatable. |
| `--dest-project` | Pluto project to load into. Defaults to the W\&B project name. **Only valid with a single `--project`** — otherwise the command exits with an error. |
| `--dry-run` | Print what would be loaded without creating runs. |
| `--force-resume` | Re-load runs already marked loaded. May duplicate media files. |
| `--flush-every` | Metric batch size, and how often non-metric rows check backpressure. Default `500`. |
| `--max-pending` | Max queued records before the loader throttles. Default `5000`. |
| `--cleanup` | Delete each run's staged files once it's confirmed loaded. |
| `--workers` | Projects loaded concurrently, default `4`. |
### `all`
Runs both phases together, and takes the export and load flags at once, minus `--input`.
`all` is usually faster than running the two commands back to back, because it **overlaps them**. Export runs in the background while a loader repeatedly sweeps the staging directory, picking up each run the moment it's fully staged — so uploads to Pluto are already going while later runs are still downloading from W\&B.
That matters because the two phases are bound by different things: export waits on the W\&B API, load waits on uploads to Pluto. Run sequentially, the wall clock is the sum of both. Overlapped, it's closer to the slower of the two.
Pair it with `--cleanup` on a large migration. Each run's staged files are deleted once it's confirmed loaded, so peak disk stays near the size of a single run instead of the whole entity.
`--dry-run` is **not** accepted on `all`. Run `export` first, then `load --dry-run`.
### Exit codes
| Code | Meaning |
| ---- | ----------------------------------------------------------------------------------- |
| `0` | Everything migrated cleanly |
| `1` | Some runs failed; the rest completed |
| `2` | Usage error, a crashed worker, a missing `migrate` extra, or a `--strict` violation |
## The staging directory
Each run is staged under `{output}/{entity}/{project}/runs/{run_id}/`:
| File | Contents |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `run.json` | Run manifest — name, config, tags, summary, state, original timestamps. |
| `part-00000.parquet`, … | Metrics, string series, system metrics, media references, console lines and artifact rows in one long schema. Parts rotate at 50 MB. |
| `files/` | Downloaded media and artifact files. |
| `_export_complete.json` | Sentinel, written **last**. Its presence means the run staged completely. |
Runs are staged atomically — written to a temporary directory and renamed into place — so an interrupted export re-does at most one run.
Re-running `export` skips any run that already has its sentinel, **regardless of which flags you pass**. If you first exported with `--no-artifacts` and want the artifacts, delete those run directories — changing the flag alone won't re-export them.
The load phase records progress in a ledger beside the export: `loaded_runs.json` for a single project, or `loaded_runs.{project}.json` when two or more projects are in scope. A load over many projects and a load over one therefore keep **separate** resume state.
## Timestamps
The importer replays each point at its original wall-clock time rather than stamping it with the import time. Run `createdAt` comes from W\&B's `created_at`, and the historical finish time is written to `statusUpdated`, which is what drives Pluto's [Duration](/pluto/run-lifecycle#imported-runs).
Pluto copies whatever dates W\&B recorded — it can't recover a date W\&B doesn't have. If a run was trained offline and synced to W\&B days later, W\&B stamped it with the **sync** date, so that's the date your migrated run gets too.
## Resuming and re-running
Every run carries the external id `wandb::{entity}/{project}/{run_id}`, which is how the loader recognizes a run it has already imported.
Re-running `load` is safe. Finished runs are skipped, and a run that was created but not fully replayed — for example if the loader was killed mid-run — is resumed and completed.
Loading into a destination that was already migrated **briefly reopens** those runs server-side before restoring their terminal status. The end state is correct, but a run can appear as `RUNNING` for a moment.
**Duplicates only ever affect media and files, never scalar metrics.** Metrics carry their original timestamps, so replaying them replaces rather than duplicates. Use `--force-resume` only when you deliberately want to re-load a run that's already marked loaded.
## Large migrations
* `--workers` controls project concurrency and, on export, the per-run file-download budget. Budget \~2–4 GB of RAM per worker.
* `--flush-every` and `--max-pending` bound how much the loader queues before it throttles.
* `--cleanup` deletes each run's staged files as soon as it's confirmed loaded, so peak disk stays close to the largest single run rather than the whole entity.
* Set `disable_system_metrics` so the import machine's own hardware stats aren't recorded into the runs you're importing:
```python theme={null}
import pluto
pluto.settings(disable_system_metrics=True)
```
The runs still receive the system metrics captured on the original W\&B host.
## What migrates
| What | Notes |
| --------------------- | --------------------------------------------------------------------------------------------------- |
| **Scalar metrics** | Original step and timestamp; booleans as `1.0`/`0.0`; `NaN` and `±Inf` preserved |
| **String metrics** | Non-numeric series become a [string metric](/pluto/visualizations/lines#string-metrics) staircase |
| **Histograms** | Pre-binned, with real bin edges reconstructed from W\&B's packed bins |
| **System metrics** | `system.*` → `sys/*`, unless `--no-system-metrics` |
| **Console logs** | stdout/stderr from `output.log`, unless `--no-console` |
| **Images** | Single and list-logged, order preserved, unless `--no-files` |
| **Image annotations** | Bounding boxes and segmentation masks — see [Annotations](/pluto/visualizations/images#annotations) |
| **Audio & video** | Including list-logged, unless `--no-files` |
| **Tables** | Plain `wandb.Table` → `pluto.Table` |
| **Custom charts** | `wandb.plot.*` panels, re-rendered as Vega |
| **Sweep membership** | Runs keep their sweep grouping as a `sweep:` tag, alongside the sweep's recorded config |
| **Run metadata** | Name, tags (plus an `import:wandb` tag), state, config and summary |
| **Run environment** | Git commit/branch, OS, Python version, CLI args, GPU inventory |
| **Artifact files** | Files of *logged* artifacts, unless `--no-artifacts` and subject to `--artifact-max-size-mb` |
## What a migrated run looks like
Migrated content renders with some limits worth knowing before you go looking for it.
| Content | Where it appears | Limits |
| ----------------------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Custom charts** | All-runs and individual-run **Charts** tabs | Up to **25 runs** overlaid per chart; only the newest step of the backing table; **cannot be added to a dashboard**, reordered, hidden, or saved in a Charts layout |
| **Plotly / matplotlib** | Charts view, run page, Files tab | JSON over **8 MB** renders as raw text instead of a chart |
| **3D point clouds** | Same | Row widths 3, 4 and 6 only; over 200,000 points are decimated |
| **HTML** | Same | Sandboxed, so external stylesheets and scripts don't load |
| **Tables** | Charts view, run page, Files tab | Newest step only — there is no step slider; no dashboard widget |
| **String metrics** | Everywhere, including dashboards | Sampled to 500 steps |
Logs named like W\&B's per-run artifacts — `run-xxxxxxxx-name:v0` — are hidden from the metrics views by design. Find them on the run's **Files** tab.
## What does not migrate
W\&B never exports the spec body for a custom Vega panel, so there's nothing to render. The chart's **backing table still imports** — the data survives, the visualization doesn't. Flagged as `custom-chart-unsupported`.
Bokeh has no renderer. `Object3D` imports only as a point cloud — mesh and molecule formats (`.obj`, `.glb`, `.pdb`) aren't handled.
`joined-table` and `partitioned-table` are skipped. Plain tables import; a table with media in its cells imports as data, with the media cells rendered as text.
Only the *files* of a run's logged artifacts are re-uploaded. Versions, aliases, the type/lineage graph and input (used) artifacts are dropped.
W\&B Reports, saved panel layouts and custom dashboards.
Registered models and their lineage.
`_`-prefixed summary fields such as `_wandb` are stripped.
## Silently dropped
Most gaps are reported in the coverage flags below. These are **not** — nothing warns you, and `--strict` won't catch them:
* **Custom charts, when you pass `--no-files`.** The panel specs live in the run's `config.yaml`, which `--no-files` never downloads. Every `wandb.plot.*` chart disappears with no flag raised.
* **History keys beginning with `_`** — `_runtime`, `_wandb` and similar.
* **History rows missing `_step` or `_timestamp`**, and `None` values.
* **System metrics not prefixed `system.`** in W\&B's events stream.
A few things migrate, but with less fidelity than you might expect:
* **W\&B `notes` don't become Pluto notes.** They're preserved under the run's config, alongside the original URL, state and summary.
* **Console output loses its stream.** Every line is recorded at `INFO`, so stdout and stderr are no longer distinguishable.
* **Histograms with no bin edges get synthetic ones** (`0..N`). Counts are exact; the x-axis is generic.
* **String values over 200 characters are dropped** — string metrics are short state labels, not free-form text. Flagged as `string-series-too-long`.
## Coverage flags
The exporter prints a flag for each dropped item, per run and as an end-of-project total. `--strict` exits non-zero if any flag was raised.
| Dropped item | Flag |
| ---------------------------------------------------- | --------------------------------------------------- |
| Hand-authored Vega, or an unresolvable backing table | `custom-chart-unsupported` |
| String value over 200 characters | `string-series-too-long` |
| Media skipped via `--no-files` | `media-file(--no-files)` |
| Unknown media type | `unsupported()` |
| Non-scalar history value | `unsupported()` |
| Failed downloads | `file-download-failed` / `artifact-download-failed` |
| Artifact over `--artifact-max-size-mb` | `artifact-over-size-cap` |
| Table columns containing media | `table-media-cell` |
| Input-artifact lineage | `artifact-input-lineage` |
| Artifact versions and non-latest aliases | `artifact-versioning` |
`--strict` fires on **any** flag, including ones you caused deliberately. Exporting with `--no-files` raises `media-file(--no-files)` for every skipped item, so `--strict` will fail that run.