Your InfluxDB history,
moved into Arc.
tsm2arc reads InfluxDB 1.x and 2.x TSM and WAL files directly off disk and streams them into Arc.
No running influxd. Crash-safe resume. Multi-field points reconstructed correctly. Built for cold volumes and EBS snapshots.
# safe first contact: decode everything, write nothing
$ tsm2arc --datadir /mnt/influxdb --dry-run --sample 10
# then load, WAL included
$ tsm2arc --datadir /mnt/influxdb/data --waldir /mnt/influxdb/wal \
--arc-url https://arc.example.net --token "$ARC_TOKEN"What it is
Migration straight off the disk
The usual export path needs a running InfluxDB. tsm2arc is built for the case where it isn't there anymore: data sitting on cold or unmounted volumes (an EBS snapshot, a decommissioned server's disk) that can be mounted read-only but are not served by any InfluxDB instance.
It decodes the TSM and WAL block codecs natively in Go, reconstructs multi-field points, and streams them into Arc's import endpoint in resumable, size-bounded, parallel chunks.
Point it at the data directory. 1.x and 2.x layouts are auto-detected; 2.x bucket names resolve from influxd.bolt.
Progress is checkpointed per shard in SQLite. Interrupted? Re-run the same command and it seeks to where it left off.
Every rename and every skip is recorded in the checkpoint and summarized at the end. Auditable, not quietly missing.
Capabilities
Built for migrations that have to finish
Terabytes of history, flaky networks, one shot at a maintenance window. tsm2arc assumes all of it.
Native TSM & WAL decode
All block codecs (timestamp, float, integer, unsigned, boolean, string) implemented in Go and validated against the real InfluxDB encoder.
1.x and 2.x, auto-detected
Same on-disk format, one tool. For 2.x, bucket IDs resolve to readable names from influxd.bolt and system buckets are skipped.
Multi-field rejoin
InfluxDB stores each field of a point as a separate TSM key. tsm2arc rejoins fields by series and timestamp into complete Line Protocol points.
WAL-only shards included
InfluxDB does not flush the WAL on shutdown, so recent data can live only in .wal files. Pass the WAL directory and nothing is left behind.
Crash-safe resume
A chunk is committed to the SQLite checkpoint only after Arc acknowledges it. On re-run, finished shards are skipped and partial ones seek to the cursor.
Parallel, with a heartbeat
Shards are fully independent, so workers scale cleanly. A live heartbeat reports shards done, chunks, rows, megabytes, and throughput.
Near-constant memory
Extraction streams one series, one block at a time. Peak heap stays a few megabytes regardless of shard, dataset, or largest-series size.
Dry run & shard profiling
A dry run decodes everything and prints per-database counts and sample lines without writing to Arc. An index-only analyze mode profiles shards fast.
Rename maps & validation
Names Arc would reject (dotted measurements, for one) are caught client-side before anything is sent: rename them explicitly, skip, or auto-map.
Quick start
Dry run first, then load
Three steps from a mounted volume to queryable history in Arc.
Install
# prebuilt binaries for Linux, macOS, and Windows on the releases page, or:
go install github.com/basekick-labs/tsm2arc/cmd/tsm2arc@latest
# container (multi-arch, linux amd64/arm64)
docker run --rm ghcr.io/basekick-labs/tsm2arc:latest --versionDry run against the source data
# 1.x: point at the InfluxDB root; 2.x: point at the v2 root.
# The layout is auto-detected either way.
tsm2arc --datadir /var/lib/influxdb --dry-run --sample 10Discovers shards, decodes every block, and prints per-database counts plus sample Line Protocol, without writing a byte to Arc. It also lists any measurement names Arc would reject, so renames are sorted out before the load.
Load into Arc
tsm2arc \
--datadir /mnt/influxdb/data \
--waldir /mnt/influxdb/wal \
--arc-url https://arc.example.net \
--token "$ARC_TOKEN" \
--verboseAlways pass --waldir: InfluxDB does not flush the WAL on shutdown, so recently written shards can live entirely in .wal files. If the run is interrupted, re-run the same command and it resumes.
Sizing workers, memory profile, rename maps, and the full flag reference? See the tsm2arc reference.
Bring your history with you
Open source and reusable beyond Arc: the extraction side emits standard Line Protocol, and new sinks are welcome.