Operator state: backup & disaster recovery¶

How to back up the watchtower/coordinator’s persistent state and recover after losing the host — without losing an in-flight swap or leaking a custody-sensitive artifact. For day-to-day running see watchtower-operations.md; for one swap’s situational status use pyrxd swap status.

Principle. Back up the durable swap state, not the liveness signal. The heartbeat file is ephemeral (rewritten every tick) — restoring a stale one would mislead the dead-man’s-switch. Below, “back up” always means the durable files.

What persistent state exists¶

State	Where (default)	What it is	If you lose it
SwapRecords	`--records-dir` (e.g. `~/.pyrxd/watchtower/swaps/<swap_id>.json`)	The per-swap FSM state the tower watches	The tower can’t watch that swap — re-derive from chain via `swap status`, or recover the recovery key-file
Pre-signed refund sidecars	`<swap_id>.refund.json` (`--refund-blobs-dir`, default = records-dir)	The operator-signed BTC refund the tower broadcasts if the maker walks. Custody-sensitive (a signed tx that pays you); carries no key/preimage	The autonomous refund can’t fire — re-create with `presign_refund.py` (you must still hold the refund key)
Claim-context sidecars	`<swap_id>.claim.json` (beside the records)	Per-swap covenant claim params (the two pkhs). Public; no key/preimage	The autonomous claim leg can’t be rebuilt — re-write from the swap setup
Durable seen-store	`<keys>.seen.sqlite` (+ `-wal`, `-shm`)	SQLite H-freshness / fire-once replay state (`SEEN-1`); WAL mode, `synchronous=FULL`	Fire-once memory is gone — see “Recovering the seen-store” (NOT a double-spend risk)
Recovery key-files	e.g. `~/.gravity_dust_run_keys.json`	WIFs + the preimage `p`. The most custody-sensitive	The preimage is gone → that swap’s asset can’t be claimed; only the refund path remains (before `t_rxd`)
~~Heartbeat file~~	`--heartbeat-file`	Ephemeral liveness, rewritten each tick	Nothing — do not back up or restore it

Sensitivity tiers (encrypt the top two at rest)¶

Secret — recovery key-files (WIFs + preimage). Back up offline / encrypted only; never to a shared/cloud path in plaintext. (And shred after a swap settles — see W1 hygiene.)
Custody-sensitive — *.refund.json (a signed tx paying you). Encrypt at rest; mode 0600.
Operationally critical, not secret — SwapRecords, *.claim.json, the seen-store. No private keys, but losing them disrupts watching/recovery. Back these up routinely.

Backing up¶

# Records + sidecars: a plain consistent copy is fine (JSON files written atomically via os.replace).
rsync -a --chmod=F600 ~/.pyrxd/watchtower/swaps/  /secure/backup/swaps/

# Seen-store SQLite: do NOT just copy the .sqlite while the tower runs — committed reservations may
# still live in the -wal sidecar. Use SQLite's online backup so the copy is internally consistent:
sqlite3 ~/.pyrxd/watchtower/keys.seen.sqlite ".backup '/secure/backup/keys.seen.sqlite'"
# (or stop the tower briefly and copy .sqlite + -wal + -shm together).

Then encrypt the backup (the refund sidecars + any recovery key-files in it are custody-sensitive), e.g. age/gpg to an offline key. Verify perms stay 0600 on restore.

Recovering after host loss¶

Restore the records-dir (+ refund-blobs-dir if separate) and the seen-store to the new host; re-apply 0600 on *.refund.json and any key-files.
Re-point the tower: --records-dir, --refund-blobs-dir, and the seen-store path to the restored locations. Start the dead-man’s-switch as a separate unit (see the ops runbook).
Re-validate the refund sidecars before relying on them. If you rotated the refund key/SPK since the backup, every sidecar is now mismatched and inert — re-run presign_refund.py and pass the matching --refund-spk to the tower (the rotation footgun in the ops runbook).
Re-arm autonomous custody if you use it (enable_autonomous_mainnet_custody, default off) — it does not persist; a restored tower comes up alert-only until re-armed.
Triage in-flight swaps with pyrxd swap status --swap-file … --check-chain: any swap whose deadline − safety_window falls near now needs action first. Pass the counter-leg locator (--btc-funding-outpoint TXID:VOUT, or --eth-contract 0x… --eth-rpc-url …) so the triage also reports whether the counterparty has already claimed and revealed the preimage — a live RXD covenant looks identical either way, and that is the difference between “keep watching” and “claim now”. Recovery files written before the harnesses started persisting btc_funding_outpoint / eth_contract_address do not carry it; older runs need the flag.
If a swap needs action the tower will not take — because the fee gate refused, or because you are recovering without a running tower — drive it by hand with the read-only cold toolkit: pyrxd swap recover-preimage to extract p from the counterparty’s own claim (provenance-checked; it never trusts the recovery file’s preimage_p_hex), then pyrxd swap build-claim or build-refund to print the raw hex. Neither broadcasts — read the decoded output, the fee, the relay floor and the deadline-aware target, then broadcast from your own node. Radiant has no RBF and no CPFP: an under-fee’d spend cannot be repaired, so this decision is deliberately yours. Anyone can pay the covenant’s script, so both commands (and swap status) report other outputs there and ignore them. If the recovery file does not record rxd_covenant_outpoint and the command picked the wrong output, pass --covenant-outpoint TXID:VOUT (the covenant funding from your run log).

Recovering the seen-store¶

If the seen-store is lost (not restored), the fire-once / H-freshness memory is gone — but this is not a double-spend risk for the claim executor: it reads live chain + mempool reality (gettxout include_mempool=true), so a covenant already claimed (even pending in the mempool) is treated as claimed before any fee carve. A fresh empty seen-store self-corrects on the next tick; the durable store is a cheap fallback, not the load-bearing idempotency. (See the 2026-06-14 autonomous-claim follow-up.) For the swap coordinator’s H-freshness replay gate, a lost seen-store re-opens the SEEN-1 window for funds first seen during the gap — restore it if you can.

DR drill¶

Periodically restore the backup onto a scratch host and run pyrxd swap status against a known swap-file to confirm the records + seen-store reload cleanly and the perms are tight. A backup you have never restored is a hypothesis, not a backup.