Copy-on-Write Legacy Checkpoint Attach

Here's the brief. I haven't edited anything yet.

Problem. With a git-refs primary, writeReservedSession (checkpoint/routing_store.go:282) sends a hex ID to the v1 branch. It also updates the ref copy, but only if one already exists (updateReadTarget). git-refs pre-push never sends v1, so the attach push "succeeds", then fails the delivery check. Under manual, the hint points to migrate-checkpoints, which refuses once git-refs is primary.

Root cause (from the code). No code path creates the ref for a hex checkpoint that exists only on v1. The refs store already queues every write it makes (updateCheckpointRef calls enqueueForPush). So once the ref exists, the existing "update the readable copy" branch already handles pushing.

Proposed change (small)

1. In writeReservedSession, before writing to v1: if the primary is git-refs, the ID is hex, the ref is missing, and the ID is not on the local deleted list, copy the v1 subtree into RefName(id). The copy reuses migratedCheckpointTree, plus the refBase and treeInRefHistory CAS logic, which I'd pull out of the per-checkpoint body of migrateBranchToRefs into one shared helper so nothing is duplicated. Then set updateReadTarget = true and let the existing code run:

   • v1 is written (local only, same as today).
   • The ref is written and queued.

   The copy has to happen before the v1 write, or the new session would be copied in and then appended a second time.

2. In reportHeldAttachedCheckpoint, remove the migrate-checkpoints branch. A hex checkpoint that isn't on a ref after the write becomes the same "not found after it was written" error a ULID gets today.

3. Update TestAttachCommit_LegacyCheckpointUnderManualPointsToMigration to expect entire checkpoint push <id>, and update the two paragraphs in ref-checkpoint-backend.md.

Tests first: your cases (1) to (5). Case (5) expects no ref, and the write stays on v1 as today.

One risk you should decide on: test (4). I read the code but haven't run it.

• Two clones copying the same checkpoint create import commits with different hashes, so the two refs share no history.
• On the second push, the non-fast-forward recovery runs ReconcileDisconnectedMetadataRef, which replays the local commits with cherryPickOnto.
• That replay overwrites path by path with no conflict check (metadata_reconcile.go:408). Replaying the second clone's import commit would overwrite the first clone's root metadata.json. Both sessions would also take the same session index, so one would overwrite the other.
• Result: both pushes succeed, but the first clone's session is lost on the remote. The doc says a conflict "leaves the ref queued"; the code I read doesn't do that.

Options:

• A (recommended): attach fetches the ref before copying. Attach already knows the remote, and today it skips the fetch because the local v1 copy counts as present. This fixes the case where one clone copies after the other has pushed. It leaves the case where both copy before either pushes, plus condensation, which runs in post-commit with no network. I'd document that remaining gap and make test (4) assert that both sessions survive in the after-push case.
• B: copy without any fetch and write test (4) to assert only that both pushes succeed. Simpler, but it hides the data loss.

Open questions

1. Fix A or B? I recommend A.
2. Include the backfill path (SessionSummary/SessionTranscript)? I recommend yes. From the code, it's the same gap: refs returns ErrCheckpointNotFound, so the write falls through to v1. The helper covers it in a few lines, and I'd add one test for it.
3. Fix the setup.go:3405 message here or in a follow-up? I'd do it here, since it also points to a command that does nothing.
4. File "migrate automatically on switch" as its own ticket? I'd say yes, but separately.
5. Brain fact fact:7f0e1e96d00fb1460edfaf64: once this lands, I'll show you the exact replacement text before changing it.

One note: I'm working in this worktree (muskan/ent-3710-copy-legacy-checkpoint-into-its-ref-on-write, clean, at main ab90209737), not ent-3710-legacy-attach-upload. I'll point the repro overlay here once you approve.

Fix A is approved. Here's what the other two are, so you can confirm you want both.

1. Backfill path. After a checkpoint is written, Entire sometimes adds to it later:

• SessionSummary adds the AI summary, for example after entire explain generates one.
• SessionTranscript adds the full transcript once a session finalizes.

These go through a different function, kindRoutingStore.Write, not writeReservedSession. For an old hex checkpoint that exists only on v1, the refs store says "not found", so the write lands on v1, which is never pushed. That's the same bug as attach: the summary or transcript stays local and nobody else sees it. Including it means those writes also copy the checkpoint into its own ref first, so the update gets pushed.

2. setup.go fix. entire configure --checkpoint-push-policy manual refuses when the backend isn't git-refs. Its error says to switch backends and then run entire doctor migrate-checkpoints. But migrate-checkpoints refuses once git-refs is primary, so following those steps in that order hits a dead end. The fix is to reword the message. Once the copy works, old checkpoints move over on their own the next time they're written, so the message can just say to switch to git-refs.

If you're fine with both, I'll start with the failing tests, then the code, then mise run check. No commit until you approve.

The fix is in, and all five cases plus the backfill and setup.go changes pass their targeted tests. mise run check (format, lint, all CI tests) is still running; I'll report as soon as it finishes. Nothing is committed.

How it works now (git-refs primary, hex checkpoint ID)

• Attach, condensation and backfills: if the checkpoint has no ref here yet, the write first copies it from v1 into its own ref, normalized the same way migrate-checkpoints does it, then writes through the primary. That queues the ref for push.
• Shared logic: the copy and migrate-checkpoints now use the same per-checkpoint import function, importBranchCheckpoint in checkpoint/migrate.go.
• Deleted checkpoints: a checkpoint on this clone's deleted list is never turned back into a ref. Its write still goes to v1, as before.
• Fix A: attach fetches the checkpoint's ref before copying. If it can't reach the remote to check, it refuses and changes nothing.
• Manual hint: attach no longer points to migrate-checkpoints. Under manual it says the checkpoint is waiting for entire checkpoint push <id>.
• setup.go: the refusal now says to run migrate-checkpoints first, then switch. I kept the step rather than dropping it as I'd said I would, because migration works before the switch. Without it, v1 checkpoints that were never pushed and never written again would stay local.

Verified

• Your original repro: the attach now pushes refs/entire/checkpoints/83/5d7ced858c83 to the remote. Before the fix the remote didn't change.
• The two-clone test proved Fix A was needed. Before Fix A it failed: the remote ended up with [first, clone-b] and clone A's session was gone. With Fix A all three sessions survive.
• Each new test failed first for the expected reason, except the two deleted-checkpoint tests, which already passed because today's code never creates a ref.
• Passing: all checkpoint package tests, the strategy and attach tests, the configure integration test, and the docs link test.

Changed or removed tests

• TestKindRoutingStore_ReservedSessionSurvivesReadTargetFailure is deleted, because the read it protected no longer happens.
• The mirror-fanout subtest now uses a deleted checkpoint to reach the branch fallback.
• Three "FallsBackToBranch" tests are renamed, since those writes now land on the ref.
• TestAttachCommit_LegacyCheckpointUnderManualPointsToMigration is now TestAttachCommit_LegacyCheckpointUnderManualWaits.

Open risks

• Backfills now include Tasks. It shares the summary and transcript path, so it gets the same copy.
• Condensation can't fetch, because it runs in post-commit with no network. If two clones both copy the same checkpoint before either pushes, one session can still be lost. This is documented in ref-checkpoint-backend.md, not fixed.
• Attach behaviour change: for a hex checkpoint with no local ref, attach now fails offline instead of copying blind.

Still pending: the Brain fact update (I'll show you the text first), a separate ticket for "migrate automatically on switch", and your approval to commit.

1. When attach can't reach the remote

This only happens when all three are true: the checkpoint has a hex ID, the primary is git-refs, and this clone has no local ref for that checkpoint yet. Attach then asks the remote whether the ref already exists.

• Remote says no such ref: attach copies from v1 and continues as normal. A repo with no remotes at all counts as this case.
• The remote can't be reached (offline, auth failure, timeout): attach stops with:
  It exits non-zero. This check runs before the confirmation prompt, the trailer amend and the checkpoint write, so the commit, v1, refs and session state are all untouched. You just run attach again once you're online.

Why refuse rather than copy anyway: if another clone already pushed its copy, copying blind gives a second ref with unrelated history. On push, recovery replays our commits over theirs, and their sessions are lost. The two-clone test showed exactly that before the fix. A pushed-commit attach needs the network to push anyway, so refusing costs little there. The real cost is an offline attach to an unpushed commit with a pre-switch checkpoint. That used to work locally and now waits until you're online.

2. Why attach no longer points to migrate-checkpoints

Yes, because of our fix. Before:

• Attach wrote the session to v1.
• entire checkpoint push only sends refs, so attach told you to run entire doctor migrate-checkpoints first.
• But migrate refuses once git-refs is primary, so following the hint did nothing.

Now attach copies the checkpoint into its own ref and queues it, so under manual it waits like any other checkpoint and entire checkpoint push <id> uploads it. There's no v1-only case left to point anywhere.

One gap I found while answering this: a checkpoint on this clone's deleted list still gets its write on v1, by design. Under manual, attach then reports "checkpoint was not found after it was written; run attach again", which is misleading: rerunning won't help. Under automatic, the delivery check fails instead. I checked and attach doesn't look at the deleted list at all. The fix would be for attach to refuse up front with "checkpoint <id> was deleted from this clone". It's small and in scope. Want me to add it (test first)?

mise run check is still running.

The first full mise run check failed. Lint was clean, but four tests failed:

• One was a real break from this change. TestRunExplainAuto_GeneratePersistsHexOnBranchUnderRefsPrimary expected a generated summary to be saved on v1. With the backfill change, it's now saved on the checkpoint's own ref. I updated the test to check for the ref and read the summary back through the normal store, and it passes.
• Three look like flakes: TestProbeConfigDir_AsksClaude, TestFetch_AbortStalledLetsProgressingFetchRun and TestExternalCommand_SigintReachesPlugin. They're timing and signal tests in packages this change doesn't touch, and all three pass when run on their own.

The full check is running again. I'll tell you the result when it finishes; nothing is committed.

One question from before is still open: should attach refuse up front for a checkpoint on this clone's deleted list? Right now it gives a misleading "run attach again" error there.

I'll start once the running mise run check finishes, since editing files mid-run would muddy its result.

Plan, test first:

• Test (in attach_commit_test.go): set up a legacy checkpoint, record it on the clone's deleted list, then attach to its commit. Attach must fail with a message naming the deletion. It must also leave everything as it was: no new ref, v1 unchanged, and nothing on the remote.
• Code: in ensureCheckpointAvailable, before the fetch and the copy, load the deleted list and refuse. The error will read: checkpoint <id> was deleted from this clone, so attach won't add a session to it; nothing was changed.

One scope note: I'd apply the refusal on both backends, not only git-refs. On git-branch, attaching to a deleted checkpoint writes it back onto v1, and that gets pushed, so the checkpoint comes back there too. Tell me if you'd rather keep it to git-refs only.

It's rare. It needs three things to line up.

The setup

1. Back on git-branch, you commit X with an agent session. Its checkpoint a1b2c3d4e5f6 goes onto v1 and is pushed. X carries the trailer Entire-Checkpoint: a1b2c3d4e5f6.
2. The repo switches to git-refs.
3. You notice the transcript leaked an API key and run entire checkpoint delete a1b2c3d4e5f6. That removes it from local v1 and from each remote's v1, and records the ID on this clone's deleted list. X is already pushed, so its trailer stays.

The usual case: attach already refuses

Later you run entire session attach <other-session> --commit X. Attach reads the trailer, looks for the checkpoint and finds nothing, because the delete removed it everywhere. It fails with "checkpoint not found". No change is needed here.

The edge case: a copy is still visible

The checkpoint can still be visible even though it's on the deleted list:

• The delete couldn't reach one remote (say a second remote was offline), so that remote's tracking ref for v1 still has it. The delete prints a retry command, but you haven't run it yet.
• Another clone that still had the checkpoint pushed it back. The docs say remote tombstones aren't built yet, so this can happen.

Now attach sees the checkpoint as present and writes to it. With our change, a deleted checkpoint's write goes to local v1:

• Under git-refs: v1 is never pushed, so the session stays local and attach fails with a misleading message. Under manual it says "not found after it was written; run attach again", and rerunning won't help. Under automatic it says "did not reach origin".
• Under git-branch: this is worse. The write puts the deleted ID back on local v1, and the next push sends it to every v1 remote. The secret transcript doesn't come back, but the delete you ran gets silently undone: the checkpoint reappears holding the new session.

My take: it takes a partial or undone delete followed by an attach to that same commit, so it will rarely happen. But the fix is a few lines, and the git-branch case quietly reverses a user's explicit delete. So I'd still add the up-front refusal on both backends. If you'd rather keep this change focused, it's also fine as a follow-up ticket.

The full mise run check rerun is still in progress.

The rerun passed everything except TestProbeConfigDir_AsksClaude. Lint is clean, and every test touched by this change passes.

That one failure looks unrelated:

• The test starts a fake claude binary, and the probe gives it 5 seconds. Both full runs failed at exactly 5.00s, which points to the subprocess being starved under the load of the full parallel suite.
• It passes on its own (3 of 3 runs).
• It's in agent/claudecode, which this change doesn't touch: git status shows no files under agent/.

So mise run check hasn't passed cleanly yet. Before any commit I'll rerun it, and if this test fails again, check whether it also fails on main under the same load.

The deleted-list refusal is still waiting on your decision:

• Add it here, on both backends (my recommendation): test first, a few lines in ensureCheckpointAvailable. It matters most on git-branch, where attaching to a still-visible deleted checkpoint undoes the delete on the next push.
• File it as a follow-up ticket and keep this change focused on the copy on write.

Nothing is committed.

Attach now refuses a checkpoint on this clone's deleted list, on both git-branch and git-refs, before it writes or fetches anything. The full mise run check is still running.

• Test first: TestAttachCommit_RefusesACheckpointDeletedFromThisClone records the checkpoint as deleted, then attaches to its commit. On both backends it failed before the change, because attach succeeded. Now it passes: attach refuses, and the local and remote checkpoint refs are unchanged.
• Code: at the top of ensureCheckpointAvailable, attach checks the deleted list. If the checkpoint is on it, attach stops with: checkpoint <id> was deleted from this clone, so attach won't add a session to it; nothing was changed. If the deleted list can't be read, attach also refuses rather than guessing.
• Docs: the checkpoint delete section of ref-checkpoint-backend.md now says the deleted list makes attach refuse and that the copy on write skips listed checkpoints.

The whole attach test suite passes. I'll report the full check result when it finishes. If TestProbeConfigDir_AsksClaude times out again, I'll check whether it also fails on main under the same load. Nothing is committed.

Here's the split I propose, in commit order. Each commit is self-contained: its tests pass on their own and its doc lines go with it. Per your memory note, commit bodies are 1–2 lines with no Co-Authored-By trailer.

1. Refactor, no behavior change (migrate.go)
2. Reserved writes copy into the ref (routing_store.go reserved path, persistent.go helper, reserved copy tests, the updated table test, removal of the obsolete read-failure test, the condensation-across-switch test, and the routing and migration doc paragraphs)
3. Backfills copy too (the backfill loop, backfill copy tests, the mirror-fanout subtest, the three test renames, the explain test, and the backfill doc bullet)
4. Attach fetches the ref first, Fix A (fetchLegacyCheckpointRef, the two-clone test, the doc paragraph on the race)
5. Drop the dead migration hint (reportHeldAttachedCheckpoint, the automatic and manual legacy attach tests, the push-policy doc paragraph)
6. Refuse deleted checkpoints (the refusal, its test, the delete doc line)
7. Configure message order (setup.go and its integration test)

How I'd build them: some files span several commits, so I'd stage hunks with git apply --cached from edited patches, since interactive add isn't available here. I'd check that each commit builds and that its own tests pass.

One cost to decide: the repo rule is mise run check before every commit, and a full run takes about 7 minutes. That's about 50 minutes for 7 commits. I'd follow the rule. The alternative is to run lint and the touched packages' tests per commit, then the full check on the final tree.

Questions

1. Is this split and wording OK?
2. Full check per commit, or targeted checks plus one full check at the end?
3. Commit on the current branch muskan/ent-3710-copy-legacy-checkpoint-into-its-ref-on-write?

The full check on the current tree is still running; I'll report it before any commit.

Agreed, three is enough. Merged version:

1. Checkpoint package (migrate.go, persistent.go, routing_store.go, the routing and copy tests, the condensation-across-switch test, the explain test, and the routing and migration doc sections)
2. Attach (attach.go, the attach tests, the push-policy and delete doc paragraphs)
3. Configure (setup.go and its integration test)

No Co-Authored-By trailer. They'd go on the current branch, muskan/ent-3710-copy-legacy-checkpoint-into-its-ref-on-write. I'd run mise run check before each commit as the repo requires, about 7 minutes each, around 20 minutes in total.

Is this wording OK to commit? The full check on the current tree is still running; I'll report it first.

The doc changes split cleanly by section:

• Commit 1: routing bullets, migration and coexistence, key files.
• Commit 2: the push-policy paragraph and the delete line.

One adjustment: the race paragraph in the migration section says attach fetches the ref first, which isn't true until commit 2. In commit 1 that sentence will just say the copy checks locally, and commit 2 adds the attach fetch. I'll stage each commit's version of the file from its contents, so the working tree stays as it is.

I'm waiting for the full check to finish, then I'll make the three commits with the messages you approved.

I don't need the monitor after all: the full check is already a background task, and it will notify me when it finishes.

So far, lint and formatting have passed. Commit 1 is staged, and on its own tree the attach, routing, migrate, condensation and explain tests pass. It also includes the replacement legacy attach tests, so no commit carries a stale test.

Once the full check reports, I'll commit:

1. Commit 1 (already staged).
2. Stage and commit attach: the attach.go change, the two-clone and deleted-checkpoint tests, and the final doc text.
3. Stage and commit configure: setup.go and its integration test.

Then I'll remove the temporary worktree in the scratchpad.

Understood: make all three commits now, then run the check once on the result. Commit 1 is in (828c9e9264). Doing commits 2 and 3: