Pin Cursor Model and Scale Prompt Timeouts

Both hold up. Implemented, verified, not committed.

1. Cursor model — fixed, and the premise was understated

--model does exist on the Cursor CLI (confirmed via agent --help: --model <model>), and the isolated E2E config seeds cli-config.json with {}, so nothing pinned a model from either direction. Added E2E_CURSOR_MODEL, threaded through cursorArgs() so the real argv and the artifact's displayCmd can't drift, and applied to both entry points (RunPrompt and StartSession — StartSession was a separate path).

Default is deliberately empty rather than a cheap model, unlike the others: Cursor's ids are account- and release-scoped, so a name hardcoded in cursor_cli.go reds the whole leg the day the account loses access. Pin a verified id in e2e.yml instead. I couldn't pick one — agent --list-models needs auth I don't have, and per your notes cursor quota has been out since 08-24.

Checking "every other paid runner has a knob" turned up that it isn't true:

So E2E_CLAUDE_MODEL was a phantom: setting it got you haiku and no warning. Wired it up, plus E2E_COPILOT_MODEL (also fixed copilot-cli.go:407, where StartSession hardcoded the model and would have ignored the new knob — gemini's equivalent at gemini.go:156 already read geminiModel()). Left droid alone: its default is a BYOK custom model that Bootstrap() declares in ~/.factory/settings.json, so a knob moving only the flag would name a model the settings don't declare.

2. The five call sites — the numbers weren't the defect, the mechanism was

runForAgents (e2e/testutil/repo.go:435) already scales the scenario timeout by TimeoutMultiplier(). WithPromptTimeout didn't, so an absolute number bound hardest on exactly the agents the multiplier exists to give room to. TestRapidSequentialCommits asked three commits of gemini in 120s while granting it a 10m scenario budget for being 2.5× slow.

promptTimeout now scales the per-test value by the runner's multiplier, upward only. The runner's own default and E2E_TIMEOUT stay verbatim — the first is already agent-specific and would double-count, the second is typed by a human for one run. Never scaling down is the one asymmetry with the scenario budget: vogon and roger-roger at 0.5× are local and deterministic, so a tighter threshold there could only fire on a loaded machine.

edge_cases_test.go:138 is the only site that genuinely bounds anything (single prompt): gemini 120s→300s, droid/opencode→240s, codex/copilot/cursor/pi→180s, claude/vogon/roger unchanged. At the two-prompt sites the scenario ctx is the worst-case ceiling either way.

I also strengthened TestEveryRunPromptResolvesThroughPromptTimeout to pin that each runner passes its own receiver as the scaler — otherwise a runner silently runs on another agent's budget. Negative-tested both branches; messages name the offending expression.

Three things I found but did not change — your call

• multi_session_test.go:47 is self-contradicting. promptTimeout := 3 * time.Minute under ForEachAgent(t, 3*time.Minute, …) with two prompts: the per-prompt number equals the entire scenario budget, so as written it can never fire for a 1.0× runner. Fixing it means raising the scenario timeout, which raises worst-case CI wall-clock across ten legs — a cost call, not a cleanup.
• Claude.StartSession passes no --model at all (claude.go:182), so interactive claude tests run on the account default while its own RunPrompt pins haiku. Every other runner pins in both paths. Pinning it is a cost win but could red the interactive tests if they lean on a stronger model, and verifying that needs a paid run.
• mise-tasks/test/e2e/canary line 63: rr_filter="${filter:-TestExternalAgent}". A user filter is applied to roger-roger's lane too, so canary <any general test> sends roger-roger at tests CI never runs it against (e2e.yml:151 scopes it to TestExternalAgent). It reported 💥 FAILED (0/3) for me on a clean tree — same shape as your codex-integration-test-private-var note. Unfiltered runs are unaffected. I left it because narrowing that filter changes a shared gate's semantics.

Verification: full canary green — vogon 56/56, roger-roger 4/4. golangci-lint --build-tags e2e ./e2e/... 0 issues, go vet -tags e2e ./e2e/... clean, mise run fmt no-op. I proved the roger-roger failures pre-existed by running the same three tests in a throwaway worktree at 86dda0156 — identical results and durations (worktree removed).

Pin landed. What --list-models changed versus what I'd assumed:

Confirmed the finding outright: auto - Auto (default) is a real listed id and is what an unpinned run gets. So the non-determinism was exactly as described, not inferred.

Corrected my own reasoning. I'd justified the empty in-code default with "a hardcoded name reds the leg the day the account loses it." That's a weak argument, because a stale id fails loudly either way — --model haiku-does-not-exist exits 1 with Cannot use this model: X. Available models: <all 226>, no fallback to auto. I rewrote the comment and CLAUDE.md with the reasons that actually hold: --list-models is account-scoped (an id this org is entitled to needn't exist for a contributor running the leg locally), and Cursor offers no haiku-tier model to be the obvious cheap default — there is nothing in the list below gpt-5.4-nano / composer-2.5 / gemini-3.6-flash-minimal, and no haiku at any version.

Pinned composer-2.5 in both e2e.yml (matrix-conditional, beside codex's gpt-5.4-mini and gemini's gemini-3.1-flash-lite) and nightly-e2e.yml's cursor smoke leg. Cursor's own model, so it's the one tuned for their agent loop, and it was the fastest of the three I smoke-tested:

What I did not verify, deliberately: those were headless -p one-word replies, so they prove the id is accepted and completes a turn — not that composer-2.5 reliably does tool use, git commits and multi-step instructions in the interactive tmux path the suite actually uses. That needs mise run test:e2e --agent cursor-cli, and CLAUDE.md says not to run paid E2E unless you ask. Say the word and I'll run one test against the pin.

Re-verified after the change: vogon 56/56, roger-roger 4/4, golangci-lint --build-tags e2e 0 issues, go vet -tags e2e ./e2e/... clean, both workflow files parse. Still uncommitted.

I also saved the canary-filter trap to memory — a filtered test:e2e:canary sends roger-roger at tests CI never runs it against and prints FAILED on clean main, which is what cost me a baseline worktree to rule out earlier.

Answered: no, composer-2.5 was the most expensive of the four, and I've re-pinned to gpt-5.4-nano-medium as you chose.

Measured per 3-commit prompt, n=3 each, all completing 3/3:

The trap in my original pick: I had reasoned from the headline input price ($0.50 vs $0.20), where composer looks merely 2.5× dearer. The real driver is cache reads — $0.20/M against nano's $0.02/M. Composer uses the fewest tokens of the four (72k vs nano's 147k) and is still 3.6× the cost, because an agentic loop re-sends the conversation every turn, so cache read is the dominant line item. Ranking these by input price gives the wrong answer.

Two methodology notes on the numbers, since they're load-bearing:

• Token counts come from the CLI's own usage block (agent -p … --output-format json reports inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens) — that's the only accounting it exposes; --list-models has no JSON mode and ~/.cursor/statsig-cache.json carries no pricing. Prices from cursor.com/docs/models-and-pricing.
• cacheReadTokens is a subset of inputTokens in every sample (fresh input computed to 0), so I billed cache reads at their own rate. The ranking is unchanged under the other interpretation.

Changes: e2e.yml and nightly-e2e.yml pin gpt-5.4-nano-medium, with the measurement table in the e2e.yml comment and the escape hatch spelled out — nano is the weakest tier offered, so a capability-shaped failure should be met by bumping to gpt-5-mini (same cost, one tier up) before assuming the CLI is at fault. I probed one prompt shape, not the ~50 harder tests. CLAUDE.md and e2e/README.md updated to match.

One earlier claim I should retract: I'd written that speed favoured composer enough to matter. It doesn't, for the nightly leg at least — that smoke step runs a single test (TestMultiSessionSequential, 2 prompts) under a 20-minute budget, so ~18s of turn-time difference is noise there.

Gates after the change: go vet -tags e2e ./e2e/... clean, golangci-lint --build-tags e2e 0 issues, agents package tests pass, both workflow files parse, and the canary was green before the workflow-only edits (vogon 56/56, roger-roger 4/4 — untouched by a CI env var). Still uncommitted. I saved the cost table to memory so the pin isn't re-litigated from the input-price intuition.

PR open: https://github.com/entireio/cli/pull/2373 (17 files, +327/−47)

One thing worth knowing about how the pin gets exercised: e2e.yml triggers only on workflow_dispatch and push: main — not on pull requests. So none of the 14 checks now queued will touch the cursor leg; the pin's first real run is either a manual dispatch or the merge to main. If you want it validated before merging, gh workflow run e2e.yml -f agent=cursor-cli (optionally -f test=TestMultiSessionSequential to keep it cheap) is the way.

What I did before pushing:

• Rebased onto current origin/main — the branch was 50 commits behind and had no commits of its own. No upstream churn in e2e/ or the workflows; CLAUDE.md had moved (the Go 1.27.1 doc change) and merged cleanly, verified by hand.
• Re-ran the gate on the upstream toolchain, since the rebase brought Go 1.26.6 → 1.27.1 and golangci-lint 2.11.3 → 2.13.2 — my earlier verification was on the old pair, and a minor golangci-lint bump has changed linter scope in this repo before. mise run check on 1.27.1/2.13.2: fmt no-op, lint 0 issues, test:ci green including the canary (vogon 56/56, roger-roger 4/4).
• Staged by explicit path rather than git add -A, since test:ci rewrites .opencode/package-lock.json. Tree came back clean afterwards anyway.

The three items I flagged but didn't change — the self-contradicting multi_session_test.go budget, Claude.StartSession pinning no model, and the canary filter widening roger-roger's scope — are written up in the PR body under "Left for a maintainer" with the reasoning for each, so they're reviewable rather than lost.

Looked at both. Copilot found three things; one was a real regression I introduced.

The real one: I broke opencode's warmup budget

Bootstrap's warmup calls RunPrompt with WithPromptTimeout(openCodeWarmupBudget) (opencode.go:215), so my scaling multiplied it by opencode's 2.0× — turning a documented 90s/30s/30s retry sequence into 180s/60s/60s. openCodeWarmupRetryBudget's own comment exists because "3 × 90s is over four minutes of blocked CI," so the change broke the exact bound the constant was written to enforce, and broke it silently: nothing fails, CI just blocks longer on the path the warmup exists to survive.

The conceptual error is worth naming, because my justification for scaling was "a number in a test file is authored for the work, not for whichever runner draws the subtest." That premise is false here — the warmup numbers were written by opencode's runner for opencode, already knowing how slow it is, so multiplying by that same multiplier double-counts.

Fix: withExactPromptTimeout (unexported) for the harness's own deadlines, WithPromptTimeout stays the scaling one for tests, and TestHarnessDeadlinesAreNotScaled fails the build on any non-test file in the package reaching for the wrong one. Negative-tested — it names opencode.go:215.

My own search was at fault: grep -rn WithPromptTimeout e2e/ | grep -v '/agents/' filtered out the one directory holding the counter-example.

Two documentation corrections, both right

• I wrote "six of the seven paid runners pin a cheap default and take an override," which contradicted the very next bullet naming droid as the no-override exception. It's five (claude-code, codex, gemini, opencode, copilot-cli); droid has the default, cursor has the override.
• My claim that --model is passed by both entry points for every runner that pins one was wrong twice: codex pins its interactive model in the config.toml that seedCodexHome writes, and droid through the Factory BYOK settings Bootstrap writes — droid.go:195 launches the binary bare. I verified both rather than taking it on trust. The replacement names all three routes (flag / settings file / nothing), which is more useful than the sentence it replaces, and keeps the real point that Claude.StartSession pins nothing either way.

Pushed as a8e42f74d. Local gate on the rebased toolchain: lint 0 issues, go vet -tags e2e clean, canary vogon 56/56 and roger-roger 4/4.

Trail findings: none yet, and that's not the same as none

entire trail finding list says "No findings found," but the gates for head a8e42f74d explain why — the findings gate reads reviewed_head: false, review_in_flight: true, reviewer_configured: true. The agent review of the current head simply hasn't finished, so that empty answer isn't yet a real one. Zero findings at every severity so far, and the gate only blocks on high.

Gate state overall: checks pending (re-running for the new commit — the previous commit was green on everything except test-core, which I never saw finish), findings pending as above, up_to_date disabled, and approvals failed with "No reviewers have approved this trail" — 1 required, self-approval not allowed, so that one needs a human regardless.

Worth repeating from before: e2e.yml runs only on workflow_dispatch and push: main, so none of these checks exercise the cursor pin. gh workflow run e2e.yml -f agent=cursor-cli -f test=TestMultiSessionSequential is the cheap way to validate it pre-merge. Want me to re-check the trail once its review lands?

Both done — here's the state.

Copilot comment: resolved. The CLAUDE.md:324 count inconsistency was genuinely fixed in a8e42f74d (now reads "Five of the seven paid runners… and…"), and GitHub had already marked the thread outdated since my edit moved the line. Resolved without replying, as asked. The other two Copilot findings were suppressed rather than posted as threads, so there's nothing to resolve for them — both are fixed in the same commit.

Failing CI is not from this PR. One real failure and one rollup:

• test-core — TestCreateAndAwaitMirror_AsyncResubmission/resubmission_after_transport_failure_reuses_the_placement, race detected during execution of test. The race is entirely in test code: cmd/entire/cli/repo_mirror_request_test.go:368 does a bare submissions++ inside an httptest handler, and the first submission calls panic(http.ErrAbortHandler). That tears down the connection, the client retries on a new connection, and the new handler goroutine touches the same counters with no happens-before edge to the aborted one — both race stacks bottom out in net/http.(*conn).serve at that one line.
• test — failed in 3s and is just the matrix rollup ([ "failure" = "success" ]), so it's an echo of the above, not an independent failure.
• Entire Gates — fails downstream of the checks gate.

Evidence it's pre-existing rather than mine: my branch changes nothing under cmd/ (git diff --stat origin/main...HEAD -- cmd/ is empty), the racing lines are byte-identical on origin/main, they date from the 2026-09-02/03 async-mirror work (65b5efb90, 7967a5ca5), and recent main runs are green — so this isn't known-red main either. Locally -race -count=8 on that test passes on an idle machine, which fits a load-dependent race.

Worth being clear that this is a genuine data race, not a timing assertion — so it will recur. It's distinct from the AsyncTimeout phase flake you already have on file, which is just a 10ms budget expiring a phase early. The durable fix is a mutex or atomic.Int64 around those two counters — a few test-only lines, but in an unrelated package, so I'd rather not smuggle it into this PR unless you want it here. Happy to put up a separate one.

I re-ran the failed leg; 12 checks pass and test-core is re-running, and I've set a background watch that'll report when it lands. The remaining blocker regardless of CI is the approvals gate — 1 reviewer required, self-approval disabled.

CI is fully green after the rerun — all 14 checks pass, including test-core, confirming the race was the flake and not the change.

Gate state for head a8e42f74d:

The trail's agent review has now actually run against the current head and found nothing, so the earlier "No findings found" has become a real answer rather than a not-yet-run one.

The only thing left is the approvals gate, which needs someone other than you — mergeable: false is entirely that. Summary of the session's PR work:

• Copilot's three findings all addressed in a8e42f74d; the one posted thread is resolved, no reply.
• The substantive one was mine: scaling WithPromptTimeout silently broke opencode's warmup retry bound (90/30/30 → 180/60/60). Fixed by splitting withExactPromptTimeout for harness-owned deadlines, with TestHarnessDeadlinesAreNotScaled to stop it recurring.
• test-core's data race is pre-existing and real — repo_mirror_request_test.go:368, unsynchronized submissions++ across httptest handler goroutines after panic(http.ErrAbortHandler). It will recur; the fix is an atomic.Int64 or mutex, which I've left for a separate PR since it's an unrelated package. Recorded in memory so the next PR it reds gets rerun rather than bisected.

Still worth remembering before merge: e2e.yml doesn't run on PRs, so the cursor pin is unexercised by any of those green checks — gh workflow run e2e.yml -f agent=cursor-cli -f test=TestMultiSessionSequential if you want it validated first.

Conflict fixed and pushed. mergeable=MERGEABLE.

The conflict wasn't textual overlap — upstream d16dce053f ("docs: slim repository instructions into task-specific references") cut CLAUDE.md from ~2,300 lines and moved the E2E section wholesale into docs/development/testing.md. Git saw the whole file as rewritten, so the conflict spanned lines 31–2085. Resolving it by taking "mine" would have reverted that entire restructuring, so instead I took upstream's CLAUDE.md verbatim at both commits and relocated my prose into its new home — the target paragraphs were byte-identical there, so it ported cleanly.

CLAUDE.md is now absent from my diff entirely, which is the correct outcome: my branch touches docs/development/testing.md instead. History stayed honest — commit 1 carries its original wording, commit 2 carries the Copilot corrections on top, rather than collapsing them into one already-correct edit.

Two things I checked rather than assumed:

• The same upstream commit also edited e2e/agents/agent.go — a doc comment pointing at CLAUDE.md, retargeted to docs/development/testing.md. My changes to that file merged around it and line 62 keeps upstream's new path.
• That refactor added docs/development/docs_test.go, which enforces a 20 KiB budget on CLAUDE.md and validates links/headings in the references. It passes; my additions carry no markdown links, and since nothing went into CLAUDE.md the budget is unaffected.

Verification on the rebased tree (223 commits of main, same toolchain: Go 1.27.1, golangci-lint 2.13.2): mise run fmt no-op, lint 0 issues, go vet -tags e2e clean, docs/development guard passes, and test:ci green with the canary at vogon 56/56 and roger-roger 4/4. Pushed with --force-with-lease.

Current state: mergeStateStatus=BLOCKED, which is now only the approvals gate — 1 reviewer required, self-approval disabled — plus CI re-running on the new head. Nothing else outstanding; the cursor pin still won't be exercised by PR checks, since e2e.yml only runs on dispatch or push to main.

Replied to cli-90 with the status for #2373 — the one PR this session created.

Current facts, re-fetched rather than recalled (three days had passed):

• OPEN, not draft, mergeable=MERGEABLE, mergeStateStatus=BLOCKED, reviewDecision=REVIEW_REQUIRED, last updated 2026-09-17.
• Rollup is 15 SUCCESS, 1 FAILURE, 1 NEUTRAL. The FAILURE is Entire Gates, which aggregates the trail gates and is failing only on approvals — every actual CI check passes, and the trail's agent review reports no findings. The NEUTRAL is "Partial Gates", a separate service, which isn't a failure state.

I reported checks: pass rather than fail deliberately: the only red entry is the gate aggregator reflecting the missing approval, which the review: REVIEW_REQUIRED field already conveys. Calling it fail would have read as broken tests and sent Stefan hunting for a CI problem that doesn't exist. The attention line says exactly what the red mark is.

Nothing was pushed, commented, merged, or edited — the request was read-only and I kept it that way.

Pre-existing on clean main (it needs opencode deps seeded locally). Let me confirm my own tests pass and run the remaining gates: