e2e: keep the harness's own deadlines off the scaling path

Commit

Stefan Haubold1mo ago

Review findings from Copilot on #2373.

The prompt-timeout scaling in the previous commit assumed WithPromptTimeout is only ever called from a test. It is not: opencode's Bootstrap warmup bounds each attempt with it (opencode.go:215), so scaling by opencode's 2.0 turned a documented 90s/30s/30s retry sequence into 180s/60s/60s. openCodeWarmupRetryBudget exists precisely because "3 x 90s is over four minutes of blocked CI", so the change broke the bound that constant was written to enforce -- and broke it silently, since nothing fails, CI just blocks for longer on the exact path the warmup exists to survive.

The premise behind scaling is false for such a number: it was written by one runner, for that runner, already knowing how slow it is, so multiplying it by that same runner's multiplier double-counts. withExactPromptTimeout (unexported) is now the option for the harness's own deadlines, WithPromptTimeout stays the scaling one for tests, and TestHarnessDeadlinesAreNotScaled fails the build on any non-test file in the package that reaches for the wrong one. Negative-tested by reverting the warmup call site: the guard names opencode.go:215.

My grep for the call sites was grep -rn WithPromptTimeout e2e/ | grep -v '/agents/', which filtered out the one directory holding the counter-example.

Two documentation corrections from the same review. The model paragraph said six of seven paid runners have both an in-code default and an env override, which contradicted the very next bullet naming droid as the no-override exception; it is five (claude-code, codex, gemini, opencode, copilot-cli), with droid having the default and cursor the override.

And the claim that --model is passed by both entry points for every runner that pins one was wrong twice over: codex pins its interactive model in the config.toml that seedCodexHome writes, and droid through the Factory BYOK settings Bootstrap writes, both passing no model flag at all (droid.go:195 launches the binary bare). The corrected text names the three routes, which is more useful than the claim it replaces, and keeps the real point that Claude.StartSession pins nothing by either route.

Verified: canary vogon 56/56 and roger-roger 4/4, lint 0 issues and go vet clean on the rebased toolchain (Go 1.27.1, golangci-lint 2.13.2).

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com Entire-Checkpoint: 01M27ZJ300H84QDBPSHPA5TC02