What happens
The SIGALRM/max_seconds timing tests — three in tests/test_budget_enforcement.py, one in tests/test_cli.py, one in tests/test_cookbook_basics.py — fail intermittently when the machine is loaded, and pass reliably when it is quiet.
Evidence from one afternoon on a Linux 6.8 box:
- Three full suites running in parallel (separate worktrees, separate venvs): one run reported exactly those five tests failing, identically on that worktree's untouched
main; they passed when re-run individually.
- The same three files run 16 consecutive times on the same machine while otherwise idle: 16 × exit 0.
So the failure mode is CPU-contention-dependent: when cores are oversubscribed, the deadline timer or the timed body overshoots its margin and the assertion on "raised in time" / "did not raise" goes the wrong way. A developer running one suite never sees it; CI on a busy runner, or anyone running suites in parallel, will.
Why it matters
A test that fails only under load is indistinguishable from a real regression in the deadline machinery — which this repo actually has strong claims about (the guard now refuses an overrun even when the timer never fired). Every false red spends someone's attention on the exact subsystem where a true red matters most.
What to consider
- Widen the margins: the timed bodies sleep close to their deadlines; scale the sleep/deadline ratio so a 2-3× scheduling delay does not flip the outcome.
- Or serialize: a
timing marker plus -p no:randomly-style isolation (run those five tests in one process, not interleaved with the rest) would remove the contention within a single suite run, though not across parallel suites.
- Or gate on load: skip-with-reason when the 1-minute loadavg exceeds the core count, so the signal stays honest instead of red.
Acceptance criteria
Five timing tests pass 20/20 while a parallel pytest suite runs on the same machine, and still fail when the guard's exit check is reverted.
What happens
The SIGALRM/
max_secondstiming tests — three intests/test_budget_enforcement.py, one intests/test_cli.py, one intests/test_cookbook_basics.py— fail intermittently when the machine is loaded, and pass reliably when it is quiet.Evidence from one afternoon on a Linux 6.8 box:
main; they passed when re-run individually.So the failure mode is CPU-contention-dependent: when cores are oversubscribed, the deadline timer or the timed body overshoots its margin and the assertion on "raised in time" / "did not raise" goes the wrong way. A developer running one suite never sees it; CI on a busy runner, or anyone running suites in parallel, will.
Why it matters
A test that fails only under load is indistinguishable from a real regression in the deadline machinery — which this repo actually has strong claims about (the guard now refuses an overrun even when the timer never fired). Every false red spends someone's attention on the exact subsystem where a true red matters most.
What to consider
timingmarker plus-p no:randomly-style isolation (run those five tests in one process, not interleaved with the rest) would remove the contention within a single suite run, though not across parallel suites.Acceptance criteria
Five timing tests pass 20/20 while a parallel
pytestsuite runs on the same machine, and still fail when the guard's exit check is reverted.