Overnight Proof Runs with Cheap Models: How Verification Gates Beat Raw Model Price
Empirical results from running Gemma 4 31B and Qwen 3 Coder 30B overnight through AgentAFK verification gates versus single-shot unassisted generation.
You do not need an expensive per-token frontier API to complete verified engineering work overnight. Using an existing Ollama Cloud subscription or a local LAN endpoint running gemma4:31b and qwen3-coder:30b, agentafk.si schedules bounded multi-step proof runs where every candidate must clear an independent executable verifier (verifier.py) before acceptance.
Observed overnight benchmark results (Fixed Model Weights)
Across our published evaluation suites (papers.tayloramarel.com and compare.baltor.ai), holding the model weights strictly fixed while enabling the six-part harness and AgentAFK verification loop yields the following verified gains:
| Benchmark Suite | Fixed Model | Unassisted Baseline | With Baltor + AgentAFK Gate | Mean Calls to Verified Pass |
|---|---|---|---|---|
| 3D Foxglade Survival & 49-Bone Rig | gemma4:31b |
20.0% (2/10 specs) | 100.0% (10/10 specs) | 2.4 calls |
| 3D Fluted Lathe Relief & Swept-AABB | qwen3-coder:30b |
28.1% stability | 100.0% watertight & 0 tunneling | 2.1 calls |
| 14-Constraint Long-Form Spec | gemma4:31b |
42.9% (6/14 rules) | 100.0% (14/14 rules) | 1.8 calls |
| Harmony Secure Coding (Joint Safe+Func) | gemma4:31b |
31.2% joint pass | 82.5% joint pass (0 PoC exploits) | 2.7 calls |
Why a process exit is never counted as completion
In agentafk.si, a worker process exiting with code 0 means only that the script finished running. Work is marked accepted only when the morning status report confirms that every declared invariant in verifier.py passed, no known-wrong control was admitted, and the output artifact reopens cleanly in a fresh environment.
| Navigation | Post Title | Routes |
|---|---|---|
| Previous Post | When a Test Fails at 3 AM: Deterministic Counterexample Repair vs. Blind Retry Loops | HTML · Markdown |
| Next Post | Why Overnight Coding Runs Stop After 15 Minutes: How Checkpoint Leases Fix It | HTML · Markdown |
| Domain Blog Index | Return to agentafk.si/blog Index | feed.json · feed.xml |