ETH2QS · agent-lab

Field report · six weeks · AI-driven, not AI-unsupervised

A six-week campaign ran on infrastructure that was mostly solved. Context was not.

Node wall-clock and agent wall-clock are decoupled with tools everyone already has: detached systemd, event-driven wakeups. What actually threatened a six-week, AI-driven Ethereum client benchmark was the one clock nobody budgets for — the orchestrating agent's own context window.

eth2quickstart.com  ·  companion to the bake-off deck  ·  2026-06-22 → 2026-08-04 (last measurement 08-03; final installer fix merged 08-04)

The rig

What, how, when, who — one screen.

What

Two numbers, per finisher

Final synced disk footprint and cold-sync duration. Seven execution clients vs. a fixed Prysm (three never finished), then a five-way consensus sweep vs. a fixed execution client.

How

Native, sequential, capped

systemd services, no Docker, one candidate at a time on a shared 12-core / ~62 GB host. Each run capped at 72 hours.

When

Six weeks, three phases

A 23-day initial campaign (Jun 22 → Jul 14, 2026), then steady-state and restart-resume follow-ons through Aug 4.

Who

Agents, with a human holding the fence

Claude Opus orchestrating, fresh Sonnet subagents building, a standing delegate lane for cheap and sandboxed work — a human operator holding the few levers that genuinely need one.

The three clocks

Two clocks are solved. The third is short — and it binds.

Node wall-clock
Solved · detached systemd, 72h
Agent wall-clock
Solved · event-driven wakeups
Agent context
The real constraint · not to scale

The context bar has no fixed duration to plot — it's short by construction, not measurement. That's the point of the next eleven slides.

The spine

The spine of it: context is the constraint that binds first.

Node time and agent time are both solvable with infrastructure that already exists. Agent context only scales if conclusions get pushed down into small durable files instead of carried across a growing conversation — that single design choice is what the agent hierarchy, the durable-state control loop, and the governance rails all exist to serve.

The timeline

Six weeks, three phases — 32 of 44 calendar days active.

Phase 1 · 23d gaps · 12d Phase 2 · 4d Phase 3 · 5d
Phase 1 · Jun 22–Jul 14

Initial campaign

Seven execution clients, then the five-way consensus sweep. Triage, installer fixes, first full syncs.

Phase 2 · Jul 26–29

Steady-state re-measure

Footprints re-read once compaction settled, plus a third CL sweep against a nethermind anchor.

Phase 3 · Jul 31–Aug 4

Restart-resume follow-ons

Bisected restart gaps from 12 minutes to ~35 hours, found no cliff, and tuned nethermind's minimal-history installer default.

Who ran it

Three tiers by design. One peer, outside it entirely.

OrchestratorClaude Opus — plans, reviews every diff, writes the durable state. Doesn't hand-write client code.
↓ dispatch  ·  ↑ summary only, not full context
BuildersFresh Sonnet subagents — one task each, report a short summary back.
DelegatesCheaper / sandboxed models — the standing lane for read-only research, routed through wrapper binaries. A design the orchestrator can reach for; no delegate landed work inside this campaign's window.

Outside the hierarchy — not dispatched, not reviewed as a child

Independent peerCodex reviews PRs adversarially, on its own initiative — it flagged an unproven "live on the site" deployment claim in this campaign's own write-ups. No arrow to this box: the orchestrator never routes work to it the way it routes work to a builder.

Governance

Rails the agent runs inside — and the two levers only a human pulls.

Always-on rails

The agent runs inside these, every time

One candidate at a time, no batching, ever. A 72-hour cap per candidate — footprint is the last sample before teardown, at sync for a finisher and at the cap for a capped run, never the peak. Conventional Commits only, never a force-push to master.

The 2 human levers

Only a human pulls these

Every destructive data-clean is gated behind an explicit human go-ahead, never assumed. And no pull request merges without an explicit human go-ahead — the operator holds that lever, the agent never takes it.

The near-miss

The gate that caught itself.

reth's out-of-the-box default is an archive node. Nothing would have flagged that as wrong — until the config-optimality gate started stamping every footprint config_optimal=yes|no. It caught the misconfiguration before a number shipped.

~2.8 TiB
reth's undetected default · archive mode
~1.2 TiB
reth pruned (--full) · projected finish
6
bug-fixes it took to trust the gate itself

Fixed 2026-06-25, before a single reth result was recorded — the rerun then hit the 72h cap at ~0.98 TiB partial, so the pruned figure stays a projection. We nearly published our own misconfiguration.

Four incidents, all caught & documented

What actually breaks over six weeks.

nethermind

13.3h silent stall

Head frozen at block 4,651, 0 peers — everything else looked healthy. P2P bind pinned to loopback; fixed by advertising the real external IP.

besu

Mid-sync deadlock

Downloader thread died; the process stayed alive and kept answering RPC. A stale pinned CL aged the sync pivot out of its servable window.

ethrex

Restart-resync cliff

Gaps through 23 min resumed; a 26-min gap stalled. Longer measured gaps discarded state and re-synced from scratch (~2h). Inherent to the current design.

erigon

Gap-close deadlock

OtterSync + a checkpoint-synced CL never issued the forkchoice update that would close the gap. Terminated per operator decision, recorded as a no-sync.

Bugs in our own measurement machinery

Two harness bugs that nearly cost us data.

SIGTTIN

A version check hung a run for 90 minutes

A detached tmux install step piped a version check through head -1. Outside the foreground process group, that raised SIGTTIN against a tty it didn't own — which stops, not kills, the whole subtree. Fixed by redirecting stdin from /dev/null.

The cap path

The disk snapshot only fired on the synced branch

A slow client hitting the 72-hour cap fell through to teardown — which wiped the datadir — before any snapshot was taken. One footprint survived only because it could be reconstructed from raw samples.

The cap path is the one you forget, and it's the one a slow client actually takes.

The durable control loop

A session dying mid-run loses nothing.

Not because the session doesn't die — it did, more than once, once to an out-of-memory event. Because what carries the campaign forward is durable state, not context.

1Human operator — approves destructive steps
2Claude orchestrator — starts and resumes runs
3Detached tmux driver — bake-off harness → systemd EL/CL services
4Small durable state — results, queue, and a handoff note, in a few markdown files
↑ a fresh session reads the state, not the logs
back toClaude orchestrator — resumes exactly where the dead session left off

If you're building the next one of these

What we'd tell the next person.

  • The third clock is the real limit. Node wall-clock and agent wall-clock are solvable with infrastructure; agent context only scales if you push conclusions to the data and keep durable state in small files.
  • Measure on every exit path, before you destroy anything. Success is the easy path — the cap and the error paths are where your data quietly disappears.
  • Gate your benchmark on config, not just on outcome. Stamp every number with "was this the client's best mode?" or you will eventually publish a measurement of your own mistake.
  • Give an agent a job and a fence. The agent owns the tedious, sustained correctness; the human owns the few irreversible levers.

Bottom line

The process, not the model, is why the verdicts hold up.

None of this made the agent smarter about Ethereum clients. It made the campaign survive session deaths, context clears, and its own mistakes — the config gate's six bug-fixes, the SIGTTIN and cap-path landmines — without losing the measurements that mattered.

Read it, reproduce it

The numbers, the harness, and every gotcha — on-site.

eth2quickstart.com/blog

One-line install → hardened Ethereum node, 12 clients, MEV integration. github.com/chimera-defi/eth2-quickstart

← → navigate  ·  N notes  ·  P print / PDF
SLIDE 01 / 14

Speaker notes