Benchmarks
We evaluate frontier agents on real-world environments and private datasets.
Mainframe engineering
We evaluate frontier agents in private production mainframe systems.
MainframeGym Batchn = 16 taskspass@1- 1Claude Opus 5.5Claude Code · medium effort60.9%
- 1Gemini 4 ArgonAntigravity · high effort29.7%
- 2Gemini 3.8 FlashTerminus 2 · medium effort12.5%
3 agentsFull resultsEnterprise software engineering
How effective, and efficient are agents on daily engineering PRs in private production codebases?
EnterpriseSWEn = 150 taskspass@1- 1Claude Sonnet 5.5Claude Code · medium effort85.3%
- 1Gemini 4 ArgonAntigravity CLI · high effort79.5%
Top 3 of 4Full resultsSubjective enterprise work
Real world is messy, and non verifiable. We present LH-Bench: expert skill based rubrics for evaluating agents on subjective tasks.
Paper GitHub DataFigma to code, expert preferencen = 33 tasksEloReported- 1GPT-5.2Codex · effort not reported1054
- 1Claude Opus 4.6Claude Code · effort not reported1039
- 3Gemini 3.1 ProGemini CLI · effort not reported907
3 agentsFull resultsProgrammatic content, expert preferencen = 41 tasksEloReported- 1Claude Opus 4.6Claude Code · effort not reported1173
- 2GPT-5.2Codex · effort not reported915
- 2Gemini 3.1 ProGemini CLI · effort not reported912
3 agentsFull results