Benchmarks
We evaluate frontier agents on real-world environments and private datasets.
Enterprise software engineering
How effective, and efficient are agents on daily engineering PRs in private production codebases?
EnterpriseSWEn = 28 taskspass@1- 1Claude Opus 5.5Claude Code · medium effort58.9%
- 1GPT-6.1 SolCodex · medium effort33.9%
- 2Gemini 3.8 FlashTerminus 2 · medium effort19.6%
4 agentsFull resultsBusiness problem solving
How well do frontier agents fare on day-to-day business problems?
CompanyGymn = 22 taskspass@1- 1Claude Sonnet 5.5Claude Code · medium effort9.1%
- 1Qwen 3.8 27BClaude Code · medium effort1.1%
2 agentsFull resultsMainframe engineering
We evaluate frontier agents in private production mainframe systems.
MainframeGym Batchn = 16 taskspass@1- 1Claude Opus 5.5Claude Code · medium effort60.9%
- 1Gemini 4 ArgonAntigravity · high effort29.7%
- 2Gemini 3.8 FlashTerminus 2 · medium effort12.5%
3 agentsFull resultsSubjective enterprise work
Real world is messy, and non verifiable. We present LH-Bench: expert skill based rubrics for evaluating agents on subjective tasks.
Paper GitHub DataFigma to code, expert preferencen = 33 tasksEloReported- 1GPT-5.2Codex1054
- 1Claude Opus 4.6Claude Code1039
- 3Gemini 3.1 ProGemini CLI907
3 agentsFull resultsProgrammatic content, expert preferencen = 41 tasksEloReported- 1Claude Opus 4.6Claude Code1173
- 2GPT-5.2Codex915
- 2Gemini 3.1 ProGemini CLI912
3 agentsFull results