Scaling RL from tacit knowledge
Certain domains are bottlenecked by human expert time. The rule that decides a fix lives in one engineer's head. We build the harness that turns it into environments.
Certain domains are bottlenecked by human expert time. The system is private and cannot be scraped. The unit of work is a ticket that names a symptom and no program. The rule that decides a fix lives in the head of one engineer with twenty years on the system, and it is in neither the ticket nor the code. That knowledge is tacit. The expert answers a question about it in minutes and would take days to write it down, and there is one of them.
Reinforcement learning on such work needs a steady supply of tasks that are real, correctly graded, and hard enough to stay discriminative as models improve. Expert-written tasks meet the first two conditions and fail the third at any scale, because each costs expert hours and a static set goes stale. Synthetic generation supplies volume and admits a task by executing its own tests, which says nothing about whether the task is real, the tests are right, or the rule it turns on was ever stated.
We propose a framework to scale RL on tacit knowledge: a continuously calibrating autonomous harness for environment design. The harness does the authoring. The expert answers only the questions a machine cannot settle.
The claim this rests on
Frontier models improve every quarter, so any thesis about training data has to say what stays scarce. Our answer is the rule held by the expert. A model can read every line of a billing system and still fill a missing business rule with an assumption, because the rule was never written anywhere a model can read. That gap closes only when someone who holds the rule states it, and the only way to scale that person is to ask them less and ask them better.
The gap is also the market. A lab pays for an environment exactly as long as its models fail on it. Tacit knowledge is the last thing a scraped corpus can supply, so it is the last place that failure persists. We build for that window and measure it rather than assume it.
The harness
The harness divides task creation into atomic steps and checks each with a versioned automated judge whose verdict is recomputed in code from rubric rows. It runs as a loop with three parts.
- Check. Judges validate a task before any rollout budget is spent and validate each trajectory after.
- Conduct. Admitted tasks roll out against real models, and their pass rates calibrate the next round.
- Consult. Every question a judge cannot settle goes to the expert asynchronously, in the original team's own words, and the answer returns as a task revision or a rubric revision.
Under these checks the loop explores widely, because everything it admits is clean, and the expert's hours go to the questions only an expert can answer. A verifier need not be one large binary reward at the end of a run. Placed as a specific check at a well-defined step, it decides one thing, and each step raises the signal-to-noise of everything downstream.
Proof on a mainframe
We chose the hardest proof we had: the decommissioned billing system of a large telecommunications company, a 432k-line COBOL, JCL and DB2 codebase, run off the mainframe on a toolchain hardened against the system's recorded behavior. Each task is a symptom-only ticket, the codebase before the original fix, and a test suite recorded from runs of the system and pinned to business observables.
On this codebase the harness explored 234 task ideas, admitted 27, and routed 442 named questions to one domain expert. Against end-to-end expert construction it reduced active expert time by an estimated eight times. No task was written by hand.
Frontier agents pass about one ticket in five. Kimi K3 under Terminus 2 passes 18.5 percent of trials and Claude Fable 5 under Claude Code passes 23.0 percent, with pass@3 of 0.44 and pass^3 of 0.13. In five failures out of six the agent finds the right programs, and the fix fails where the expert's knowledge would have carried it. It stops before the full chain, fills a missing business rule with an assumption, or verifies against a fixture written from that assumption.
That failure pattern is the thesis in one sentence. The agents locate the code. They lose on the rule.
Calibrating the harness like the agent
A harness that grades agents must itself be graded. We sample its judges for blind expert review and revise them when the review disagrees, the same discipline we apply to the models. The checks before rollouts are scored for precision and recall against that review. The trajectory judge is scored for agreement with human grading of the same rows. Those measurements are in progress and will appear in the paper with the benchmark release.
Partial credit has to keep the meaning of success. If reward is the fraction of checks passed and nine of ten checks guard existing behaviour, doing nothing earns 0.9. We grade with gates. Full reward requires every positive case and every guard, and a long task pays by the contiguous prefix of verified steps an agent reaches. A later step cannot compensate for an earlier failure.
Where the knowledge comes from next
The expert's tickets seeded the first benchmark. The next seed is the expert's own sessions. Humboldt is our coding agent for hired experts, and every session it runs lands on a transcript that records the demand, the tool calls, the executed results, and what the expert accepted or refused. The acceptance is the tacit knowledge, stated at the moment it was used.
The public reference point for turning sessions into tasks is SWE-Together, which recovered 109 tasks from 11,260 sessions, a yield below one percent. Our first tenet is to augment human seeds, so one session should become many scenarios on the same substrate. The multiplier per session and the share that human-validated checks accept are the two numbers we will report first, because the whole plan turns on them.
The first test is cheap and we will run it before any expert is hired at scale. Mine the sessions into tasks, run frontier agents against them, and read the pass rate. A task the frontier already passes carries no signal, whatever it cost to mine. A task it fails is a point on the curve.
Pretraining scaled on text that already existed. The next curve scales on knowledge that was never written down, and it needs the people who hold it in the loop.
Work with us