Build the environment
Reproduce the tools, observations, and constraints the model will encounter in use. Make each task resettable, isolate side effects, and define the state that constitutes completion.
Resettable tasks and tool environmentReinforcement learning
Give the model a task it can learn to solve.
Some tasks have many valid solutions, and demonstrations cover only a fraction of the possible paths. Reinforcement learning lets a model explore those paths and learn from the result. We build the task environment, define what earns reward, and study how the policy changes as it learns to use evidence and tools.
TaskCompare account revenue.Two periods. Read-only data.
Trajectory ASum after line-item joinInvoice total repeated per line
Trajectory BSum invoice recordsGroup by account + period
Wrong aggregationWithhold reward
Verified resultOutcome feedback
Scored trajectories inform the next policy update
The verified result supplies the learning signal.
Sum each invoice once, group by account and period, and compare the returned totals with the reference result. Correct values under the task constraints supply the outcome signal for a policy update.
Make the feedback specific enough to learn from.
An agent samples different tool-use trajectories. Outcome checks distinguish a valid answer from a plausible failure, and scored attempts inform the next policy update.
What should a successful attempt earn?
For an agent, a fluent answer can conceal an incomplete task. We ground rewards in executable tests, verified state changes, or evidence-backed expert judgments. Efficiency rewards are gated on correctness; trajectory review checks whether shorter attempts still complete the work. Independent evaluations test whether reward gains reflect useful behavior.
The research work.
Each intervention has a purpose, a record of what changed, and a way to assess its effect.
Validate the reward
Separate correctness, evidence, and efficiency signals. Challenge the grader with plausible but wrong completions, and compare its judgments with expert review before using it to train a policy.
Reward function and grader evaluationFollow the learning dynamics
Run policy updates with methods such as GRPO. Track which actions receive useful feedback, how exploration changes, and whether the model is finding better solutions on tasks outside the training set.
Policy checkpoints and rollout analysisWhat determines
whether it works.
We set the evaluation around the application’s requirements, including the failures an aggregate score can hide.
- Reward validity
- Compare reward with independent task completion checks; inspect high-scoring failures and behavior that exploits the grading rules.
- Transfer
- Test new task instances, different evidence, and changed tool responses to distinguish a useful policy from a training-set shortcut.
- Behavior and cost
- Measure success alongside tool calls, invalid actions, tokens, and elapsed time; compare complete trajectories across checkpoints.
What leaves
the lab.
Environment and reward harness
Executable tasks, tool definitions, reward components, and independent checks of the grader’s behavior.
Training experiment
A reproducible training run with policy checkpoints, reward breakdowns, and sampled success and failure trajectories.
Independent evaluation
Task-level comparisons with the starting policy, including generalization, efficiency, and the failure modes still unresolved.
Before we begin.
Is RL always the next step after fine-tuning?
RL is useful when repeated attempts can produce informative feedback. If the main gap is format following or a missing tool convention, demonstrations may be sufficient. We first establish that the environment and reward can support learning beyond the existing policy.
What does a reward look like?
For code, it can be an executable test result. For search, it can combine finding the relevant material with support for the answer. For an enterprise task, it can inspect the final system state. Subjective criteria need an expert rubric and a calibrated judge.
Work with the lab
Bring us a research problem.
Tell us where the model falls short and what better performance would mean for your team.