Learning from execution
Reinforcement learning for analytical agents.
Executable SQL can still answer the wrong question. An agent may choose the wrong population, multiply rows in a join or ignore a metric's definition. We design training tasks around the complete analytical workflow, where these decisions have observable consequences.
The research question
Can an agent recover from an unhelpful query and reach a verifiable answer?
Sum each invoice once, group by account and period, and compare the returned totals with the reference result. Correct values under the task constraints supply the outcome signal for a policy update.
An agent samples different tool-use trajectories. Outcome checks distinguish a valid answer from a plausible failure, and scored attempts inform the next policy update.
The engagement
From task to training experiment.
Give the agent an environment it can investigate.
The environment combines a versioned schema, metric definitions and read-only query tools. A task might require comparing retention across product cohorts after a release. To answer it, the agent must find the relevant tables, establish the population and time window, execute its queries and inspect the result. Tool calls and responses are recorded so a failure can be reproduced and attributed to a specific decision.
Reward the outcome the task requires.
A successful database call is not a sufficient learning signal. We construct verifiers around result properties and curated task expectations, then test them against shortcuts such as empty answers, guessed values and joins that inflate the result. Reinforcement learning uses verified outcomes to update the policy that chooses the tool sequence. Access controls remain enforced by the environment; they are not a soft reward that the model can trade against another objective.
Make consistency part of the release decision.
Evaluation holds out schemas and question families, includes requests that need clarification, and repeats tasks to expose unstable behavior. We compare the trained policy with its starting model using the same tools. Correctness and access constraints are evaluated before query cost and latency. The deployment carries forward the tested tool contracts and runs a final quality check with its actual sampling, precision and timeout settings.
Evaluation design
What the experiment must establish.
- Task correctness
- Verify the population, time window and expected result properties. SQL syntax or execution success alone does not establish a correct answer.
- Verifier integrity
- Challenge the reward with empty outputs, plausible guesses and misleading results; review its agreement with domain experts.
- Consistency and transfer
- Repeat tasks on unseen schemas and question families, including ambiguous requests and recoverable tool failures.
- Execution cost
- Measure query work, tool calls, tokens and end-to-end latency within the set of behaviors that satisfy the task constraints.
Delivered artifacts
A body of work your team can build on.
- A versioned task suite and replayable environment with explicit tool and access contracts.
- An audited reward specification, evaluation protocol and records of grader failure modes.
- Training configurations, candidate policy checkpoints and a baseline comparison across held-out tasks.
- A serving integration with execution traces and a measured release decision.
Technical foundations
Further reading.
Explore another program
Data experiments that target the failure.Work with the lab
Bring us a research problem.
Tell us where the model falls short and what better performance would mean for your team.