Domain judgment
Capture how experts distinguish a supported conclusion from a plausible one, and teach the model to preserve that distinction.
Research at Stacklane
Teaching models to work inside a domain.
A model can know the terminology and still make the wrong decision. We study how expert judgments, task environments and outcome feedback can teach open models the behaviors a business depends on.
Expertise becomes useful when the learning signal is precise.
A corrected answer contains more than the right words. It can reveal which evidence matters, which assumption failed, or when the model should have asked for clarification. The research is in turning those distinctions into examples that teach a repeatable behavior.
Supervised fine-tuning learns from reviewed demonstrations. Preference optimization learns from comparisons between responses. Both depend on the coverage and consistency of the judgments in the dataset. More examples of the same easy case will not resolve a blind spot in a different part of the task.
Domain post-trainingDomain supervisionEvidence + expert judgmentWhat to extract. When to abstain.
Parameter-efficient adaptation
Frozen baseTrainable update
Target behavior
FindingRevision mismatch
EvidenceSpecification rev. C, §4.2
ExceptionTest report missing → review
Unseen documents test transfer
Change behavior without rewriting the entire model.
Low-rank adaptation learns a weight update, ΔW = BA, while the base weights remain frozen. Domain examples define the target behavior; independent documents test whether it transfers.
Train the decisions between a question and its answer.
For an agent, the answer is only the end of the trajectory. The model must decide where to search, what to inspect and whether the evidence is sufficient to continue.
TaskCompare account revenue.Two periods. Read-only data.
Trajectory ASum after line-item joinInvoice total repeated per line
Trajectory BSum invoice recordsGroup by account + period
Wrong aggregationWithhold reward
Verified resultOutcome feedback
Scored trajectories inform the next policy update
The verified result supplies the learning signal.
Sum each invoice once, group by account and period, and compare the returned totals with the reference result. Correct values under the task constraints supply the outcome signal for a policy update.
Make the feedback specific enough to learn from.
An agent samples different tool-use trajectories. Outcome checks distinguish a valid answer from a plausible failure, and scored attempts inform the next policy update.
Environments make those decisions observable.
We construct tasks with explicit tool access, resettable state and a way to check the outcome. A code-search agent can be assessed on whether its cited implementation supports the answer. A tool-using model can be checked against the resulting system state, rather than the confidence of its explanation.
Reinforcement learning updates the policy from these scored attempts. The difficult part is designing feedback that rewards the intended behavior. A short trajectory is useful only if it completes the task; a persuasive response is useful only if its claims survive inspection.
Reinforcement learning and task environmentsImprovement must survive a change of examples.
Hill climbing requires a useful development signal and a final test that remains independent of it. We separate training data, development evaluations and held-out release tasks. Depending on the domain, that separation may be by document family, repository, customer or time period, so closely related examples cannot make generalization look easier than it is.
We compare changes to the data, model and task harness separately. Ablations ask whether an apparent gain came from the training method, a better example mix, or different tools. Error analysis examines the cases an average score hides, including unsupported claims, failed actions and capabilities lost during specialization.
Serving constraints belong in that comparison. A candidate’s quality, latency, memory footprint and cost per completed task determine whether it is useful in the intended environment.
Evaluation and model selectionResearch organized around a behavior to improve.
Our programs connect a concrete domain problem to its data, training intervention and evaluation design.
Tool-use policies
Train agents to gather evidence and complete tasks in an environment with explicit outcomes and operating constraints.
Efficient specialization
Evaluate model size, adaptation and serving choices against the quality and runtime requirements of a particular workload.
Work with the lab
Bring us a research problem.
Tell us where the model falls short and what better performance would mean for your team.