Domain specialization
Post-training for technical review.
An expert notices the revision that changes a conclusion, the finding that contradicts a specification, or the evidence that is missing. We work with enterprise teams to make those distinctions teachable, then test whether an adapted model can carry them into new cases.
The research question
Can a model apply the review standard to an unfamiliar case?
Source evidence paired with an expert’s decision.
Keep the base weights frozen. Learn a small adapter.
Identify the issue, cite evidence and flag missing information.
Low-rank adaptation learns a weight update, ΔW = BA, while the base weights remain frozen. Domain examples define the target behavior; independent documents test whether it transfers.
The engagement
From task to training experiment.
The training example is a complete case.
A document collection does not, by itself, describe how an expert reaches a decision. We assemble cases that pair a task with the available evidence and a reviewed assessment. The assessment identifies the finding, the source that supports it and any question left unresolved. Before training, related documents and revisions are grouped into the same split so a familiar case cannot masquerade as generalization.
Adapt the behavior. Keep the evidence current.
We establish a prompting-and-retrieval baseline, then test supervised fine-tuning on reviewed cases. Preference optimization can address choices that demonstrations leave ambiguous: which evidence supports a stronger conclusion, when a qualification is necessary, or when another source is required. Retrieval continues to provide the documents for each request, with access and revision controls. The model learns a way of reviewing; the index remains the source of current evidence.
Evaluate the judgment behind the answer.
The candidate and baseline review the same unseen document families. Expert rubrics distinguish a supported finding from a plausible explanation, and distinguish useful deferral from refusing every difficult case. We inspect missed contradictions and grader disagreements alongside the aggregate comparison. A candidate reaches deployment review only when the task gains survive these checks and the serving configuration meets the workload's constraints.
Evaluation design
What the experiment must establish.
- Evidence support
- Assess whether the cited source supports each material claim and whether the model uses the applicable revision.
- Judgment under uncertainty
- Include contradictory and incomplete records. Measure missed issues, unsupported conclusions and useful coverage together.
- Generalization
- Hold out related document families before creating training pairs; compare with the same prompting and retrieval baseline.
- Deployment fit
- Recheck quality at the deployed precision and context limits, then measure latency, memory use and review effort.
Delivered artifacts
A body of work your team can build on.
- A reviewed case corpus with provenance, labeling guidance and an independent evaluation split.
- A reproducible training recipe and candidate adapter or checkpoint, subject to the base model's license.
- A comparison report with expert judgments, failure analysis and a recommendation to release or iterate.
- A deployment design connecting the model to current evidence inside the agreed data boundary.
Technical foundations
Further reading.
Explore another program
Reinforcement learning for analytical agents.Work with the lab
Bring us a research problem.
Tell us where the model falls short and what better performance would mean for your team.