On August 20, Harvey published the first results from Tenet, an open-weight legal model it developed with Fireworks. The wider research effort also included separate diligence work with Baseten. Four days later, Thomson Reuters launched Thomson-1.0, which also starts from an existing open-weight foundation. Notably, neither company trained a base model from scratch.
A base model is the general-purpose model before someone gives it additional training for a narrower job. Think of it as a very capable new hire who still needs to learn how your team works. Instead of training a base model from scratch, Harvey started with Kimi K3 and trained it further on long legal tasks. Thomson Reuters took the same general route with Qwen3.5-397B-A17B as the foundation for Thomson-1.0-Large.
Harvey also tried different foundations for different jobs. Its research used GLM-5.2 for diligence and document review, later adding Qwen3.5-122B-A10B for another diligence experiment, while Qwen3.8-27B handled firm knowledge. These are research projects that Harvey plans to fold into its products over time, rather than features you can assume are already in the product.
Supposedly, Tenet completes almost twice as many of Harvey's long legal tasks as the Kimi K3 base. That is Harvey's claim, based on Harvey's own evaluation, a point I will come back to. Across its other reported evaluations, the specialized models approached the quality of much larger commercial models at considerably lower cost.
What stands out is how Harvey got there, and the work deserves both credit and a closer look. It did not simply reach for the biggest model on the shelf and hope for the best. Harvey tested different models, used targeted data, changed the surrounding workflow, and trained the model for a defined job. On its own measures, that produced better work at lower cost.
If you have read Own Your AI, this should sound familiar. The model is one replaceable part of a larger system. Harvey's research shows what you need before you can replace that part with confidence: representative legal tasks and a way to grade the result. An attractive benchmark chart is no reason to buy Harvey blindly, and it certainly is no reason to start pricing 150 GPUs. The useful question is how Harvey measured and improved the work, and which parts a law firm can copy at a rather more sensible scale.
How Harvey Built Legal Cases Without a Law Practice
Reportedly, Harvey's training dataset contains roughly 1,750 "task environments." I will call them training cases because Harvey used them to improve the model. Each case contains an assignment written as a partner might give it to an associate, a folder of source materials, and a rubric describing a strong deliverable. One example asks the model to draft a diligence memo from contracts, correspondence, and filings, then grades the memo against dozens of pass-or-fail criteria. It is the senior reviewer's red pen, turned into software.
An eval set uses the same ingredients for a different purpose. It is a fixed collection of representative assignments, source materials, and scoring rules that lets you compare two versions of a system. The training cases help Harvey change the system. Separate eval cases show whether the change helped. Keep testing on the exercises used for practice and even a mediocre student eventually learns the exam.
Harvey built both sides of that machinery. The training cases provide feedback while the model improves. Automated graders compare each attempt with expert rubrics. Cases held back from training then help Harvey decide which combination of model, tools, and workflow performs best for each task.
Harvey, of course, is not a law firm. It could not pull these cases from its own legal practice, and it says it did not use customer data. Instead, Harvey combined synthetic and publicly available legal material with paid human-expert data. Mercor and other partners helped build the expert datasets and review the synthetic material. The firm-knowledge experiment, for example, used roughly 100 million tokens of synthetic client matters rather than a customer's files.
Fair play to Harvey: manufacturing credible legal assignments is expensive and difficult. A law firm starts one step ahead because it already produces instructions, source materials, deliverables, and senior review every day. Turning that raw material into safe training or evaluation cases still requires permission, redaction, structure, and a fair amount of work. Yet the raw material exists. It becomes useful to automation once the firm treats it as structured, reusable information rather than a pile of unrelated documents.
Harvey's cases measure performance on the jobs Harvey wants its system to handle. A public leaderboard cannot tell Harvey whether its diligence workflow improved. LAB, Harvey's Legal Agent Benchmark, can. LAB also sits behind the headline result, though, which deserves a closer look.
Grade Your Own Homework
Tenet completes almost twice as many of the LAB tasks Harvey held back from training as base Kimi K3, and 20% more on LAB Contracts. Impressive, until you ask how closely the exam resembled the lessons.
Harvey trained on cases that, in its words, "share the structure of LAB tasks," then evaluated on LAB cases kept out of training. The exact facts were new, but the assignments had a familiar shape, used similar rubrics, and ran through a similar set of tools. That is a home-field advantage. Changing the facts of last week's training exercise does not create an independent exam when the instructions, source bundle, marking scheme, and workflow remain familiar.
Harvey disclosed this overlap in the appendix, to its credit. The disclosure makes the result easier to understand. It is useful evidence that Harvey improved its system for Harvey's target tasks. It tells us less about legal work in another product, firm, practice area, or jurisdiction.
The evidence outside LAB is more persuasive. Harvey ran Mercor's APEX Agents (Corporate Law) in its own setup, where Tenet beat the Kimi K3 base on tasks it had not seen during training. Mercor's earlier APEX benchmark adds a stronger independent check: Mercor ran and graded Tenet on 100 legal tasks kept from Harvey, without sharing the tasks, runs, or task-level scores until the evaluation was complete. Tenet also improved over the base model on Crosby's Redline Bench.
So what should you take from the next model announcement? Ask who owns the benchmark, how closely the test resembles the training material, and how the model performs on evaluations the vendor does not control. Those questions will tell you more than another decimal place on a leaderboard.
A vendor needs independent benchmarks to persuade outsiders. The team building a system needs an internal eval set shaped around the work that system is meant to do.
Weak as Marketing, Strong as Infrastructure
The LAB overlap is no reason to dismiss Harvey's work. Harvey deserves credit for publishing a serious blueprint for teams building AI internally: define the target work, create representative cases and rubrics, test the model and workflow together, and use the results to improve both. Any firm serious about building its own AI architecture should take a close look and adapt that method to its own workflows.
Against that background, LAB is exactly what Harvey needs internally. Harvey can change a model, a prompt, or the surrounding tools and then run the same cases again. The result tells its engineers whether the work got better, worse, faster, or cheaper.
A law firm considering the product has to read the results from a different position. LAB shows that Harvey improved its system on Harvey's chosen tasks. Whether the same system fits that firm's jurisdictions, practice areas, and ways of working requires evidence from the firm's own cases. Harvey's method deserves attention; its headline scores still need outside evidence before they guide a purchasing decision.
That makes the eval set a durable asset. Base models will come and go. Products will change. Your own cases keep score on the work you care about.
Own your evals.
Post-Training Cut Cost and Improved Task Performance
Post-training adapts an existing model to a defined job. Harvey and Thomson Reuters did not invent the approach. Cursor's Composer 2, for example, started from the open Kimi K2.5 base and was trained further for coding inside Cursor's own environment. Harvey gives us a useful legal example of what the approach can buy.
Harvey's training rewarded shorter answers when two attempts achieved the same quality. That made cost part of the target, alongside the legal result, rather than a bill to worry about afterwards.
The rubrics carry much of the legal expertise. A typical Tenet training case reportedly has around fifty small pass-or-fail criteria; some have hundreds. An automated grader checks the model's work against those criteria. The process makes a senior reviewer's judgment repeatable, although a bad rubric or a bad grader will repeat bad judgment just as efficiently.
The firm-knowledge experiment offers another lesson. Harvey and Engram trained a Qwen3.8-27B model to study a synthetic collection of firm matters. More study improved quality and reduced the amount of text the model had to process for each query. Simply giving a larger model more room to reason cost more and produced smaller gains. The 27-billion-parameter model is far smaller than the largest commercial models. It might even run on a good laptop.
Harvey's latest diligence research with Baseten makes the role of the surrounding system even clearer. A large M&A data room may contain thousands of documents and up to 80 million tokens of text, far more than one model can sensibly read in one go. Harvey's setup works like a diligence team. A coordinating agent divides the data room into manageable review tasks, sends those tasks to other agents, and combines their findings into the final memo.
This type of setup is known as a recursive language model, or RLM, harness. Alex Zhang, Tim Kraska, and Omar Khattab introduced RLMs as a general way for models to work through material too large for one context window, and the approach is beginning to appear in other agent systems. Harvey's experiment is notable because it takes a new idea and tests it on end-to-end legal work.
Redesigning that division of labor produced the largest initial gain. Across seven models on Harvey's 50 held-out synthetic data rooms, the average share of rubric criteria passed rose from 23.3% with the old workflow to 62.4% with the RLM setup. The old setup read no more than 1% of each data room. Most RLM runs came close to reading all of it. In Harvey's experiments, greater coverage was associated with higher pass rates.
Harvey then post-trained the coordinating model inside that workflow. A Qwen3.5-122B-A10B model reportedly went from passing 29.9% of the criteria to 63.0% on the held-back data rooms. It learned to delegate more work, adjust the number of review tasks to the size of the data room, and draft the memo while the other agents were still reporting back.
There is a useful warning in the same paper. Giving the reviewing agents another layer of agents reduced the average score in Harvey's smaller test. More agents created more ways to lose the thread. Architecture matters, but additional complexity still has to earn its place in the evals.
The cost figures explain why all this matters commercially. In Harvey's reported experiments, the firm-knowledge model cut cost per query by 90%. Its Review Table model cost roughly one-tenth as much per cell as the strongest baselines while improving quality on the target tasks.
Tenet's training reportedly required around 150 NVIDIA B300 GPUs over two months, roughly 1,750 training cases, and expert-written rubrics. That is a serious research programme. Your firm probably does not need one.
Improve the System Before You Fine-Tune the Model
Fine-tuning sits at the expensive end of a much longer list. Start with clearer instructions. Improve the source material and search. Change the tools the model can use, the context it receives, and the way a long job is divided. You can also encode recurring processes as skills. These changes are cheaper, easier to reverse, and often solve the problem without retraining the model.
Every change still creates the same practical question: did the work improve?
An eval set gives you an answer. For a personal workflow, start with ten to twenty representative cases from recent work. A system serving a whole practice area may eventually need one or two hundred. Treat those numbers as starting points. The right size depends on how varied the work is and how costly a mistake would be.
Your first eval set can be a spreadsheet with one row per case. Add columns for the instruction, the folder containing the source material, one strong reference output, three to five must-have criteria, critical failures, jurisdiction, practice area, and difficulty. Treat the reference output as an example of good work. Another answer may be better if it meets the criteria. You do not need Harvey's fifty-point rubrics to compare a new prompt, search setup, skill, or model on your own work.
Ideally, the system offers an API, which lets another program run the cases automatically. Compare one change at a time. When real work reveals a new failure, add a case that captures it. Traces plus lawyer feedback can help you find those cases. If you keep adjusting a workflow until it passes the same examples, bring in a few fresh cases before a consequential comparison or purchasing decision. Otherwise you have prepared the system for a familiar exam and little more.
Real client work is not free evaluation data. Confirm that each matter may be reused, minimize and redact personal or confidential data, maintain privilege and the applicable access controls, and decide how long copies will be kept. Use synthetic or public substitutes when real files cannot be reused. Choose cases across the jurisdictions, matter types, difficulty levels, and known failure modes you care about. The cleanest files rarely teach you where a system breaks.
Fine-tuning comes last, if it comes at all. I have not fine-tuned a generative legal model because better instructions, data, tools, and workflows take you a long way. I would want the evals to show that those cheaper options had reached their limit first.
The first model I would consider fine-tuning is an embedding model for German legal language. An embedding model turns text into numerical representations that a search system uses to decide which passages are similar. A general-purpose embedding model may place a GmbH and a GmbH & Co. KG close together because the names and surrounding language overlap. In German company law, and even more in tax law, they live in different universes. Teaching the search model that distinction may improve a legal workflow long before a generative model needs additional training.
Whether you change a prompt, redesign the workflow, or train a model, your eval set tells you if the work improved. It can only do that job if you control it.
The Last Mile Is Portability
Harvey says it wants firms to "own their intelligence." If the model remains inside Harvey's platform, relies on Harvey's infrastructure, and can only be tested there, that ownership still depends heavily on Harvey.
Portability makes the asset useful beyond one vendor. An export should include the task instructions, source files or stable references, rubrics, reference outputs, grader settings, run results, model and prompt versions, and human annotations in documented, non-proprietary formats.
The contract should give the customer the right to run those cases against another model outside the platform. It should also cover export and deletion when the relationship ends.
An eval set built from authorized, representative examples of your work can travel with you. A vendor's manufactured cases can still teach you something, but they will never capture all your jurisdictions, practice areas, and favorite ways for a matter to go sideways.
In Own Your AI, I warned about owning the model while depending on the system around it. Models change. Vendors change. Your eval set lets your judgment travel with you.