AI Progress Beyond Model Launches

AI Progress Beyond Model Launches

On October 12, Z.ai released GLM-Legal, a language model with additional legal training. Its headline result was a 19.2% pass rate on the Legal Agent Benchmark, a set of legal tasks developed by Harvey. That put it seventh of 79 models on Vals AI's leaderboard, at $0.15 per task.

A 19.2% pass rate means it didn't pass the remaining 80.8% of tasks: more than four out of five, yet it still came seventh. The ranking sounds better than the pass rate. Neither tells you much about whether the model will handle your work, which is the problem I discussed in Before You Buy Legal AI, Build the Exam. Yet the next model launch brings another round of benchmark charts and excited posts.

Meanwhile, some of the developments I find more interesting get far less attention in my LinkedIn or X feed. They address problems I run into when building legal workflows: too many documents to examine, slow classification, search that misses an important qualification, or tax returns whose complex layouts don't survive text extraction.

Recursive language models (RLMs), for example, offer a way to work through document collections too large for a model to read in one go. Decision models make choices without writing an answer each time. Late-interaction search allows more detailed matching between a question and a passage. And document extraction is improving at preserving layout, tables, and references to the original page.

In this "look behind the hype" post, I want to explain how these approaches work, where their limits remain, and why they deserve your attention. Let's start with the pile of documents.

RLMs: Working Through Large Document Collections

Reviewing a large document collection takes more than finding one useful passage. Upload everything, and you run into the model's context limit. Summarize first, and the summary may drop the detail that mattered. Retrieve a handful of relevant passages, and nothing guarantees you've covered what a broader review requires.

The context window is the model's scarce working space. Instructions, source material, intermediate findings, and conversation compete for it. Think of a reading desk: the model works with what's on it.

Despite the name, an RLM isn't a model but a way of organizing a model's work, and it doesn't depend on any particular one. It puts the documents in a filing room next door: a separate programming workspace, with material stored in variables, named containers holding text or other data. A variable can hold an entire data room without its contents appearing in the model's context. At first, the model knows where the data is but hasn't read any of it.

Rather than loading the entire collection into its context, the model writes small commands to explore the data: list documents, search for a phrase, return clause 14. The program runs them on the stored text and sends back the results. Those results land on the desk; the rest stays in the filing room.

The recursive part lets the model hand bounded tasks and selected material to subagents: further calls to the same model or a different one, each working at its own desk. A subagent whose task is still too large can divide it again. I've covered tool use and delegation in What Agents Need (And What They Actually Are).

Picture a diligence review. The coordinator assigns supply agreements to one agent, which delegates termination rights and exclusivity to further subagents. That agent combines their findings before reporting back. Findings travel up the hierarchy. A partner running a diligence team works the same way: she needs the issues list, not every associate's reading notes.

Alex Zhang, Tim Kraska, and Omar Khattab introduced RLMs at the end of 2025. DSPy, the framework behind the agent in What Your Agent Is Doing When You're Not Watching, has its own RLM module, and it's what I've mainly used to understand the mechanics. Prime Intellect's Prime Agent, a coding agent released in August, is built around the idea, and Harvey has tested the approach on M&A diligence.

The model still needs a sensible review plan: tasks that keep amendments with their agreements, and a record of which documents have actually been examined. Every extra layer of delegation adds cost and delay, so it needs to earn its place.

For document-heavy legal work, the appeal is straightforward. The model can examine a large collection in manageable parts, delegate detailed reading, and bring the findings together without squeezing every document and every intermediate step onto one desk.

Decision Models: Classifying Instead of Generating

Jev is TypeSafe AI's first publicly released decision model, part of what it calls System One Models. Jev has attracted some launch hype of its own. So let's look behind that too: what makes a model designed for quick, structured decisions useful?

Classification has existed for decades. Sorting documents by type, routing work to the right next step, and scoring results against criteria have always been useful jobs for classifiers. Generative models can do them too, but a model built to write text spends time and computing power on every decision. A traditional classifier is fast, but someone has to train it for each task. Jev aims to handle different classification tasks based on the information and questions you give it, without a separately trained classifier for each one.

The System One name comes from Daniel Kahneman's Thinking, Fast and Slow, which distinguishes fast, intuitive System 1 thinking from slow, deliberate System 2 thinking. Jev returns a choice from a defined list, a score on a rubric, or the probability that a statement is true. Choices and scores come with confidence values, not prose. TypeSafe reports response times between 70 and 500 milliseconds.

One example from my own work is comment review. In my document-review workflows, every comment on a draft gets one next action: resolved, needs client clarification, needs an explanation, or needs a revision to the document. Smaller language models could make that call, but each call added delay, and the delays added up across comments and documents. Running those calls in the background meant users could keep working while the model classified the comments. They still had to wait for the updated statuses.

For that job, I need a status and a confidence score, not a written explanation for each comment. That makes comment review an obvious candidate for a decision model. So is first-pass triage, such as flagging contracts that may contain a change-of-control provision, or documents that may be privileged, for a lawyer to review. Sending these decisions to a small model is the routing I described in Own Your AI: hard reasoning goes to a capable model, classification to something small, fast, and cheap.

To use one well, you still need clearly defined categories and checks against examples from your own work. A reported confidence of 0.9 isn't an established 90% accuracy rate. The savings depend on the workflow, and faster answers alone won't fix every cause of a lagging interface.

The same economics apply to evaluation. A typical rubric in Harvey's training cases, which I covered in last month's post, reportedly has around fifty pass-or-fail criteria, and some have hundreds. Paying a generative model for each of those judgments adds up. With a reliable decision model, those checks could become affordable enough to run whenever you change a prompt, a tool, or the model doing the work.

Late Interaction: Matching More of the Detail

In a common retrieval-augmented generation (RAG) setup, the documents stay outside the model's context. A search retrieves passages that look relevant, and the model receives them when it answers. RAG handles large collections too, and an RLM can use retrieval as one of its tools. The two differ in how the material gets examined. Late interaction improves that search step in either setup.

Many search systems compress each searchable passage into one numerical representation, an embedding, and compare it with an embedding of the query. In effect, each passage gets one catalog card. Take an illustrative case: a query asks whether the seller owes an indemnity for tax liabilities. One clause creates the indemnity; the next excludes taxes arising after closing. On their catalog cards, both clauses are about the seller, indemnification, and taxes, so they can look broadly similar despite a legally important difference. Keyword search alongside meaning-based search, and a second pass that re-sorts the results, still help with problems like this one.

Late interaction keeps multiple representations per passage, usually one for each word or word piece, each shaped by the words around it. Instead of one catalog card, the passage gets an index entry for every word. At search time, the system compares each part of the query with those entries and combines the best matches into a score for how relevant the passage is. Words like "excluding" and "after closing" get more of a chance to count.

The idea isn't new. ColBERT, a research model from 2020, introduced it. What's current is the work built on it. On October 7, Perplexity released pplx-embed-v2-late, a family of late-interaction models for text that can also search page images directly. In July, LightOn released mLateOn, a multilingual late-interaction model whose training covers German. For German legal documents, that second detail interests me more than any leaderboard position.

In Build the Exam, I said the first model I'd consider fine-tuning is an embedding model for German legal language, because a general one may place a GmbH and a GmbH & Co. KG close together. Before doing that, I want to know whether finer-grained matching already handles German legal and tax documents better, even when they look little like what the model was trained on. A general retrieval benchmark won't answer that question. Only tests on those documents will.

Finer-grained matching has a price. An entry for every word instead of one card per passage means a much larger index for the search to run against: more storage, more memory, and more operational complexity. Compression and specialized indexing reduce the burden. More detailed matching also doesn't guarantee that the right passages come back, or that anyone reads them correctly. Whether it's worth the price depends on the material, the accuracy you need, and the resources you have.

OCR: Preserving More Than Text

We've had OCR (optical character recognition) for decades, so it's easy to treat it as something you set up once and forget. I think that's a mistake in legal workflows. In The Structured Data Mindset That Unlocks Legal Automation, I wrote that you can spend months refining document parsing alone. The tools haven't stood still since. Better extraction can give every later step better material to work with, even if the model doing the analysis stays the same.

Take a financial schedule. If extraction separates amounts from their row labels, the model has to work out which number belongs where. The same problem appears when text from two columns gets mixed together or a footnote ends up in the middle of a clause. With better layout recognition, reading order, and table extraction, the model doesn't have to guess.

There's plenty happening in OCR. LightOnOCR-3, released on October 8, is one example: its authors say it returns the position of each region on the page, describes images, and turns charts into tables. Those positions matter more than they sound. With them, a value can point back to the exact spot on the page it came from.

Datalab is another provider I follow closely, and I rate its OCR models highly. It recently released Indra OCR, focused on fast processing without a graphics processor, and Chandra 2.5, which adds structured descriptions of figures and charts alongside faster processing. These are different kinds of progress: getting through more pages on ordinary processors, and getting more usable information out of each page.

If you chose your extraction tool a while ago, it's worth looking again. Not every document needs OCR, and some models can now work with page images directly. Compare what the tools preserve on your own documents, along with their speed and cost.

Putting It Together: A Diligence Review

Picture a data room full of supply agreements, financing documents, employment contracts, and financial schedules. The deal team has a checklist of issues to look for. Here's where I'd consider the approaches above, depending on which part of the review needs improvement.

Some of the documents are scans, and the financial schedules contain tables with amounts, dates, and footnotes. Before any legal analysis, you need usable text. Better OCR and layout extraction could keep those amounts beside the right labels and preserve the reading order. That saves the reviewing model from having to reconstruct a table from a jumble of numbers.

Then there's the job of sorting the collection. A decision model could classify documents by type and send them to the appropriate review task, with uncertain classifications going to a person. That becomes useful when the collection is large enough that repeated classification calls take noticeable time. Documents marked out of scope would stay on the document list, so the team could check those decisions rather than lose sight of the documents.

The reviewing agents also need to find relevant provisions and the firm's guidance on how to assess them. Late-interaction retrieval is one option to test when a search finds passages about the right topic but misses the specific wording that matters, such as a consent requirement triggered by a change of control. The search results need to identify whether a passage comes from a target-company contract or the firm's playbook. One is evidence; the other tells you what to look for.

Finally, the whole collection may be too large for one model's context window. An RLM setup would let a coordinating model inspect the document list, divide the review into manageable assignments, and combine the findings. The assignments still need to keep amendments with their agreements, and the coordinator needs to consider findings across document types. A restriction in a financing document may matter to the supply-contract review too.

What Deserves Your Attention

You don't need to follow every launch to understand where AI is going. Sometimes a new model deserves the attention. Often the more useful progress comes from a familiar technique applied differently.

Before an announcement gets my time, I ask:

1. Which recurring problem does it address?
2. What changes compared with the way people handle that problem today?
3. What evidence is missing for my documents and my workflow?

Run GLM-Legal through them. The recurring problem is cost: legal workflows make a lot of model calls, and $0.15 per task is the number I'd look at, not the 19.2%. What changed is a Flash variant of GLM-5.3 with extra legal training. The missing evidence is everything outside Harvey's benchmark, starting with the documents I'd actually run it on.

If you're buying rather than building, ask the vendor:

• What happens when the data room doesn't fit in the context window, and how do I know every document was read?
• Which steps use a full model, and which use something smaller and faster?
• How does your search handle an exception sitting right next to the obligation it limits?

Knowing roughly how these systems work also makes AI agents easier to work with, as I argued in What Your Agent Is Doing When You're Not Watching. You can see what an agent can or can't reach, what it hands off, and where its output needs checking. The leaderboard won't tell you any of that. Most of what decides whether legal AI works on your documents happens around the model. That's what my next post is about: the skills lawyers need when AI agents change the work.