Mecha vs. Robot AI
As you think about how you’re going to implement AI in your law firm (as I’m sure you are), you should develop some opinions about how you integrate AI. There are different ways, and the differences matter. One of the basic questions to ask is: what is the relationship between the human and the AI tools. The first patter to consider is the raw LLM, the “robot”.
The Robot Model
In the robot model, you simply type in what you want and you get an answer. This is the most common way to use LLMs and it has some advantages. The main advantage is simplicity. It doesn’t take any coding or any expert knowledge. Unfortunately, for legal applications there are two big problems with this approach:
- Hallucination. An LLM can just make stuff up (including very real sounding legal citations). If you use an LLM and you don’t catch all the hallucinations, you can get disbarred.
- Bad Logic. This isn’t to say LLMs can’t do logic. They can. It’s just that in the legal industry our logic is often very context dependent (based on law, circumstances, recent court judgements, etc). LLM’s aren’t trained to focus on this stuff and sometimes the level of complexity required for making a legal argument is just too much for them.
Unfortunately, you can’t easily solve these things with prompt engineering. One issue is that if you increase the context length (i.e., the amount of input information) you can actually get worse results!
The Agent Model
One potential way of dealing with the problems of the robot model is to give the AI more tools to work with. For example, you could give it a calculator, or even the ability to look up and verify cases. An AI that can make decisions about how to use a set of tools is called an Agent. Agents are great for many things, but they have some problems:
- Creating an Agent takes work. The specific tools you need might not be available via an off-the-shelf LLM. So you might need a specific AI. The result: it costs time and money to create.
- Agents can still go wildly off course. If your project requires a number of small but important decisions along the way (as legal documents often do), the agent could go very far off course and give you a completely unsuitable answer because it made many small decisions wrongly.
Agents are amazing and they will soon be everywhere, but they may not solve every problem for a legal professional.
The Mecha Model (HITL)
In some cases you simply can’t avoid putting a human in the loop (HITL). But this doesn’t mean you can’t use AI. It just means that you need to be the Agent. The key to a useful HITL AI system is proper division of the task into parts where an AI (or an algorithm) can do the job vs key decisions that a human should make.
For example, when drafting a patent you don’t really want an AI to draft your claims. The issues that need to be considered are too complex and the result is too important to leave it to an AI. So, an AI can provide you with outlines or draft claims, but an experienced practitioner needs to put significant effort into carefully crafting them.
However, once you have made the key decisions, there are many things an AI can do to maximize the impact of these decisions. But when it comes to the next key decision, a powerful “Mecha” model will make an intuitive interface available for the experienced human to intervene, review, and ultimately determine the next course of action.
Mecha models aren’t perfect. They can take even more up-front investment than Agentic models. And they need to be created by people who deeply understand the task at hand — specifically, how the task can be broken down and where the key moments for human intervention are among the subtasks. But even as a lawyer, you can be using AI to leverage your decision making ability into enhanced productivity.