A usage dashboard can look healthy while lawyers spend their afternoons repairing AI output. The system records an answer. The lawyer still has a matter to finish.
In our AM100 law firm engagement, roughly eight weeks of engineering delivered private AI that eliminated about $1.7M a year in cloud-AI expense. We also changed the experience that had left attorneys redoing the work. In the corporate practice, outputs needing edits or a redo fell from 83% to 24% at the follow-up about three months after launch.
Here's how to investigate stalled adoption in one legal workflow, make the output easier to check, and give the team a practical way to keep improving it.
Watch what happens after the answer appears
Choose one practice and one recurring task. For a corporate group, that might be finding a relevant precedent and preparing a note for the lawyer reviewing the matter.
Ask regular users and people who've stopped using the tool to walk through recent examples. Include work they completed without AI. Follow each task until the responsible lawyer accepts the result.
Record the request, sources opened, corrections, time spent checking, and any wait for a senior decision. Ask the lawyer to identify the point where the tool stopped helping.
Different failures need different fixes. A missing source points toward retrieval or access. A wrong citation points toward verification. A useful answer trapped in a separate interface points toward integration. Repeated disagreement about what a good answer contains calls for the practice to settle its standard.
Give the practice a shared definition of useful output
Have the responsible lawyers define what they need to accept the task. For a precedent note, that could mean the issue addressed, the relevant source, the exact supporting passage, the reason it may apply, and the questions still requiring judgment.
Collect examples that meet the standard and examples that fail it. Include situations where the right response is to ask for missing context or say the sources aren't sufficient.
In the AM100 engagement, we tested against historical matters the team understood. The system worked from information available before the deal and couldn't see the test matter's own documents. That gave the lawyers a more useful basis for evaluation than a demonstration that could reproduce an answer already in its inputs.
Use some examples to improve the workflow, then keep others unseen for evaluation. Have reviewers resolve disagreements about the expected answer before using those examples to score the system.
If you’re a law firm managing partner and want to grow your firm without making partners carry more work, we’ll identify the biggest bottleneck standing in your way, build the plan to remove it, and implement it in eight weeks* or less - guaranteed or we continue to work for free and you don’t pay.
Find my bottleneck* Eight-week delivery, or twelve weeks when substantial data foundations are required, as agreed before work begins. If a covered stage fails its agreed benchmarks, we provide up to four additional weeks of remedy work at no charge, then waive or refund that stage’s fee if it still fails. Successfully delivered stages remain payable. Agreed client-caused delays pause the clock. Delivery guarantee terms.
Make each AI answer easier to verify
Build the workflow so the supporting evidence arrives with the answer:
- Check the user's access before retrieving documents.
- Retrieve relevant passages from the permitted collection.
- Ask AI for a structured answer tied to those passages.
- Check that each cited source exists, the reference resolves, and the quotation matches.
- Show unresolved questions beside the proposed answer.
- Give the lawyer a clear way to accept, correct, or reject the result.
A useful instruction is:
Use the supplied sources to prepare a precedent note. Identify the issue, cite the document and passage supporting each material point, explain possible relevance, and list missing context. If the sources don't support a point, leave it unresolved. Keep the source quotation separate from your interpretation.
The code can check source references and quotation matches. The reviewing lawyer decides whether the reasoning applies to the matter. Put that review in the team's existing workflow so checking an answer doesn't become another administrative task.
These were central design choices in the AM100 system: access before retrieval, checkable sources, and the ability to decline an unsupported answer.
Work through the failure that stops reuse
For example, a lawyer asks for a precedent note about a transaction issue. The system returns a polished paragraph and a document name. The lawyer must search for the document, find the passage, and work out whether it supports the claim.
Change the output. Show the issue, a link to the exact passage, the quotation, the proposed interpretation, and any missing matter context. The lawyer can now start by checking the relevant evidence.
If the source isn't available to that user, the workflow should route the request for permitted material. If the quotation can't be found, it should flag the answer for correction before presenting it as supported.
Capture the failure in the review queue with its source and the correction the lawyer made. A knowledge-management owner can then decide whether to change the source collection, instructions, fixed checks, or evaluation examples. That turns feedback into a maintainable process.
Measure completed work and voluntary return
During the pilot, track the time to an accepted result, correction effort, source failures, and tasks abandoned after an AI attempt. Compare similar work, and record changes in matter mix or staffing that affect the result.
Track use alongside those measures. Define which lawyers and tasks are eligible, what counts as use, and whether it is required. Keep those definitions consistent so the practice can interpret the trend.
A workflow that produces more answers but leaves the same checking burden needs another pass. Inspect the examples that caused the burden and decide which change to test next.
In the AM100 corporate practice, the innovation lead removed the usage mandate at launch. Lawyers could choose whether the system helped. At the follow-up, the reduction in rework gave leadership a concrete way to judge that usefulness.
Give someone time to keep the workflow useful
Choose owners for the source collection, access changes, evaluation set, user feedback, and technical operation. Plan who handles each failure and which checks must pass before a change reaches the practice.
Our first handover plan asked too much of busy attorneys. We changed it so knowledge-management staff and named champions became the main workflow builders. The firm also adopted a credit of up to 100 hours per attorney each year for approved build and adoption work. That gave the improvement work a place alongside billable targets.
If the next decision is which workflow deserves investment, use the project-selection guide. If a missing capability requires a new tool, use the build-or-buy evaluation.
Want to see how roughly eight weeks of engineering produced a system lawyers chose to use and removed about $1.7M in annual cloud-AI expense? The full AM100 case follows the diagnosis, build, and handover.
If you’re a law firm managing partner and want to grow your firm without making partners carry more work, we’ll identify the biggest bottleneck standing in your way, build the plan to remove it, and implement it in eight weeks* or less - guaranteed or we continue to work for free and you don’t pay.
Find my bottleneck* Eight-week delivery, or twelve weeks when substantial data foundations are required, as agreed before work begins. If a covered stage fails its agreed benchmarks, we provide up to four additional weeks of remedy work at no charge, then waive or refund that stage’s fee if it still fails. Successfully delivered stages remain payable. Agreed client-caused delays pause the clock. Delivery guarantee terms.
