Why partners still correct AI drafts that use your firm's precedents

Find the correction that keeps coming back

Start with one document type and the first drafts your team sends for review. Compare those drafts with the partner mark-ups. For each substantive correction, ask what information would have let the associate get that position right before review.

Some corrections restore a firm position that was already agreed. Others respond to a client's instructions, a new fact, or a negotiation. Some expose a disagreement between partners about what the standard should be. Give these different reasons their own entries in the review log.

That distinction decides where to spend time. Repeated corrections to an agreed position are candidates for a drafting check. Missing matter facts need better intake. Unresolved partner preferences need a decision before someone encodes them. If most corrections depend on new deal information, a larger precedent library won't address that cause.

In the business-law engagement, we captured positions from prior mark-ups and closed deals, wrote them down, and had the firm's partners approve them. We configured existing tools first, then built the missing check around those standards. The firm's lawyers retained the decisions about the current deal.

The follow-up found that 78% of first drafts passed without partner overrides on standard firm negotiating positions, up from ~18%. Partners still made other changes required by each deal. The two studies each covered one working week in the same group, using the same instrument and work categories. Staffing and deal mix stayed stable; the follow-up week carried more concurrent matters.

That gives your team a useful starting question: which recurring partner corrections should an associate be able to resolve before submitting a draft?

Make one approved position usable

In our conversation with David Rowley, he described a product-development principle: involve the people who will use the work in defining a good outcome and measuring whether it was achieved. Applied to drafting, our recommendation is to have partners define the standard and the review test before the team automates the check.

An old deal records a choice made in a particular context. A partner-approved position explains when the firm wants that choice used again. Keep both the approved instruction and its source, so a reviewer can inspect why a suggestion appeared.

Use a small position card for each recurring issue. Here's an illustrative card with neutral placeholders your lawyers can replace with their own approved content.

Keep the wording specific enough that two reviewers can apply the same test. An instruction to use the firm's usual approach leaves the decision unfinished. If partners disagree, record the disagreement and resolve it before the card becomes an active standard.

Give the AI a bounded drafting job

The check needs three inputs: the current draft, confirmed matter context, and the approved positions that apply. Source documents should be available only under the firm's existing permissions. In our engagement, documents stayed in the firm's document system and its security team applied access rules before retrieval.

Your implementation should select current, approved cards using fixed fields such as document type and represented role. If a required fact is missing or two cards conflict, return a question for the lawyer. Don't ask the model to settle an unresolved firm policy from whichever old document it finds first.

Use AI for the interpretation work: locate the relevant draft passage, compare its meaning with the approved position, and explain the difference. Ask it to return the draft passage, position ID and version, approved wording, source reference, proposed change, and any question that needs an answer.

Keep mechanical checks in code. Confirm that the position ID exists, its version is approved, the cited passage can be found, and required output fields are present. These checks establish that the references and records are valid. The reviewing lawyer still decides whether the proposed interpretation fits the matter.

For each applicable position, require one of three outputs: the draft appears to follow the position; the draft appears to depart from it; or the check can't decide from the available information. A missing clause should follow the partner's rule for that issue. The system shouldn't silently treat an absent passage as a pass.

Return suggestions inside the drafting and review process the team already uses. The associate checks the evidence and edits the draft. A matter partner decides on exceptions. Store the decision and its reason with the draft version, so a later review can distinguish a missed standard from an approved departure.

Work through a draft and an exception

Here's what that looks like. An associate prepares Document Type A for a matter with Role R. Both conditions on POS-014 are confirmed. The draft contains Position B, borrowed from an earlier deal, while the current card specifies Position A.

The check returns a compact review record:

The associate checks the passages and notices a client instruction that supports asking for an exception. The partner agrees to Position B for this matter. Record that approval against the matter and leave the default card unchanged.

That matters for the next draft. A negotiated exception should remain attached to the circumstances that justified it. The knowledge owner can collect recurring exceptions for a partner review, but only an approved change should create the next version of the standard.

If the represented role were missing, the correct output would instead request that information. If the user lacked permission to read the source, the workflow would need an authorized reference or a permitted reviewer. A plausible suggestion without those inputs doesn't complete the check.

Test the positions without giving away the answer

Choose past matters that the partners can assess, including ordinary drafts, valid exceptions, missing facts, and cases outside the card's scope. Reserve some examples for evaluation after the team finishes writing the cards and prompts. If you use every example to tune the system, you lose a useful test of how it handles other work.

For each historical test, use source material that was available before that matter. Exclude the completed test deal and its mark-ups from retrieval. Where a position card incorporates a decision learned from the held-out matter, use an earlier version or a different test matter. Otherwise, the answer has leaked into the test through the card even if the final document is absent.

Our business-law check was tested against known closed deals with retrieval restricted to earlier deals. The partners accepted, corrected, or rejected the suggestions. You can use that approach to build an evaluation set for your own drafting rules.

Score each position separately. Did the check select an applicable card? Did it miss a departure? Did it flag a valid exception as wrong? Did the source support the suggestion? Did it ask for missing context? Have partners record the expected answer before seeing the system's suggestion, then investigate disagreements rather than treating every difference as a model failure.

Also measure the work around the output. Track active associate checking time, partner correction time, and repeat passes on comparable documents. Keep waiting time separate. A check that catches more departures but adds a long verification task may need a simpler output or fewer positions in scope.

Define the release threshold with the practice leader before testing. Require applicable sources and resolved review ownership, set an acceptable miss rate for each class of issue, and decide which failures stop the pilot. Averages alone can hide a repeated failure on the position the partner cares about most.

Put the approved route into daily work

Start with a small set of recurring corrections in one document type. Assign a partner to approve the positions, a knowledge owner to maintain them, a technical owner to manage the workflow and access checks, and an operating sponsor to protect time for the test. One person can hold more than one role, but each responsibility needs an owner.

Check what your current drafting software can already do before commissioning a build. For example, Pactly's position reference describes position descriptions, suggested clauses, examples, and published versions. The buying question is whether a product can run your cards, permissions, review output, and evaluation adequately. A feature list alone won't answer that.

If partners haven't agreed on the positions, start with a shared approved reference and manual review. If existing software passes the workflow test, configure it. Consider a custom component when a demonstrated gap remains in source access, comparison, or review routing. Defer unrelated tool changes while you test that gap.

During the pilot, log each override as a missed standard, valid matter exception, missing input, outdated card, or interpretation error. The knowledge owner reviews the pattern and proposes changes; partners approve changes to the standard. Re-run the affected tests before releasing a new version, and retain a way to return to the previous approved version.

Keep the business measures separate. Reduced review time creates capacity. Retired subscriptions remove expense. New revenue or better matter margin needs its own evidence. Record software, support, and maintenance costs alongside any savings so the next spending decision rests on the full operating picture.

Give partners fewer routine corrections to make

The first useful step is to take one recurring correction, write its applicability and exception rules, and have the responsible partner approve the card. Then test whether an associate can use it to submit a better first draft. That result tells you what the AI workflow needs to reproduce.

In the business-law case, bringing approved positions into drafting helped reduce median partner review to 20 minutes per primary deal document. The firm also removed ~$250K in annual AI-tool expense. The improvement came from making the firm's judgment usable before review and putting the resulting check into daily work.