A lawyer leaves a deal team, and IT removes their access to the matter. Before you give another practice access to the firm's AI system, check whether that lawyer's account would stop receiving information from the matter and still be able to find the work it needs. You can test both outcomes with two test accounts and two invented matters. Agree on what each account should be able to see, then run the same questions before and after one permission changes.
At one corporate law practice we worked with, the firm eliminated about $1.7M a year in cloud-AI expenses when the private system we built replaced recurring tools. The firm's security team connected its existing matter permissions so that the system checked which documents an attorney could see before it sent any to the model. That gave the firm one place to change who could see a matter. Here's how your risk and IT teams can check that a change there reaches the AI workflow before you expand it.
Give both accounts something they should always find
Ask your risk lead to write down the expected permissions before IT runs the test. In an approved test environment that mirrors the rollout configuration, give two test accounts, Alex and Blair, access to a synthetic General matter. Give only Alex access to a separate synthetic Deal matter. Put one invented memo in each, with a distinct fact in its body and a title you can recognize. Use a plain question about each memo without putting its answer in the question, so you can tell whether the system pulled the fact from the source.
Use the accounts as ordinary users would. Have IT confirm what each account can open in the document system, then ask the AI the two questions and record what it returns. When a memo should be available, look for the test fact and a citation that opens the expected memo. When a memo should be restricted, check that its fact, title, snippet, and citation stay out of the answer and search results. Have an authorized system owner also check which documents reached the model. A clean answer can hide a retrieval problem if the model received the memo and simply left it out.
Save that baseline before changing access. You need Alex to find the Deal memo once to show the test can surface it. You also need Blair to find General while Deal stays restricted, so you can see how the same workflow behaves for an account outside the deal team.
If your firm wants to make more of its work reusable across practices, the baseline shows that the AI can find approved material for a lawyer outside the deal team. That gives you a result to preserve as you change who can see the Deal memo.
If you’re a law firm technology or risk leader and want to grow your firm without opening sensitive matter files to people who shouldn’t see them, we’ll identify the biggest bottleneck standing in your way, build the plan to remove it, and implement it in eight weeks* or less - guaranteed or we continue to work for free and you don’t pay.
Find my bottleneck* Eight-week delivery, or twelve weeks when substantial data foundations are required, as agreed before work begins. If a covered stage fails its agreed benchmarks, we provide up to four additional weeks of remedy work at no charge, then waive or refund that stage’s fee if it still fails. Successfully delivered stages remain payable. Agreed client-caused delays pause the clock. Delivery guarantee terms.
Repeat both questions after you remove one permission
Have the permissions owner remove Alex from Deal and record when the change should take effect under the firm's approved access policy. Keep the documents, questions, and Blair's permissions the same. At the agreed deadline, use fresh conversations and repeat both questions from both accounts. Check the source system and the AI results as you did in the baseline. If Alex can still open Deal in the document system, start with the membership rule there. If the document system denies access while the AI still receives the memo, have the retrieval owner check which permissions the AI used.
Here’s what that looks like in practice. This example uses two invented memos. The change is due to take effect at 10:10. In the table, “returned” means the account could open the memo in the document system, the model received it, and the AI showed its fact and citation. “Blocked” means the document system denied access and the memo stayed out of model inputs and visible results. Alex’s Deal result fails because the memo still reached the model after the document system denied access.
| Test account and memo | Expected before | Observed before | Expected after 10:10 | Observed after 10:10 |
|---|---|---|---|---|
| Alex, General | Allowed | Returned | Allowed | Returned |
| Alex, Deal | Allowed | Returned | Restricted | Document system denied access. The memo reached the model. The visible answer showed no Deal details. |
| Blair, General | Allowed | Returned | Allowed | Returned |
| Blair, Deal | Restricted | Blocked | Restricted | Blocked |
Keep expansion on hold if any restricted memo reaches the model or shows up in the interface. If Alex stops finding both memos, keep expansion on hold then, too. A broken connection or an empty index could make both results disappear, so you need the General memo to come back before silence on Deal counts as evidence. After the owner resolves the mismatch, reset the synthetic permissions and rerun the full before-and-after test. If the system owner can’t confirm which sources reached the model, hold expansion until you can collect that evidence. Clear this check only when all required observations match the approved matrix and the system applies the change by the agreed deadline.
Give risk and IT one record to sign
Copy these fields into your rollout record so the person approving the change can see the agreed rule, what happened, and which workflow the result covers.
| Record | Fill in for your test |
|---|---|
| Approval and scope | Risk owner, IT owner, workflow and configuration, test accounts, and two synthetic memos |
| Before each change | One row per account and memo. Record the question, expected access, and observed document access, model sources, answer, and citations. |
| Membership change | Permission changed, system owner, time applied, and deadline approved by risk |
| After the deadline | Repeat each row with expected and observed results. Attach evidence from the approved test environment. |
| Rollout decision | Hold or clear this workflow for the tested change. Record the approver, any repair owner, and the retest date. |
Before the next expansion, have risk and IT sign the expected outcomes and run the card together. Keep the decision tied to the workflow you tested. A rollout that also exposes saved answers or shared conversations needs separate evidence for those paths, because a new retrieval result tells you only which documents the model can receive now.
If you’re trying to grow your firm, start with your goal, look at what it’ll take to achieve it, and then identify what’s keeping you from taking action. A permission check can show you whether one workflow is ready to expand. If the team can already find approved work and still spends hours rebuilding drafts, look at where that work is coming back before you decide what to change. Our guide to why law firm AI adoption stalls walks through how to study that rework.
If you’re a law firm technology or risk leader and want to grow your firm without opening sensitive matter files to people who shouldn’t see them, we’ll identify the biggest bottleneck standing in your way, build the plan to remove it, and implement it in eight weeks* or less - guaranteed or we continue to work for free and you don’t pay.
Find my bottleneck* Eight-week delivery, or twelve weeks when substantial data foundations are required, as agreed before work begins. If a covered stage fails its agreed benchmarks, we provide up to four additional weeks of remedy work at no charge, then waive or refund that stage’s fee if it still fails. Successfully delivered stages remain payable. Agreed client-caused delays pause the clock. Delivery guarantee terms.