Testing a legal tool

A useful evaluation gives a tool a justified role in the service: the work it can help with, the evidence for relying on it and the checks it needs. This note develops the testing proposal in The task before the tool. Start with a decision the results can change, and assess the work the client receives, including what it takes to make an AI answer usable.

Choose a test worth its cost

Start with one bounded task and an existing way of doing it. A proposed chronology tool might need to distinguish agreed delivery dates from dates merely suggested in correspondence. State the evidence it will receive, the questions it must answer, the person who will check its output and the failures that would rule it out.

When the client will use the tool directly, include representative users in the evaluation. Assess whether they can interpret the result and recognise when further advice is needed.

For a small, one-off task, checking the answer directly may cost less than building a reusable evaluation. A recurring review process, or a large task with expensive mistakes, may justify more. Budget for preparing cases, expert assessment, investigation of disagreements and future upkeep. Decide who will pay before commissioning that work. A reusable practice capability and a client-specific testing exercise may need different funding arrangements; neither should be treated as free.

A few carefully chosen examples can expose a reason to reject a tool. Passing them does not establish reliability. If credible testing would cost more than the proposed tool could save or protect, the sensible choice may be an existing system or a different method.

Prepare cases that can be judged

Use only material suitable and authorised for this purpose. Keep the evidence together with its provenance, relevant company standards and known gaps. A test of early case assessment must contain only the information and applicable law available at the specified time. The eventual judgment is not automatically the reference answer to an earlier decision under uncertainty.

Have qualified reviewers prepare and check a reference assessment for each case. Record the material issues, supporting passages, acceptable conclusions and circumstances that require more information. Where reviewers reasonably disagree, preserve the disagreement. A preferred paragraph is too narrow a definition of success. Practical evaluation guidance likewise stresses realistic tasks, explicit criteria and checking how the system is graded.¹

Separate a sample of ordinary work from a challenge set designed to expose particular failures. A challenge set might deliberately concentrate missing attachments and conflicting amendments, but cannot then establish how often such problems occur in ordinary work. Report the number and kinds of cases tested. Small sets leave substantial uncertainty; repeating the same cases does not create more independent evidence.

Keep a real test in reserve

Separate cases used to develop the system from cases reserved to assess it. The separation should reflect what you want to generalise to. For unfamiliar matters, keep whole matters apart. For new contract families, keep related templates, agreements and amendments together. Randomly splitting passages may put near-copies on both sides. This applies the principle of grouped evaluation to legal work.²

Privacy does not create independence. If prompts, retrieval rules or code are repeatedly adjusted after inspecting the reserved results, that set becomes part of development. Repeated adaptive use can overfit even a holdout set.³ Keep familiar failures to check that fixes still work, but use fresh reserved cases for a further assessment of unfamiliar work. Local separation cannot prove that a foundation model has never encountered similar material.

Compare the work that gets delivered

State the comparison method, the cases to be tested and the standard for acceptable performance separately. A tool can outperform a weak existing method while still falling short of what the service requires.

Compare the existing method, an available product and the proposed tool on the same task, underlying records and requirements. Count preparation, review and correction, as well as generation time. Record the resources each approach receives and repeat trials where outputs vary. A quick answer followed by a long repair is not a quick service.

A whole-service comparison may properly credit better document preparation, provided its cost is counted. To isolate a model’s contribution, keep the surrounding process fixed. State which question the comparison answers.

Make consequential failures visible

Assess material omissions, unsupported findings, source fidelity, warranted uncertainty and human review effort separately. Check whether citations support the claims made from them; a real source can still be misrepresented.⁴ Test whether the tool asks for missing information when an answer would otherwise be unwarranted.

An average score can hide an unacceptable failure. Missing the amendment that changes a deadline may disqualify a chronology tool for the intended use. Repeated false alarms have a different cost: someone must clear them, and useful warnings may become harder to find. Record those consequences rather than compressing everything into one percentage. Investigate surprising successes as well as failures; a correct conclusion reached from the wrong evidence offers little reassurance.

Put the results to work

Name the person accountable for the service and the person maintaining the evaluation. Agree with the client which changes require renewed assessment and who may approve continued use. The review should consider changes in the task’s purpose, applicable law, company standards, documents, intended users and system. Set a review interval that reflects the consequences and pace of change.

Keep a dated version of the task definition, reference assessments, test cases and acceptance criteria, together with the model and configuration used. When the benchmark changes, record the substantive reason and have a qualified reviewer check the revised assessments. Preserve earlier versions and explain which results remain comparable. Retain applicable cases to check that existing protections still work, and reserve fresh cases for the changed task. A difficult result is a reason to investigate the system or the assessment; changing the standard requires its own justification.

Use the findings to choose a role for the tool: further development, a defined use with specified checks, or another method. Record the work the evaluation covers and what remains to be established. A revised task may call for a different product, a configured workflow or dedicated code, and each choice needs evaluation against the updated requirements.

Sample subsequent work to look for failures the benchmark missed, and identify who can narrow or suspend the tool’s use. This follows NIST’s broader emphasis on evaluation in context and continued monitoring.⁵ Test results concern defined tasks and conditions. Follow whether material issues were found in time to act and whether the client could decide with a clearer account of the options. A later win or loss does not by itself establish whether the earlier advice was sound.

Notes

1. Anthropic Engineering, “Demystifying evals for AI agents” (9 January 2026). Vendor-authored engineering guidance, not independent validation of the proposed legal method.Back to text
2. The scikit-learn documentation on grouped data. Keeping whole matters or contract families apart is the legal application proposed here; the appropriate grouping depends on what the test is meant to establish.Back to text
3. Cynthia Dwork et al., “Generalization in Adaptive Data Analysis and Holdout Reuse” (2015). The authors’ abstract and version metadata were consulted. The paper’s formal methods for controlled reuse are not implemented by simply keeping documents private or saving fresh cases.Back to text
4. Varun Magesh et al., “Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools”, Journal of Empirical Legal Studies 22 (2025); author-hosted full text. The study distinguished correctness from support by cited sources in particular US legal research products. It did not test the tools proposed here.Back to text
5. NIST, Artificial Intelligence Risk Management Framework 1.0 (2023), §3.1; GOVERN 1.5 and 2.1; MEASURE 2.13 and 3.1; MANAGE 2.1 and 2.4. These address review, responsibilities, evaluation of measurement, emerging risks, alternatives and changing or ending use. The framework is voluntary; the detailed legal evaluation and versioning arrangements here are proposed applications.Back to text

This is a proposed approach assembled from evaluation research and engineering guidance. The legal examples and funding choices are proposals; no deployed evaluation programme is reported here.