The task before the tool

I want AI to help me give clients better choices, early enough to use them. That begins with understanding what they want to achieve. The questions I pursue should help them decide what to do next.

This gives me a starting point for choosing tools. As the inquiry develops, a tool may make a previously unaffordable question worth pursuing. What we discover may change the task itself. The client’s purpose gives that exploration direction.

The disputed date

Consider a manufacturer whose main supplier stops delivering a component. Operations wants deliveries resumed; finance wants to stop paying; management wants to preserve the relationship. I need to help the people responsible weigh those aims and agree what matters most now.

Suppose the immediate aim is to obtain replacement supply while preserving a worthwhile claim. We need to know how long stocks will last and how soon another supplier could deliver. Those facts connect the legal position to the practical choices: seeking damages, buying elsewhere or negotiating a way to keep production running.

An email then suggests that the purchasing manager accepted a later delivery date. Establishing its significance requires the whole exchange, the contract terms and an account of the manager’s authority. I would build a record that distinguishes what was written, what someone recalls and what we infer, with each important proposition linked to its source.

That is what an excellent dataset means in a matter: evidence suited to the decision, with its origins and gaps visible.¹ The applicable law and commercial context belong beside the documents. If an exchange refers to a conversation, speaking to the purchasing manager may be the most useful next step. Better organisation of the available evidence helps us see what else to obtain.

An effective change to the deadline would alter the basis of the demand; a rejected extension could strengthen reliance on the original date. Either finding helps us choose our next move before substantial effort goes into a position. Early case assessment brings the merits together with costs, timing and prospects of recovery.²

Meanwhile, suppose replacement supply cannot arrive before the stock runs out. We need to establish whether the existing supplier can resume production. If a cash shortage is preventing it from buying materials, a staged payment arrangement with suitable protections might serve both parties. Its prospects depend on evidence that the supplier can actually perform. The inquiry may lead the client to change its immediate aim from replacing the supplier to keeping it in production.

A tool with a reason to exist

The disputed date gives a tool a precise job. An existing document system may suffice to find the exchange. Where dates and amendments run through a large record, a dedicated chronology could extract them, link each to its source and keep a proposed date distinct from an agreed one. We could automate extraction while reserving the legal effect of an exchange for assessment. How much to automate and whether to build a specific tool are separate choices.

The same inquiry can improve future contracts. If a replacement supplier needs drawings or tooling held by the original supplier, effective access becomes a commercial requirement. The company’s standard can specify the protection it needs and the alternatives it will accept.

For each task, I would keep asking whether the system can perform it well enough and whether automation improves the service. A working demonstration gives us something to investigate; the client’s benefit gives us a reason to proceed.

The task may call for an existing product, a configured workflow or its own code. Where inputs and rules can be specified precisely, a small program may be easier to inspect and test. A model might extract candidate dates while code checks their sequence against an agreed specification. Before building, I would agree with the client what improvement would justify the work and how much to spend establishing it.

What would count as better?

Public benchmarks can help identify candidates. A private evaluation set would address the work we intend to give them: suitable, authorised cases with reference assessments prepared and checked by qualified reviewers. It could test whether a method finds the disputed date or the missing right to tooling, and whether the evidence supports its answer.⁵

For an early assessment, the reference must reflect what could be known at the time. A sound decision can turn out badly; a later judgment brings information the adviser may never have had. Where competent reviewers reasonably disagree, the test should preserve the range of conclusions the evidence supports.

A private benchmark takes expert time and needs upkeep as documents, standards or systems change. The investment is easier to justify when it supports repeated use or protects against consequential errors. For a modest one-off task, direct checking may be proportionate. The testing should be sufficient for the reliance proposed; a handful of difficult examples can expose failures while leaving substantial uncertainty about dependable use.

That judgement needs an owner and occasions for review. I would agree who is responsible for the task, who maintains its tests and who can change or suspend its use. A new company standard, an unfamiliar contract or a changed client objective may alter what a good answer requires. We should then review the task and the benchmark together, before deciding whether the existing tool still has a useful role. Periodic checks of work in use would help us notice changes we had not anticipated.

I can imagine a more continuous service growing from this work. The supply dispute could leave the company with a clearer record of its dependencies, a revised contract standard and tested checks for future agreements. When the standard changes, a targeted review could identify existing contracts that need attention. Each new inquiry could begin with useful work already done, provided the material is authorised for reuse, maintained and checked for its fit with the matter.

Some of that work could move into the client’s own organisation. A purchasing team could use tested checks directly, referring uncertain or consequential departures for advice. Helping the client handle recurring decisions itself would be part of the service’s value.⁷

If AI makes that preparation and review economical enough, clients could justify inquiries they would otherwise forgo. The gain would be the chance to address a dependency while the supplier is still delivering and there is time to arrange alternatives. That is a direction I would like to work towards; its value would have to be demonstrated in the decisions it helps clients make.

The client would receive a clearer account of the choices, the evidence behind them and the next useful step. For the manufacturer, that might be proceedings or a safer arrangement with its supplier. The purpose of the work is to help it choose while there is still time to secure the next delivery.

Notes

1. Timnit Gebru et al., “Datasheets for Datasets”, Communications of the ACM 64(12), 86–92 (2021). Published metadata and the authors’ proposal were consulted. The authors ask how a collection was assembled, what it omits and how it was transformed. Applying those questions to legal evidence is an analogy, not a validated procedure. They neither turn the evidence into a training dataset nor establish its completeness or truth.Back to text
2. CPR Institute, Early Case Assessment Guidelines and Corporate Early Case Assessment Toolkit (2010 edition), especially pp. 7–11, 14–20. Professional guidance, not causal evidence that early assessment improves outcomes. Its US procedural examples are not presented here as Swiss requirements.Back to text
3. Linda Babcock, George Loewenstein, Samuel Issacharoff and Colin F. Camerer, “Biased Judgments of Fairness in Bargaining”, American Economic Review 85(5), 1337–1343 (1995). An incentivised experiment with 94 student pairs using the same case materials. It tested when participants learned their assigned side, not the effectiveness of an AI review process.Back to text
4. Yuta Koreeda and Christopher D. Manning, “ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts”, Findings of EMNLP, 1907–1919 (2021). The dataset uses 17 propositions across 607 non-disclosure agreements. No claim about current model performance is drawn from the paper’s historical results.Back to text
5. Anthropic Engineering, “Demystifying evals for AI agents” (9 January 2026). Vendor-authored engineering guidance on evaluations with checked reference assessments; not independent evidence that the legal method proposed here is effective. The further engineering sources and proposed protocol appear in the companion note.Back to text
6. Varun Magesh et al., “Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools”, Journal of Empirical Legal Studies 22 (2025), especially §§4, 6–7; author-hosted full text. These historical findings concern particular US legal research products. The study does not evaluate Harvey, Legora or the proposed contract and chronology tools.Back to text
7. Richard Susskind, conversation with David B. Wilkins, The Practice, Harvard Law School Center on the Legal Profession (September/October 2026). Susskind distinguishes improving existing processes from creating new ways to meet legal needs, and emphasises empowering non-lawyers and avoiding disputes. These are his arguments and expectations. The proposed purchasing-team use is my application and would require its own evaluation.Back to text

The manufacturer, supply dispute and proposed tools are hypothetical illustrations. The argument sets out a view of how legal work should be organised. It does not report a tested working method, an existing private evaluation set or a demonstrated litigation advantage.