Jonas Hertner — Home
Menu
Search

Part 1 of 2 · · Revised

Learning from our work with AI

Using AI to do better work and develop the judgement to take on the next problem.

I want to understand more after working with AI than I did before. The prospect that interests me is a practice in which today’s work helps me see the next problem more clearly.

Consider a contract-review tool that finds a supplier’s proposed liability limit is usual in comparable agreements. The client wants the deal, and the terms may be commercially sensible. To advise, I need to connect the clause to the business: which losses could a supplier failure cause, what could the client recover, and what would insurance or another arrangement cover? That is the relationship I want to become better at understanding.

It calls for recognising a question worth pursuing and knowing how to investigate it, with sources, tools and other people. Where software reduces the time needed for comparison and checking, I would use the saving to investigate the exposure. Learning should follow the abilities the service requires.

The gain from assistance can be substantial in its own right. In a randomised study of 487 management consultants, access to ChatGPT and training in its use helped them perform data-science tasks beyond their existing skills. On a later test without ChatGPT, the researchers found no evidence that they answered technical questions better than the control group.1 Doing more with a tool and developing understanding that remains available afterwards are achievements we can examine separately.

Existing expertise needs attention too. In 1983, Lisanne Bainbridge described how industrial automation could leave operators responsible for exceptional failures while giving them less occasion to use the skills those failures required.2 For legal work, I take this as a reason to give the abilities we retain a deliberate place in how we organise the practice.

I would begin with a difficulty the work had exposed. If a colleague caught an exposure I had missed, we could use their explanation to devise a short exercise. On a suitable unfamiliar contract, I could identify the loss that concerned me and explain my reasoning before seeing a recommendation. AI could suggest a counterexample or a different set of commercial facts, with the material checked by someone qualified. My colleague would have something specific to challenge, and I would have a chance to revise a view I had actually formed.

Before seeing the feedback, I would also record how confident I was. Afterwards, we could compare that confidence with the errors and sound judgements the discussion revealed.

Returning to the problem later, with my earlier explanation out of sight, would help reveal what I had retained. Experiments with prose passages found that retrieving studied material improved recall after a week more than repeated study.4 Legal understanding requires a further test: can I recognise a different exposure behind apparently usual terms and explain why the advice should change? That is what I would want the exercise to help me do.

The help should develop with the learner. A worked example can give a beginner a way into an unfamiliar problem; independent problem solving can become more useful with experience.5 I would want the tool’s first response to reflect that difference. It could offer an explanation, a hint or a challenge to a view I had already formed, according to what I was trying to learn. We would still need to establish which approach helped in the work itself.

On a live matter, the client may need a checked answer promptly. A separate practice session gives us time to explore a mistake, try another approach and return to a point we almost understood. As an independent lawyer, I would have to make room for that work or arrange it with others. Maintaining the capabilities my service needs belongs in the cost of running the practice. Clients should receive the benefit of work that can safely be eliminated.

Over time, the practice could accumulate useful teaching material: a difficult judgement explained, a misleading comparison corrected, an exercise that reveals a recurring weakness. AI could help turn checked material suitable and authorised for reuse into further opportunities to practise. We could assess its value through what we subsequently understood and handled better on unfamiliar problems.

As methods improve, the division of labour can change. A review that reliably protects the client merits investment; another may be better performed by a colleague, a specialist or a tested system. My learning should follow the responsibilities I undertake. When the next apparently standard clause arrives, I want to see more of what it means for the person relying on my advice.

Notes

  1. Wiles et al., “Generative AI and the temporary upskilling of knowledge workers”, Nature Human Behaviour (2026). The account here follows the published abstract. A null knowledge result is not proof of equal knowledge, or of skill loss. The study did not test long-term changes in experienced lawyers.Back to text
  2. Lisanne Bainbridge, “Ironies of Automation”, Automatica 19(6), 775–779 (1983). An analysis of industrial automation, not an experiment on legal work. Automation and augmentation are compatible; see also Douglas Engelbart’s Augmenting Human Intellect (1962).Back to text
  3. Zana Buçinca, Maja Barbara Malaya and Krzysztof Z. Gajos, “To Trust or to Think”, Proceedings of the ACM on Human-Computer Interaction 5(CSCW1), article 188 (2021). The experiment measured immediate decisions, not lasting learning. Benefits varied across participants and measures. Participants rated the deliberation interfaces as more complex; preference differences between these AI interfaces were not significant.Back to text
  4. Henry L. Roediger III and Jeffrey D. Karpicke, “Test-Enhanced Learning”, Psychological Science 17(3), 249–255 (2006). In two prose-learning experiments, restudy helped recall more after five minutes; retrieval practice did better after a week. Retaining a studied passage is a narrower achievement than recognising a problem in an unfamiliar contract.Back to text
  5. Kalyuga et al., “When Problem Solving Is Superior to Studying Worked Examples”, Journal of Educational Psychology 93(3), 579–588 (2001). The qualitative finding is reported in the authors’ abstract; the full paper was not consulted. For attempts followed by instruction, see Sinha and Kapur’s 2021 meta-analysis: benefits depend on the learners, domain and instructional design.Back to text
  6. Bastani et al., “Generative AI without guardrails can harm learning”, PNAS 122(26), e2422633122 (2025). Nearly 1,000 pupils at one school, over four maths sessions. The tutor combined several safeguards and teacher-supplied material; the experiment does not isolate a single design feature or establish loss of existing expertise.Back to text

Eric C. So’s The Collision: What AI Does to Us (2026) prompted this inquiry. The discussion draws particularly on his distinction between useful effort and drudgery, AI as a tutor, and the relationship between agency and accurate self-assessment (pp. 54–63, 149–150, 169–173, 184–203). So also proposes unaided checks of understanding. The professional application here allows relevant sources and tools while examining the learner’s own reasoning on changed problems. The empirical claims have been checked against the published papers or, where stated, their abstracts. The legal exercise is a proposal, rather than a reported experience or a tested training programme.