Patent and legal reasoning datasets for AI training: why expert process matters
Granted patents, judgments and signed contracts are the obvious legal material for AI, and a great deal of it is already public and freely available to anyone building a model. Much scarcer is a record of how a skilled professional reached the finished document. That means the claim amendments considered and rejected, the reasons an objection was answered one way rather than another, and the risk judgements in an advice note that never appear in the final version. For UK patent attorney firms, law firms and regulatory consultancies, that gap may be where much of the value in their files lies.
This article looks at why some AI research now focuses on the reasoning process rather than only the final answer, which kinds of patent and legal material record that process, and the professional rules that decide what a firm can responsibly license.
Why process matters to AI developers
A published piece of research shows the idea clearly. In Let's Verify Step by Step, researchers at OpenAI compared two ways of training a reward model, the component that scores a model's step-by-step solutions to maths problems. "Outcome supervision" gave feedback only on whether the final answer was right, while "process supervision" gave feedback on each intermediate step. On problems from the MATH dataset, process supervision significantly outperformed outcome supervision and produced a much more reliable reward model. The authors also describe an alignment benefit: process supervision more directly rewards a model for following a chain of reasoning that humans endorse, whereas a model trained on outcomes alone can be rewarded for reaching a correct answer through incorrect reasoning.
The paper is open about the cost. Final answers to the maths problems could be checked automatically, but there was no simple way to automate step-by-step feedback, so the team relied on human data labellers to judge each step, and released the resulting dataset of around 800,000 step-level labels. The authors expect easy-to-check final answers to become less common in more complex domains, and they say it is unknown how far their results generalise beyond maths. Law and patent work is a good example of such a domain. There is rarely an automatically checkable answer, and judging whether each step of an argument is sound takes real expertise. Material that already records that step-by-step reasoning, written by qualified people on real matters, is hard to produce at scale, which is why it is likely to interest AI developers.
Legal AI research points in a similar direction. LegalBench, an open benchmark built collaboratively with legal professionals, organises its 162 tasks around six types of legal reasoning: issue spotting, rule recall, rule application, rule conclusion, interpretation and rhetorical understanding. Its authors argue that earlier benchmarks treated anything involving legal data or laws as "legal reasoning", without distinguishing between these different skills. They also note that each LegalBench task tests one type of reasoning in isolation, that the benchmark leans towards US law and questions with objectively correct answers, and that tasks testing several steps together are left for future work. Material in which a professional works through a single problem from spotting the issue to reaching a conclusion speaks directly to that gap. We make the same distinction between reasoning and outputs for engineering firms in What engineering knowledge is valuable for AI training (and what isn't).
What "process" looks like in patent work
Patent prosecution is unusually rich in recorded reasoning, because so much of it is written down as argument. Useful material can include:
- Drafting decisions. How an invention disclosure became a set of claims, what fallback positions were built into the dependent claims, and why particular terms were defined the way they were.
- Responses to examination reports. The analysis of cited prior art, the choice between amending and arguing, and the reasoning that explains why the amended claim is novel and inventive.
- Freedom-to-operate and validity opinions. How a competitor's claims were construed, where the risk was judged to sit, and what was recommended as a result.
- Internal review. A senior attorney's comments on a trainee's draft can be among the clearest records of expert judgement a firm holds.
It is worth separating out what is already public. Under section 118 of the Patents Act 1977, once a UK application has been published the Intellectual Property Office must, on request, let people inspect the documents relating to it and to any resulting patent, subject to prescribed restrictions. Before publication, the office cannot release them without the applicant's consent, apart from limited exceptions. The IPO's Manual of Patent Practice explains that the restrictions cover, for example, documents the office prepared for internal use only and documents the comptroller has directed should be treated as confidential. It also says that certain documents are available free online through One IPO Search, sometimes with personal or sensitive details redacted, and that an applicant's observations rebutting a novelty or inventive step objection should not be treated as confidential. The IPO's guidance on monitoring applications confirms that reports and correspondence between the office and the applicant or their attorney can be viewed on the file of a published application. So the formal exchange with the examiner is often not secret. What stays inside the firm is the thinking around it: the client's instructions, the options weighed before a response was filed, and the internal discussion that explains why the filed response says what it says. That private layer is what can turn a public file into a record of expert process.
What "process" looks like in legal and regulatory work
Law firms and regulatory consultancies hold similar material in different forms. Examples include research notes that trace how a question was narrowed and answered, issues lists and risk matrices prepared at the start of a matter, mark-ups of contracts with the reasons for each change, and regulatory submissions together with the analysis of which requirements applied and why. Precedents, practice notes and training material written in-house are often especially useful, because they set out a firm's reasoning in general terms and contain little or no client information.
The professional rules that shape what can be used
The features that make this material useful also make it sensitive, so the professional rules come first.
Confidentiality. Paragraph 6.3 of the SRA Code of Conduct for Solicitors requires solicitors to keep the affairs of current and former clients confidential unless disclosure is required or permitted by law or the client consents. Registered patent and trade mark attorneys have a similar duty under rule 1.8 of the Code of Conduct in IPReg's Core Regulatory Framework, which requires them to keep a client's affairs confidential unless permitted by law or the client consents. Guidance to IPReg's earlier Rules of Conduct, which the framework has since replaced, said that the duty continues until the client permits disclosure, waives confidentiality or the information is made public and ceases to be confidential. The SRA Code refers expressly to former clients, as IPReg's earlier rules did, so firms should not assume the duty ends when a matter closes.
Privilege. Legal professional privilege belongs to the client, and, as the SRA's guidance on privilege puts it, it is the client who can assert or waive it. The same guidance warns that privilege can be lost if a document is shared with a third party. Under section 280 of the Copyright, Designs and Patents Act 1988, where a registered or European patent attorney acts for a client on matters such as the protection of an invention, design, technical information or trade mark, the related communications and documents are privileged in the same way as if the attorney were the client's solicitor. A firm is therefore not free to treat privileged advice as its own asset to license.
Personal data. Files contain names, contact details and sometimes far more sensitive information about clients, inventors, counterparties and staff. Our checklist for anonymising professional documents under UK GDPR covers the practical steps, including the ICO's motivated intruder test.
In practice, this points to a layered approach. The easiest material to clear is the firm's own: precedents, training notes, internal guidance and reviewer commentary written in general terms. Client matters can sometimes be used with the client's informed consent, or after the facts have been abstracted so far that no client, matter or invention can be identified, and for patent work that has to include checking that the reasoning can't be traced back to a published file. Privileged advice, material under strict non-disclosure terms and anything linked to live disputes should normally stay out altogether. Firms should take their own advice on how their engagement terms and regulatory duties apply.
Process data can also be created, not only extracted
Not every useful dataset has to come from historic files. Some material can be produced fresh, such as an experienced attorney working through a realistic but fictional claim-drafting problem and explaining each step, or a senior solicitor reviewing an AI model's analysis of a contract and recording where it went wrong and why. Because no client matter is involved, this route avoids most client confidentiality questions, provided the scenario is not a thinly disguised version of a real matter. It also lets a firm decide how much of its approach to share, and earn from the expertise itself rather than from its documents.
What this means for UK firms
For most firms, the useful contribution to patent and legal reasoning datasets for AI training is unlikely to be bulk documents that already sit in public databases. It is more likely to be the reasoning around them, in a form that has been cleared for use. An internal inventory is a sensible first step: which kinds of reasoning the firm records well, which of it belongs to the firm rather than to clients, and which can be anonymised or recreated without risk. Our guide on how UK engineering firms can license their know-how to AI covers commercial models such as one-off licence fees and revenue share in more detail. It is written for engineering consultancies, but much of it carries across to other professional firms.
How ID43 Labs works with legal and patent firms
ID43 Labs works with UK firms to identify reasoning-rich material and expert-led tasks, prepare and anonymise them, and license the resulting datasets to AI labs on a revenue-share basis. We do not buy or sell personal data, we do not ask firms to disclose privileged or confidential client material without the necessary consent, and we do not give legal advice. If your firm wants an initial view of what its expertise could be worth, see our page for legal, patent and regulatory firms.
This article is general information, not legal or regulatory advice. Firms should take independent advice on confidentiality, privilege, data protection and their professional obligations before licensing any material.