Use Simple Checklists to Assess Legal AI Tools

Which decision are you making?

Different people may need to sign off on different parts: the legal task owner, IT/security, privacy, records management and professional supervisor.

Checklist 1 — Review a legal AI product

Ask the vendor or product owner to document: the exact feature being tested; supported legal tasks and jurisdictions; model and source-data descriptions where available; how responses are sourced; what happens when the tool cannot answer; product changes; and how a user corrects or reports an error.

Evidence to retain: current documentation, written responses, a dated demonstration record and the exact tested configuration. An unanswered question is an item to investigate, not an automatic pass.

Checklist 2 — Protect client and organization information

Before processing data, confirm who controls access, where information moves, whether the provider retains it, which subprocessors are involved when relevant, how long records remain available, what deletion controls exist, and how incidents are handled. Check the organization's applicable client duties and privacy rules.

Evidence to retain: approved data-flow diagram, vendor documentation, applicable agreements, permission records and security-review decision. Do not use real client records in a demonstration unless their use has been specifically cleared.

Checklist 3 — Verify legal answers and citations

Test questions that mirror the team's work. For each answer, record whether the cited authority exists, supports the exact proposition, applies to the jurisdiction and relevant date, and contains any limitation that changes the conclusion. Include examples where the correct response is “I cannot verify this.”

Evidence to retain: question, original source, AI output, reviewer note and final acceptance or correction. Separate unsupported statements from merely incomplete wording.

Checklist 4 — Control actions by AI agents

Ask whether a connected tool can only read information or can also create, modify, submit or delete records. Identify permissions, sign-off requirements, activity logs, exceptional cases and the way to pause or revoke access. Use a low-risk test before expanding the scope.

Evidence to retain: vendor capability list, permission settings, task approvals, test logs and review-owner sign-off. The presence of a connector is not itself permission to use a client system.

Checklist 5 — Decide whether a pilot can progress

Define success and failure before testing. Include examples from the real task distribution, realistic exceptions, who reviews every output, and when the tool must be stopped. Compare the recorded results with the team's acceptance criteria rather than a generalized promise about AI accuracy.

Evidence to retain: test set description, known limitations, error register, corrective actions and dated owner decision.

A short worksheet that makes the checks usable

This is an original editorial worksheet format. It is not a certification or standardized legal opinion.

How these checklists relate to wider guidance

The NIST Generative AI Profile supports a risk-aware approach to evaluation. The ABA's 2026 legal practice discussion emphasizes lawyers' independent responsibility for accuracy, confidentiality and oversight.

TechCorpLegal's existing legal AI metrics guide provides related discussion of measurement. The checklists here serve a different purpose: they record the questions, evidence and review decisions required before or during use. Any relationship to a separate proprietary scoring method would require its own verified explanation.

What these checks cannot establish

A completed worksheet cannot guarantee professional compliance, eliminate security risk or prove that a tool will remain reliable after an update. A result observed on fictional documents is not proof of performance on confidential or specialized matters. Review should be repeated when the product, its data access or its intended task changes materially.

Frequently asked questions

Do we need to check every tool the same way? Use a consistent baseline, then add checks for the actual task, data sensitivity and jurisdiction.

Can we accept a vendor's accuracy statement as proof? Treat it as vendor information and request the method, sample and limitations. Test tasks relevant to your organization.

Who should approve a legal AI pilot? Approval depends on governance and professional duties; commonly relevant functions include legal leadership, IT/security, privacy and the people supervising the work.

When should the checklist be revisited? After material product changes, new access permissions, new uses, significant errors or policy updates.

Choose your evaluation checklist

Begin with the decision you face and document one clear, reviewable use case. Use the complementary Testing and Data Tools directory to identify possible test methods (B02-002).