Choose the legal question you want to test
Some datasets contain questions and reference answers. Others contain court opinions or legislative records that must first be processed into examples. The right choice depends on the work:
| Intended task | Material to investigate | Key limitation to review |
|---|---|---|
| Legal reasoning | Curated questions and reference answers | Task design and scoring method |
| Case-law retrieval | Court opinions and metadata | Jurisdiction, coverage and citation quality |
| Legislative research | Structured bills and legislative actions | Versioning, amendments and current status |
| Contract clause identification | Agreements labeled for the relevant clauses | Confidentiality, rights and label consistency |
| Document classification | A reviewed set of known document categories | Bias, language variation and missing categories |
Featured starting points from published sources
The following are research starting points, not a ranking or a representation that all records may be copied into a commercial project.
LegalBench — legal reasoning tasks
The LegalBench paper, published in the NeurIPS 2023 Datasets and Benchmarks Track, describes a collaboratively developed benchmark of 162 tasks across different types of legal reasoning. It is relevant when studying whether a model performs defined legal reasoning exercises. The research publication alone does not establish that every task, document or redistribution method has identical rights.
Check next: task list, model-evaluation method, source documents, current repository terms and suitability to the legal work being assessed.
CourtListener — court records and supporting data
Free Law Project's CourtListener documentation describes APIs, bulk data and other access routes for legal records. It is useful for investigating case-law search and citation-related research. Available data, eligibility and reuse conditions depend on the chosen access method and relevant rules.
Check next: coverage, source provenance, permitted extraction method, update frequency and any rights attached to underlying documents.
Caselaw Access Project — digitized case law
Harvard's Caselaw Access Project terms describe its caselaw data and metadata as available under CC0 1.0, while distinguishing those records from other site materials and requiring compliance with applicable laws and third-party rights.
Check next: the exact data being used, whether it falls within the stated designation, completeness, and whether any personal information or third-party material requires additional care.
Congress.gov API — structured legislative information
The official Congress.gov API exposes structured congressional information, including bills and their actions. It is relevant to research on U.S. legislative processes, but legislative records are not interchangeable with judicial precedent or advice on current legal obligations.
Check next: API documentation, key requirements, update dates, endpoint coverage and the source's applicable use conditions.
Understand task coverage before comparing datasets
A benchmark's headline result becomes less informative if its test questions do not resemble the intended application. Ask whether the examples cover the relevant countries, languages, dates, document types and difficult exceptions. Find out whether reference answers were written or checked by subject specialists.
A dataset can be extensive but still contain duplicated records, scanning errors, incomplete opinions or labels that do not correspond to the intended task. Keep those limitations alongside any summary of results.
Dataset review card: what to record
| Field | Why it matters |
|---|---|
| Source and producer | Establishes provenance and a responsible publisher |
| Intended task | Explains what an evaluation could meaningfully test |
| Jurisdiction and period | Identifies where the data applies and when |
| Coverage and exclusions | Reveals where results may not generalize |
| Version and update date | Supports repeatable testing |
| Access method | Describes how records may be obtained |
| Rights and restrictions | Identifies whether the proposed use is permitted |
| Personal-data screening | Highlights privacy and confidentiality concerns |
| Review record | Preserves who checked the data and what remains unknown |
This is a suggested directory card, not a published assessment of any featured collection.
Example: using LegalBench responsibly
Suppose a team wants to test whether an AI model can distinguish an issue in a legal text. First select tasks that actually address that skill. Then read the benchmark documentation, define how answers will be scored, preserve the model and test conditions, and inspect the cases where the model fails. A result on this test should be reported as performance on those tasks under those conditions, not as proof that the model can reliably advise on real client matters.
Sources, updates and reuse
Before using a dataset, follow the current official source and read its applicable terms. Do not assume that open visibility grants rights to republish, combine with other databases or train models. This directory describes candidate sources and review criteria; it does not reproduce or distribute their content.
Frequently asked questions
Is an open dataset automatically suitable for commercial AI development? No. Check the exact materials, source terms, applicable law and intended processing purpose.
Does a high benchmark result prove an AI lawyer is accurate? No. It describes performance on a defined evaluation; legal professional work requires separate task-specific and human review.
Can I create my own test dataset? Yes, provided the material and processing are authorized. Document provenance, labels, quality checks and the intended use.
How is this different from the Testing and Data Tools directory? This page focuses on underlying collections and benchmarks. B02-002 focuses on evaluation methods and tools.
Explore a dataset with a clear purpose
Choose a task, compare the documented sources, then check licensing and suitability before downloading or processing anything. Continue to Open Legal Data Sources for records that may be used as source material (B02-011).
Find your next useful step
Start with the sources and practical checks above, then continue to a related guide.
Explore Legal AI Datasets