AI-Voyager: A Public-Interest System for Human-AI Civic Reasoning to Contest and Shape the Rules That Govern AI
Organizations: Department of Computer Science, University of North Carolina, Chapel Hill, USA · School of Data and Information Sciences, University of North Carolina, Chapel Hill, USA · Society-Centered AI Lab
Abstract
Governments are writing rules for artificial intelligence that decide what counts as an automated decision or a harm, often before the people they affect have a chance to question them. We introduce AI-Voyager, a public-interest system for human-AI civic reasoning, in which people and AI work together to understand, contest and shape the rules that govern AI. A resident can find their city's rules on AI in hiring or upload their country's AI strategy, read it as an affected worker would, open the passage behind any answer, and draft a revision. Its architecture turns 3,653 governance documents from city, state, federal and national governments into passages that keep their page and edition, and retrieves evidence through bounded agentic search that verifies each statement. On 360 new questions that require finding the relevant documents, it reached 83.0% macro recall, against 80.5% for the strongest baseline, 73.7% without iterative search and 99.1% on a benchmark with fixed documents. In a study with 24 participants new to two AI bills, most proposed changes asked how provisions would apply. Participants chose lenses for the kind of explanation they wanted, and citations signaled grounding but rarely led them to check the source. By binding AI assistance to the exact text of public documents, this work introduces a grammar of operations and a design space that inform how such systems are designed and evaluated, and how people take part in governing AI.
Figures & tables
| Operation | Design Goal | What a person does | Interface |
|---|---|---|---|
| Find | DG2 | Locate a policy by place, keyword, topic, or question | World Map, Bill Library, AI Chat |
| Bring | DG2 | Add a document from their own jurisdiction | Upload |
| Read | DG1 | Read the text and its reading aids | Full text, summary, mind map |
| Ask | DG4 | Question one version or the whole collection | AI Companion, AI Chat |
| Reframe | DG3 | View the same source from another position | Lenses |
| Verify | DG4 | Open a cited passage in its source | Expandable citations |
| Design goal | Interface | Technical component | Evaluation |
|---|---|---|---|
| DG1. Legible governance record | Version header, status labels, per-edition citation numbers | Record and version model, canonical catalogue (§ 4.2 ) | Version-specific gold sets (§ 5 ) |
| DG2. Discovery without expertise | World Map, Bill Library, AI Chat, document upload | Scope resolution, dense search, source-balanced retrieval (§ 4.3 ) | 120 discovery questions (§ 5 ) |
| DG3. Perspectives on one source | Five lenses and General Analysis | Lenses share one version scope and evidence pool (§ 4.3 ) | User study (§ 7 ) |
| DG4. Verifiable facts, distinct from interpretation | Expandable citations, evidence windows | Passage provenance (§ 4.2 ), statement verification (§ 4.3 ) | Benchmarks, ablations (§ 6 ), user study (§ 7 ) |
| DG5. Revision with judgments left to the user | Edit Bill, Launchpad | Proposal labeling in the reviewer (§ 4.3 ), versioned drafts | Not evaluated directly; participants’ proposed changes (§ 7 ) |
| Collection | Records | Versions | Pages | Passages |
|---|---|---|---|---|
| U.S. federal bills | 1,404 | 1,799 | 244,194 | 635,863 |
| U.S. state bills | 707 | 1,647 | 16,379 | 70,274 |
| National strategies | 152 | 152 | 11,306 | 65,288 |
| National legislation | 29 | 29 | 5,721 | 39,392 |
| City policies | 26 | 26 | 421 | 1,720 |
| Total | 2,318 | 3,653 | 278,021 | 812,537 |
| CAP | Primary policy domain | ||
|---|---|---|---|
| 1 | Macroeconomics | 50 | 15 |
| 2 | Civil rights and civil liberties | 56 | 15 |
| 3 | Health | 197 | 23 |
| 4 | Agriculture | 34 | 13 |
| 5 | Labor and employment | 100 | 18 |
| 6 | Education | 148 | 21 |
| Layer | Dimension | Tension | AI-Voyager’s choice | Evidence from this work |
|---|---|---|---|---|
| Technical | Unit of evidence | Readable summaries vs. faithful source text | Source passages only, with character-level provenance | Reading aids kept apart from evidence (§ 4.2 ) |
| Representation over time | Corpus-level structure vs. qualifications and editions kept intact | Versioned passages bound by manifests | 3,653 versions with resolvable citations (§ 4.1 ) | |
| AI agency | Coverage vs. cost and predictability | Bounded agentic retrieval with statement verification | 83.0% vs. 73.7% macro recall with and without iteration, at twice the calls (§ 6.3 ) | |
| Human-AI | Interpretation | Breadth of perspectives vs. steering the reading | Five lenses over one shared evidence base | Lenses saved time; General Analysis left room for own thinking (§ 7.2 ) |
| Verification burden | Convenience vs. scrutiny | Model checks each statement; users can open each citation | Citations read as grounding cues, rarely used to verify (§ 7.2 ) | |
| Societal | Participation | Lower effort vs. meaningful influence | Reading, drafting, and public discussion | Participants asked how provisions apply; no institutional channel yet (§ 7.2 ) |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Subset | Questions | Gold | Gold (%) | Gold/Q |
|---|---|---|---|---|
| ContractNLI | 194 | 276 | 21.1 | 1 / 1.42 / 3 |
| CUAD | 194 | 272 | 20.8 | 1 / 1.40 / 7 |
| MAUD | 194 | 306 | 23.4 | 1 / 1.58 / 7 |
| PrivacyQA | 194 | 453 | 34.7 | 1 / 2.34 / 14 |
| Overall | 776 | 1,307 | 100.0 | 1 / 1.68 / 14 |
| Subset | Max Gold/Q | Full-capacity | Max precision (%) |
|---|---|---|---|
| ContractNLI | 3 | 4 | 35.6 |
| CUAD | 7 | 8 | 17.5 |
| MAUD | 7 | 8 | 19.7 |
| PrivacyQA | 14 | 15 | 15.6 |
| Overall | 14 | 15 | 11.2 |
| Subset / configuration | Q | Gold | p95 Latency (s) | Model calls per question | |||
|---|---|---|---|---|---|---|---|
| A. LegalBench-RAG-mini: Voyager-RAG (776 questions) | |||||||
| Overall | 2 | 776 | 1,307 | 62.0 | 77.3 | 7.0 | 4.8 |
| ContractNLI | 2 | 194 | 276 | 55.9 | 80.8 | 6.5 | 4.6 |
| CUAD | 2 | 194 | 272 | 53.6 | 78.8 | 6.9 | 4.8 |
| MAUD | 2 | 194 | 306 | 61.6 | 81.1 | 7.0 | 4.9 |
| PrivacyQA | 2 | 194 | 453 | 76.8 | 68.5 | 7.4 | 5.0 |
| ID | Question |
|---|---|
| Q1 | What is this proposed law about? |
| Q2 | What are two important rules or provisions? |
| Q3 | Who might be affected by this law, and how? |
| Q4 | How might different people or groups have different opinions about this bill? |
| Q5 | What would you change or make clearer about this bill? |
| Interview Question | Follow-up Probe |
|---|---|
| Q1. If you were preparing to participate in a real public policy discussion, which of the AI systems would you prefer to use and why? | What features or aspects of that system would influence your choice? |
| Q2. Thinking about the two AI systems you used, what differences did you notice in how you explored and understood the legislation? | Did the systems affect how you approached the tasks differently? |
| Q3. How did you decide which information to trust or include in your policy contributions? | What role did the policy documents, AI responses, citations, or perspectives play in those decisions? |
| Q4. How did the availability of different AI perspectives affect the way you thought about the legislation? | Were the perspectives useful, realistic, limited, repetitive, or distracting? Did you encounter anything you had not considered before? |
| Q5. Did either AI system influence how you formed your own interpretation or recommendations about the legislation? | Did either system make it easier or harder to develop a judgment independently of the AI? |
| Q6. Did you encounter any information that was difficult to distinguish as an actual policy requirement, an AI interpretation, or an AI recommendation? | How did you handle that uncertainty? Did it affect anything you included in your final contribution? |
| Condition | First Task | Second Task | N |
|---|---|---|---|
| 1 | General, Policy A | Multi-Perspectives, Policy B | 6 |
| 2 | General, Policy B | Multi-Perspectives, Policy A | 6 |
| 3 | Multi-Perspectives, Policy A | General, Policy B | 6 |
| 4 | Multi-Perspectives, Policy B | General, Policy A | 6 |
| Total | 24 |
AI Governance
Artificial intelligence governance studies how laws, institutions, and technical controls shape AI development, deployment, accountability, and social effects.
See all 412 papers