AdaQE-CG: Adaptive Query Expansion for Web-Scale Generative AI Model and Data Card Generation

Haoxuan Zhang; Ruochi Li; Zhenni Liang; Mehri Sattari; Phat Vo; Collin Qu; Ting Xiao; Junhua Ding

cs.AIMar 16, 2026

AdaQE-CG: Adaptive Query Expansion for Web-Scale Generative AI Model and Data Card Generation

Authors: Haoxuan Zhang, Ruochi Li, Zhenni Liang, Mehri Sattari, Phat Vo, Collin Qu, Ting Xiao, Junhua Ding,

Abstract

Transparent and standardized documentation is essential for building trustworthy generative AI (GAI) systems. However, existing automated methods for generating model and data cards still face three major challenges: (i) static templates, as most systems rely on fixed query templates that cannot adapt to diverse paper structures or evolving documentation requirements; (ii) information scarcity, since web-scale repositories such as Hugging Face often contain incomplete or inconsistent metadata, leading to missing or noisy information; and (iii) lack of benchmarks, as the absence of standardized datasets and evaluation protocols hinders fair and reproducible assessment of documentation quality. To address these limitations, we propose AdaQE-CG, an Adaptive Query Expansion for Card Generation framework that combines dynamic information extraction with cross-card knowledge transfer. Its Intra-Paper Extraction via Context-Aware Query Expansion (IPE-QE) module iteratively refines extraction queries to recover richer and more complete information from scientific papers and repositories, while its Inter-Card Completion using the MetaGAI Pool (ICC-MP) module fills missing fields by transferring semantically relevant content from similar cards in a curated dataset. In addition, we introduce MetaGAI-Bench, the first large-scale, expert-annotated benchmark for evaluating GAI documentation. Comprehensive experiments across five quality dimensions show that AdaQE-CG substantially outperforms existing approaches, exceeds human-authored data cards, and approaches human-level quality for model cards. Code, prompts, and data are publicly available at: https://github.com/haoxuan-unt2024/AdaQE-CG.

Figures & tables

Figure 1. A comparison of card generation paradigms: (a) Zero-Shot Generation (top), (b) Template-Guided RAG (middle), and (c) AdaQE-CG (bottom).

Figure 2. AdaQE-CG: Hybrid-module pipeline for automated card generation. Module 1 (IPE-QE): iteratively extracts field values from document chunks via context-aware query expansion to produce the initial card. Module 2 (ICC-MP): conditionally enriches incomplete fields using the MetaGAI Pool when similar cards are available, producing the final card.

Metric	Definition
Faithfulness (F)	Accurately reflects information from source materials without introducing unsupported claims or omitting key points
Relevance (R)	Content focused on the specific category being evaluated, avoiding unrelated or off-topic information
Accuracy (A)	Statements are factually correct based on available references and can be directly verified
Consistency (C)	Information is internally consistent within the card, with no contradictions or logical gaps
Usefulness (U)	Provides clear, practical, and helpful information for users or researchers who want to understand or use the model or dataset

Table 1. Evaluation metrics for assessing the generated model and data cards. Each metric is scored on a 1-5 scale by domain experts.

Card Type	Method	Faithfulness		Relevance		Accuracy		Consistency		Usefulness		Average
Card Type	Method	$\rho$ =0.215, $r$ =0.243		$\rho$ =0.228, $r$ =0.375		$\rho$ =0.204, $r$ =0.218		$\rho$ =0.118, $r$ =0.141		$\rho$ =0.331, $r$ =0.386
		GPT	Gemini	GPT	Gemini	GPT	Gemini	GPT	Gemini	GPT	Gemini	GPT	Gemini
Model	Zero-shot	4.0 (4.02)	3.0 (4.33)	4.0 (4.64)	3.0 (4.50)	3.0 (4.07)	3.0 (4.33)	1.0 (4.75)	1.0 (4.66)	4.0 (3.72)	3.0 (4.05)	3.2 (4.24)	2.6 (4.37)
Model	CardGen ( Liu et al., 2024 )	3.0 (4.07)	2.0 (4.36)	2.0 (4.74)	2.0 (4.62)	2.0 (4.08)	2.0 (4.36)	3.0 (4.73)	3.0 (4.65)	2.0 (3.97)	2.0 (4.19)	2.4 (4.32)	2.2 (4.43)
Model	Human-Annotated	1.0 (4.09)	4.0 (4.26)	3.0 (4.66)	4.0 (4.49)	1.0 (4.11)	4.0 (4.26)	2.0 (4.75)	4.0 (4.62)	3.0 (3.83)	4.0 (4.04)	2.0 (4.29)	4.0 (4.33)
Model	AdaQE-CG	2.0 (4.08)	1.0 (4.37)	1.0 (4.79)	1.0 (4.65)	4.0 (4.05)	1.0 (4.37)	4.0 (4.73)	2.0 (4.65)	1.0 (4.02)	1.0 (4.24)	2.4 (4.33)	1.2 (4.46)

Table 2. Performance evaluation of LLM-as-a-Judge for model card and data card generation across quality dimensions. GPT = GPT-5-nano; Gemini = Gemini-2.5 Flash-Lite. Values are shown as Rank (Score). Correlation coefficients between GPT and Gemini:

\rho

= Spearman correlation,

r

= Pearson correlation. Significance levels: *

p<0.001

. Within each cell, bold indicates better performance (lower rank or higher score), underline indicates the second-best performance.

Card Type	Method	Faithfulness		Relevance		Accuracy		Consistency		Usefulness		Average
Card Type	Method	$\rho$ =0.509, $r$ =0.467		$\rho$ =0.204, $r$ =0.231		$\rho$ =0.358, $r$ =0.366		$\rho$ =0.310, $r$ =0.319		$\rho$ =0.446, $r$ =0.453
		Group 1	Group 2	Group 1	Group 2	Group 1	Group 2	Group 1	Group 2	Group 1	Group 2	Group 1	Group 2
Model	Zero-shot	4.0 (2.73)	4.0 (2.94)	4.0 (4.12)	4.0 (3.01)	4.0 (2.69)	4.0 (2.85)	4.0 (3.70)	4.0 (3.17)	4.0 (2.68)	4.0 (2.79)	4.0 (3.18)	4.0 (2.95)
Model	CardGen ( Liu et al., 2024 )	3.0 (3.29)	3.0 (3.21)	2.0 (4.53)	3.0 (3.33)	3.0 (3.28)	3.0 (3.17)	2.0 (4.19)	3.0 (3.39)	3.0 (3.26)	3.0 (3.09)	2.6 (3.71)	3.0 (3.24)
Model	Human-Annotated	1.0 (3.69)	1.0 (3.53)	1.0 (4.66)	1.0 (3.62)	1.0 (3.66)	1.0 (3.43)	1.0 (4.32)	1.0 (3.59)	1.0 (3.55)	1.0 (3.35)	1.0 (3.98)	1.0 (3.50)
Model	AdaQE-CG	2.0 (3.47)	2.0 (3.31)	3.0 (4.38)	2.0 (3.43)	2.0 (3.44)	2.0 (3.22)	3.0 (4.05)	2.0 (3.45)	2.0 (3.35)	2.0 (3.17)	2.4 (3.74)	2.0 (3.32)

Table 3. Human evaluation of model card and data card generation across quality dimensions. Group 1 and Group 2 represent two independent sets of human evaluators assessing the same algorithms and card types. Values shown as Rank (Score). Correlation coefficients between Group 1 and Group 2:

\rho

= Spearman correlation,

r

= Pearson correlation. Significance levels: *

p<0.001

. Within each metric, bold indicates best performance (lowest rank or highest score), underline indicates the second-best performance.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Card	Field	Description
Model Card	Model Details	Information about the model developer, architecture, size, training methodology, modalities, version, license, and contact details
	Intended Use	Primary applications, target users, supported languages/domains, out-of-scope uses, and age restrictions
	Generative Capabilities	Generation quality, content types, length limitations, consistency, latency, and customization options
	Safety Considerations	Content safety measures, bias analysis, fairness metrics, red team testing, jailbreaking resistance, and child safety
	Training Data	Training corpus details, data filtering processes, demographic representation, language coverage, consent/privacy, and evaluation datasets
	Performance Metrics	Generation quality metrics, safety metrics, factual accuracy, bias metrics, cultural sensitivity, and robustness measures

Appendix

Table 4. Definitions of Model and Data Card for Generative AI

Card Type	Category	Metadata Tags	Correlation ( $\rho$ )	Frequency (n)
Model Cards	Popularity	likes	0.408*	123,013
		downloads	0.285*	123,013
	Task	text-to-speech	0.211*	3,145
		text-to-audio	0.173*	2,781
		image-text-to-text	0.132*	2,872
		text-to-image	0.105*	1,223

Appendix

Table 5. Spearman Correlations between WCCI Scores and Metadata Tags. * indicates

p<0.001

Panel of Figure 3 — Figure 3. Field-level WCCI decomposition across generative AI task categories. (a) Model cards show substantially lower completeness with minimal Safety and Ethical Considerations. (b) Data cards achieve higher completeness dominated by technical specifications, with responsible AI dimensions contributing smaller segments.

Panel of Figure 4 — Figure 4. Query expansion effectiveness across rounds 2-10. Lines (left y-axis) show active cards undergoing query expansion per field; bars (right y-axis) track cumulative quality improvements across five dimensions.

Figure 5. Quality improvement contribution of ICC-MP. Blue bars show cases where ICC-MP improved quality scores over IPE-QE-only cards, green bars show no improvement, and orange bars indicate tied scores.

Document Information Extraction

Document Information Extraction turns text, tables, forms, and images into structured, searchable data.

See all 290 papers

MetaGAI: A Large-Scale and High-Quality Benchmark for Generative AI Model and Data Card Generation
Apr 26, 2026Haoxuan Zhang, Ruochi Li, Yang Zhang +4Automated Evaluation Generative AI Evaluation
University of North Texas, Denton, TX, USA · North Carolina State University, Raleigh, NC, USA
Automatic Model Card Generation Using an LLM
Aug 25, 2026Tajkia Rahman Toma, Balreet Grewal, Cor-Paul Bezemer
University of Alberta, Edmonton, Canada
Toward Reusability of AI Models Using Dynamic Updates of AI Documentation
Apr 19, 2026Peter Bajcsy, Walid Keyrouz

Apr 26, 2026J/K move · Enter open · S save

MetaGAI: A Large-Scale and High-Quality Benchmark for Generative AI Model and Data Card Generation

Haoxuan Zhang, Ruochi Li, Yang Zhang, Zhenni Liang+3

University of North Texas, Denton, TX, USA · North Carolina State University, Raleigh, NC, USA

Automated Evaluation Generative AI Evaluation

Open paper AskPDF arXiv

The rapid proliferation of Generative AI necessitates rigorous documentation standards for transparency and governance. However, manual creation of Model and Data Cards is not scalable, while automated approaches lack large-scale, high-fidelity benchmarks for systematic evaluation. We introduce MetaGAI, a comprehensive benchmark comprising 2,541 verified document triplets constructed through semantic triangulation of academic papers, GitHub repositories, and Hugging Face artifacts. Unlike prior single-source datasets, MetaGAI employs a multi-agent framework with specialized Retriever, Generator, and Editor agents, validated through four-dimensional human-in-the-loop assessment, including human evaluation of editor-refined ground truth. We establish a robust evaluation protocol combining automated metrics with validated LLM-as-a-Judge frameworks. Extensive analysis reveals that sparse Mixture-of-Experts architectures achieve superior cost-quality efficiency, while a fundamental trade-off exists between faithfulness and completeness. MetaGAI provides a foundational testbed for benchmarking, training, and analyzing automated Model and Data Card generation methods at scale. Our data and code are available at: https://github.com/haoxuan-unt2024/MetaGAI-Benchmark.

Abstract

Document Information Extraction

Explore similar work

MetaGAI: A Large-Scale and High-Quality Benchmark for Generative AI Model and Data Card Generation

Automatic Model Card Generation Using an LLM

Toward Reusability of AI Models Using Dynamic Updates of AI Documentation