cs.AIMar 16, 2026

AdaQE-CG: Adaptive Query Expansion for Web-Scale Generative AI Model and Data Card Generation

Authors: Haoxuan Zhang, Ruochi Li, Zhenni Liang, Mehri Sattari, Phat Vo, Collin Qu, Ting Xiao, Junhua Ding, 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Transparent and standardized documentation is essential for building trustworthy generative AI (GAI) systems. However, existing automated methods for generating model and data cards still face three major challenges: (i) static templates, as most systems rely on fixed query templates that cannot adapt to diverse paper structures or evolving documentation requirements; (ii) information scarcity, since web-scale repositories such as Hugging Face often contain incomplete or inconsistent metadata, leading to missing or noisy information; and (iii) lack of benchmarks, as the absence of standardized datasets and evaluation protocols hinders fair and reproducible assessment of documentation quality. To address these limitations, we propose AdaQE-CG, an Adaptive Query Expansion for Card Generation framework that combines dynamic information extraction with cross-card knowledge transfer. Its Intra-Paper Extraction via Context-Aware Query Expansion (IPE-QE) module iteratively refines extraction queries to recover richer and more complete information from scientific papers and repositories, while its Inter-Card Completion using the MetaGAI Pool (ICC-MP) module fills missing fields by transferring semantically relevant content from similar cards in a curated dataset. In addition, we introduce MetaGAI-Bench, the first large-scale, expert-annotated benchmark for evaluating GAI documentation. Comprehensive experiments across five quality dimensions show that AdaQE-CG substantially outperforms existing approaches, exceeds human-authored data cards, and approaches human-level quality for model cards. Code, prompts, and data are publicly available at: https://github.com/haoxuan-unt2024/AdaQE-CG.

Figures & tables

Figure 1. A comparison of card generation paradigms: (a) Zero-Shot Generation (top), (b) Template-Guided RAG (middle), and (c) AdaQE-CG (bottom).
Figure 2. AdaQE-CG: Hybrid-module pipeline for automated card generation. Module 1 (IPE-QE): iteratively extracts field values from document chunks via context-aware query expansion to produce the initial card. Module 2 (ICC-MP): conditionally enriches incomplete fields using the MetaGAI Pool when similar cards are available, producing the final card.
MetricDefinition
Faithfulness (F)Accurately reflects information from source materials without introducing unsupported claims or omitting key points
Relevance (R)Content focused on the specific category being evaluated, avoiding unrelated or off-topic information
Accuracy (A)Statements are factually correct based on available references and can be directly verified
Consistency (C)Information is internally consistent within the card, with no contradictions or logical gaps
Usefulness (U)Provides clear, practical, and helpful information for users or researchers who want to understand or use the model or dataset
Table 1. Evaluation metrics for assessing the generated model and data cards. Each metric is scored on a 1-5 scale by domain experts.
Card TypeMethodFaithfulnessRelevanceAccuracyConsistencyUsefulnessAverage
ρ\rho =0.215*, rr =0.243*ρ\rho =0.228*, rr =0.375*ρ\rho =0.204*, rr =0.218*ρ\rho =0.118*, rr =0.141*ρ\rho =0.331*, rr =0.386* 
  GPTGeminiGPTGeminiGPTGeminiGPTGeminiGPTGeminiGPTGemini
ModelZero-shot4.0 (4.02)3.0 (4.33)4.0 (4.64)3.0 (4.50)3.0 (4.07)3.0 (4.33)1.0 (4.75)1.0 (4.66)4.0 (3.72)3.0 (4.05)3.2 (4.24)2.6 (4.37)
ModelCardGen ( Liu et al., 2024 )3.0 (4.07)2.0 (4.36)2.0 (4.74)2.0 (4.62)2.0 (4.08)2.0 (4.36)3.0 (4.73)3.0 (4.65)2.0 (3.97)2.0 (4.19)2.4 (4.32)2.2 (4.43)
ModelHuman-Annotated1.0 (4.09)4.0 (4.26)3.0 (4.66)4.0 (4.49)1.0 (4.11)4.0 (4.26)2.0 (4.75)4.0 (4.62)3.0 (3.83)4.0 (4.04)2.0 (4.29)4.0 (4.33)
ModelAdaQE-CG2.0 (4.08)1.0 (4.37)1.0 (4.79)1.0 (4.65)4.0 (4.05)1.0 (4.37)4.0 (4.73)2.0 (4.65)1.0 (4.02)1.0 (4.24)2.4 (4.33)1.2 (4.46)
Table 2. Performance evaluation of LLM-as-a-Judge for model card and data card generation across quality dimensions. GPT = GPT-5-nano; Gemini = Gemini-2.5 Flash-Lite. Values are shown as Rank (Score). Correlation coefficients between GPT and Gemini: ρ\rho = Spearman correlation, rr = Pearson correlation. Significance levels: * p<0.001p<0.001 . Within each cell, bold indicates better performance (lower rank or higher score), underline indicates the second-best performance.
Card TypeMethodFaithfulnessRelevanceAccuracyConsistencyUsefulnessAverage
ρ\rho =0.509*, rr =0.467*ρ\rho =0.204*, rr =0.231*ρ\rho =0.358*, rr =0.366*ρ\rho =0.310*, rr =0.319*ρ\rho =0.446*, rr =0.453* 
  Group 1Group 2Group 1Group 2Group 1Group 2Group 1Group 2Group 1Group 2Group 1Group 2
ModelZero-shot4.0 (2.73)4.0 (2.94)4.0 (4.12)4.0 (3.01)4.0 (2.69)4.0 (2.85)4.0 (3.70)4.0 (3.17)4.0 (2.68)4.0 (2.79)4.0 (3.18)4.0 (2.95)
ModelCardGen ( Liu et al., 2024 )3.0 (3.29)3.0 (3.21)2.0 (4.53)3.0 (3.33)3.0 (3.28)3.0 (3.17)2.0 (4.19)3.0 (3.39)3.0 (3.26)3.0 (3.09)2.6 (3.71)3.0 (3.24)
ModelHuman-Annotated1.0 (3.69)1.0 (3.53)1.0 (4.66)1.0 (3.62)1.0 (3.66)1.0 (3.43)1.0 (4.32)1.0 (3.59)1.0 (3.55)1.0 (3.35)1.0 (3.98)1.0 (3.50)
ModelAdaQE-CG2.0 (3.47)2.0 (3.31)3.0 (4.38)2.0 (3.43)2.0 (3.44)2.0 (3.22)3.0 (4.05)2.0 (3.45)2.0 (3.35)2.0 (3.17)2.4 (3.74)2.0 (3.32)
Table 3. Human evaluation of model card and data card generation across quality dimensions. Group 1 and Group 2 represent two independent sets of human evaluators assessing the same algorithms and card types. Values shown as Rank (Score). Correlation coefficients between Group 1 and Group 2: ρ\rho = Spearman correlation, rr = Pearson correlation. Significance levels: * p<0.001p<0.001 . Within each metric, bold indicates best performance (lowest rank or highest score), underline indicates the second-best performance.
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

CardFieldDescription
Model CardModel DetailsInformation about the model developer, architecture, size, training methodology, modalities, version, license, and contact details
Intended UsePrimary applications, target users, supported languages/domains, out-of-scope uses, and age restrictions
Generative CapabilitiesGeneration quality, content types, length limitations, consistency, latency, and customization options
Safety ConsiderationsContent safety measures, bias analysis, fairness metrics, red team testing, jailbreaking resistance, and child safety
Training DataTraining corpus details, data filtering processes, demographic representation, language coverage, consent/privacy, and evaluation datasets
Performance MetricsGeneration quality metrics, safety metrics, factual accuracy, bias metrics, cultural sensitivity, and robustness measures
Appendix
Table 4. Definitions of Model and Data Card for Generative AI
Card TypeCategoryMetadata TagsCorrelation ( ρ\rho )Frequency (n)
Model CardsPopularitylikes0.408*123,013
  downloads0.285*123,013
 Tasktext-to-speech0.211*3,145
  text-to-audio0.173*2,781
  image-text-to-text0.132*2,872
  text-to-image0.105*1,223
Appendix
Table 5. Spearman Correlations between WCCI Scores and Metadata Tags. * indicates p<0.001p<0.001 .
Figure 3. Field-level WCCI decomposition across generative AI task categories. (a) Model cards show substantially lower completeness with minimal Safety and Ethical Considerations. (b) Data cards achieve higher completeness dominated by technical specifications, with responsible AI dimensions contributing smaller segments.
Figure 4. Query expansion effectiveness across rounds 2-10. Lines (left y-axis) show active cards undergoing query expansion per field; bars (right y-axis) track cumulative quality improvements across five dimensions.
Figure 5. Quality improvement contribution of ICC-MP. Blue bars show cases where ICC-MP improved quality scores over IPE-QE-only cards, green bars show no improvement, and orange bars indicate tied scores.
More in this topic

Document Information Extraction

Document Information Extraction turns text, tables, forms, and images into structured, searchable data.

See all 290 papers

Explore similar work

CardsList
Apr 26, 2026

MetaGAI: A Large-Scale and High-Quality Benchmark for Generative AI Model and Data Card Generation

The rapid proliferation of Generative AI necessitates rigorous documentation standards for transparency and governance. However, manual creation of Model and Data Cards is not scalable, while automated approaches lack large-scale, high-fidelity benchmarks for systematic evaluation. We introduce MetaGAI, a comprehensive benchmark comprising 2,541 verified document triplets constructed through semantic triangulation of academic papers, GitHub repositories, and Hugging Face artifacts. Unlike prior single-source datasets, MetaGAI employs a multi-agent framework with specialized Retriever, Generator, and Editor agents, validated through four-dimensional human-in-the-loop assessment, including human evaluation of editor-refined ground truth. We establish a robust evaluation protocol combining automated metrics with validated LLM-as-a-Judge frameworks. Extensive analysis reveals that sparse Mixture-of-Experts architectures achieve superior cost-quality efficiency, while a fundamental trade-off exists between faithfulness and completeness. MetaGAI provides a foundational testbed for benchmarking, training, and analyzing automated Model and Data Card generation methods at scale. Our data and code are available at: https://github.com/haoxuan-unt2024/MetaGAI-Benchmark.

Haoxuan Zhang, Ruochi Li, Yang Zhang +4

University of North Texas, Denton, TX, USA · North Carolina State University, Raleigh, NC, USA

Automated EvaluationGenerative AI Evaluation
AskPDF
Aug 25, 2026

Automatic Model Card Generation Using an LLM

Model cards are structured documents that summarize key information about machine learning models to improve transparency, usability, and accountability. However, they often lack a consistent structure, and many models provide no model cards, making comparison and interpretation difficult. This paper presents two contributions. First, we propose MCTidy, an LLM-based approach that reorganizes existing model cards into a standardized template to improve clarity and comparability. Second, we introduce MCGenie, an LLM-based system that generates model cards directly from model repository data. We apply MCTidy to 48 Hugging Face model cards and evaluate information retention, section alignment, hallucination, and stability. Our findings show high information retention with minimal textual loss, accurate section assignment, rare hallucinations primarily in descriptive sections, and strong stability across runs. We assess MCGenie by generating model cards for the same 48 models and assessing semantic similarity, factual correctness, and sensitivity to input resources. The generated model cards achieved high semantic similarity (mean around 0.9); over half were fully correct, and most remaining errors were minor. Generation quality depended strongly on the availability of supporting resources, particularly associated papers. Overall, our findings demonstrate the potential of LLM-based methods to enable scalable, standardized model card documentation.

Tajkia Rahman Toma, Balreet Grewal, Cor-Paul Bezemer

University of Alberta, Edmonton, Canada

Topic pending
AskPDF
Apr 19, 2026

Toward Reusability of AI Models Using Dynamic Updates of AI Documentation

This work addresses the challenge of disseminating reusable artificial intelligence (AI) models accompanied by AI documentation (a.k.a., AI model cards). The work is motivated by the large number of trained AI models that are not reusable due to the lack of (a) AI documentation and (b) the temporal lag between rapidly changing requirements on AI model reusability and those specified in various AI model cards. Our objectives are to shorten the lag time in updating AI model card templates and align AI documentation more closely with current AI best practices. Our approach introduces a methodology for delivering agile, data-driven, and community-based AI model cards. We use the Hugging Face (HF) repository of AI models, populated by a subset of the AI research and development community, and the AI consortium-based Zero Draft (ZD) templates for the AI documentation of AI datasets and AI models, as our test datasets. We also address questions about the value of AI documentation for AI reusability. Our work quantifies the correlations between AI model downloads/likes (i.e., AI model reuse metrics) from the HF repository and their documentation alignment with the ZD documentation templates using tables of contents and word statistics (i.e., AI documentation quality metrics). Furthermore, our work develops the infrastructure to regularly compare AI documentation templates against community-standard practices derived from millions of uploaded AI models in the Hugging Face repository. The impact of our work lies in introducing a methodology for delivering agile, data-driven, and community-based standards for documenting AI models and improving AI model reuse.

Peter Bajcsy, Walid Keyrouz

Topic pending
AskPDF