Children's book illustrations carry developmental, pedagogical, and cultural responsibilities that are often formally codified in guidelines. As AI systems begin entering children's publishing pipelines, both as evaluators and as generators, a natural question arises: can they reliably meet the structural, safety, and cultural standards required? We investigate this across three criteria: guideline compliance, safety, and cultural alignment, using Indonesia's Book Leveling Guideline as a government-issued evaluation framework with explicit visual criteria for four reading levels. To understand how well VLMs can identify and apply these guideline criteria, we first evaluate them on real published children's book pages and compare their assessments against a human baseline. To then examine whether AI-generated illustrations meet the same standards, we apply the VLM judge to images produced by two generation models and cross-check them against a subset of human illustrator raters. We find that VLMs are reliable on explicit criteria such as compliance and safety, but fall short on cultural alignment, systematically missing content that human raters flag as culturally inappropriate. AI-generated illustrations show the same blind spot: they satisfy structural and safety criteria but fail to capture implicit cultural details that human raters consistently identify.
Figures & tables
Figure 1: Left: original human illustration. Middle and right: AI-generated images by Imagen 4 and FLUX.1 [dev], respectively. Both AI images depict a shower fixture, which is uncommon in Indonesian homes where bathing typically uses a dipper from a standing tub. This cultural detail is captured by the human illustrator but missed by both generators.
Level
Reader Profile
Ages
Reading Ability
A
Emergent Reader
0–7
Emergent; requires full scaffolding
B1
Early/Beginning Reader
6–8
Short sentences; builds reading accuracy
B2
Early/Beginning Reader
7–9
Moderate sentences; richer vocabulary
B3
Early/Beginning Reader
8–10
Simple paragraphs; dialogue introduced
Table 1: Reader classification in Indonesia’s Book Leveling Guideline.
Figure 2: A sample page from reading level A (0–7 yr). The guideline checklist for this level requires: 2D/3D illustration style, illustration dominant over text, full color, no dialogue balloons, and at most 3 sentences.
Figure 3: Row-normalized confusion matrices for level classification (N=20). Cell values are raw counts. A and B3 are anchor levels; B2 is the hardest level for all annotators.
Model
Cult. Avg
Top-rating %
Safety FP
Human (majority)
1.92
23.3%
0.0%
Claude Sonnet 4.6
1.85
15.0%
0.0%
Qwen3-VL 8B
1.60
0.0%
0.0%
SEA-LION-v4-8B
1.85
0.0%
0.0%
Table 2: Cultural alignment and safety on the 20-page human subset. Cultural score is on a 1–3 scale. Top-rating % is the share of pages receiving the highest cultural rating. Safety FP = rate of incorrectly flagging safe pages.
Figure 4: The one page from Level B2 flagged as potentially unsafe by Claude.
Criterion
FLUX.1 [dev]
Imagen 4
Q1: Visual guidelines
1.35
2.18
Q2: Cultural elements
1.53
1.76
Q3: Age-appropriateness
1.65
2.12
Q4: Logical coherence
1.59
1.87
Overall
1.53
1.97
Preferred (%)
12.5
40.6
Table 3: Task 2 human evaluator scores per criterion (scale 1–3, higher is better). Level A, 3 raters, 2 books, 3 pages each.
Figure 5: Examples of anatomy failures in AI-generated pages. Left: Imagen 4. Middle and right: FLUX.1 [dev].
Figure 6: An Imagen 4 generated page where batik patterns appear on the mat, clothing, and toys with similar repetitions. While batik is a real Indonesian cultural element, the uniform overuse across all objects was flagged by an annotator as unrealistic.
Although image generation has boosted various applications via its rapid evolution, whether the state-of-the-art models are able to produce ready-to-use academic illustrations for papers is still largely unexplored. Directly comparing or evaluating the illustration with VLM is native but requires oracle multi-modal understanding ability, which is unreliable for long and complex texts and illustrations. To address this, we propose AIBench, the first benchmark using VQA for evaluating logic correctness of the academic illustrations and VLMs for assessing aesthetics. In detail, we designed four levels of questions proposed from a logic diagram summarized from the method part of the paper, which query whether the generated illustration aligns with the paper on different scales. Our VQA-based approach raises more accurate and detailed evaluations on visual-logical consistency while relying less on the ability of the judger VLM. With our high-quality AIBench, we conduct extensive experiments and conclude that the performance gap between models on this task is significantly larger than general ones, reflecting their various complex reasoning and high-density generation ability. Further, the logic and aesthetics are hard to optimize simultaneously as in handcrafted illustrations. Additional experiments further state that test-time scaling on both abilities significantly boosts the performance on this task.
Cultural representation in Large Language Model (LLM) outputs has primarily been evaluated through the proxies of cultural diversity and factual accuracy. However, a crucial gap remains in assessing cultural alignment: the degree to which generated content mirrors how native populations perceive and prioritize their own cultural facets. In this paper, we introduce a human-centered framework to evaluate the alignment of LLM generations with local expectations. First, we establish a human-derived ground-truth baseline of importance vectors, called Cultural Importance Vectors based on an induced set of culturally significant facets from open-ended survey responses collected across nine countries. Next, we introduce a method to compute model-derived Cultural Representation Vectors of an LLM based on a syntactically diversified prompt-set and apply it to three frontier LLMs (Gemini 2.5 Pro, GPT-4o, and Claude 3.5 Haiku). Our investigation of the alignment between the human-derived Cultural Importance and model-derived Cultural Representations reveals a Western-centric calibration for some of the models where alignment decreases as a country's cultural distance from the US increases. Furthermore, we identify highly correlated, systemic error signatures (ρ>0.97) across all models, which over-index on some cultural markers while neglecting the deep-seated social and value-based priorities of users. Our approach moves beyond simple diversity metrics toward evaluating the fidelity of AI-generated content in authentically capturing the nuanced hierarchies of global cultures.
Erin MacMurray van Liemt, Aida Davani, Sinchana Kumbale +2
Children increasingly have access to Large Language Models (LLMs), which may expose them to responses that are developmentally inappropriate or require age-sensitive safety, guidance, and boundaries. Existing LLM safety evaluations largely focus on general harmful-content avoidance and do not explicitly target child-facing safety. We introduce KIDBench, a benchmark for evaluating child-facing LLM safety for ages 7-11 using a LLM-as-a-Judge rubric grounded in developmental-psychology. KIDBench contains realistic child queries across ten categories, with single-turn prompts and multi-turn child-actor simulations. We compare no-cues prompts with no child context, implicit-cues prompts that suggest a child speaker, and explicit age instructions. Implicit-cues improve scores by 8.6-46.8% over no-cue, while explicit age provides an additional 9.9-30.4% improvement over implicit-cues. Cross-lingual and cultural evaluations show uneven safety behavior across languages and country contexts. Multi-turn simulations show peak quality drops of up to 0.959 points on the 1-5 scale. We also introduce KIDGuardLlama, a child-safety evaluator, and KIDLlama, a child-safe response model. Code, data, and evaluation resources are available at https://github.com/MichiganNLP/kidbench.