Right Values, Wrong Places: Diagnosing and Fixing Record Misplacement in LLM-Generated JSON

Yiwei Zhang; Chengke Wu; Jiahao Li; Li Wang; Yao Yao; Jianqiang Li

cs.AIAug 26, 2026

Right Values, Wrong Places: Diagnosing and Fixing Record Misplacement in LLM-Generated JSON

Authors: Yiwei Zhang, Chengke Wu, Jiahao Li, Li Wang, Yao Yao, Jianqiang Li

Organizations: Shenzhen University · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences

Abstract

Large language models can generate valid JSON that preserves field values yet binds them to the wrong records or hierarchy levels. We introduce JSONPlacement, a controlled task supplying all values and assignment rules for transforming flat records into hierarchical JSON. Across six open-weight models, we identify two error types: record-assignment errors and hierarchy-realization errors. Whole-record moves account for 98.6% of strictly traceable synthetic misplacements. We predict that aligning input with target traversal should reduce assignment errors, while replacing nested output with record-level paths should reduce hierarchy errors. With predictions fixed before generating new instances, we test both interventions separately and together. On synthetic instances, each nearly eliminates its targeted error while leaving the other. Combined, they achieve exact matches for three 8-12B models on all new synthetic instances and a real-data validation set. These findings motivate evaluating record placement explicitly and jointly designing input organization and output representation to preserve value-path bindings.

Figures & tables

Figure 1: JSONPlacement and two error types. (a) Correct transformation. (b) Record-assignment error. (c) Hierarchy-realization error. Each four-cell color group represents one record. All three JSON structures are valid and preserve every value, but only (a) places all values correctly.

Structure	Levels	Branching factors	Records
Original	5	$2\times 3\times 2\times 2\times 2$	48
Shallow–wide	2	$6\times 8$	48
Deep–narrow	5	$2\times 2\times 2\times 2\times 3$	48

Table 1: Synthetic target structures. Levels count grouping fields rather than full JSON path depth.

	Original			Shallow–wide			Deep–narrow
Model	VP (%)	VPA (%)	EM	VP (%)	VPA (%)	EM	VP (%)	VPA (%)	EM
Qwen3.5-4B	85.96	53.08	0/24	97.83	97.14	9/24	76.06	37.04	0/24
Qwen3.5-9B	92.62	83.51	0/24	99.48	99.39	18/24	78.99	62.50	0/24
Qwen3.5-27B	99.74	99.57	20/24	100.00	100.00	24/24	98.26	98.00	12/24
Gemma 4 12B IT	97.22	53.39	1/24	100.00	99.74	22/24	87.50	70.20	0/24
Gemma 4 31B IT	99.91	99.91	23/24	100.00	100.00	24/24	98.35	98.00	12/24

Table 2: Baseline performance across six models and three target structures.

Figure 2: Baseline VP and VPA on Original as records per instance increase. Red bars mark values that are preserved but misplaced; numbers above the axis give exact matches out of 24. Each record count uses separately generated instances.

Model	Order	Path	EM	VPA (%)	Other	Outside	Omitted	EM	VPA (%)	Other	Outside	Omitted
			Original					Deep–narrow
					Record errors					Record errors
Qwen3.5-9B	–	–	0/24	80.82	111	0	110	0/24	60.33	121	3	286
	✓	–	24/24	100.00	0	0	0	23/24	99.48	6	0	0
	–	✓	0/24	80.42	84	0	104	0/24	72.14	107	0	103
	✓	✓	24/24	100.00	0	0	0	24/24	100.00	0	0	0

Table 3: Validation on new instances for the two interventions and their combination.

Synthetic, 27–31B (48/64/96 records, 24 each)
	Baseline errors			EM
Model	Oth.	Out.	Omit.	Base	Order	Path	Both
Qwen3.5-27B	9	0	16	60	72	53	72
Gemma 4 31B	1	0	17	56	72	62	72
PDB, 8–12B (48 instances)
Qwen3.5-9B	8	14	22	18	48	11	48

Table 4: Gains of each intervention for larger models and real records.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	Definition
VP	Target leaves matched one-to-one to output values, ignoring paths / all target leaves
VPA	Target leaves with the correct value and type at the target path / all target leaves
Output precision	Correctly bound target leaves / all raw output leaf entries, including duplicates
Whole-instance accuracy	Entirely correct outputs / instances with model outputs; no missing or extra paths, duplicate declarations, or conflicts

Appendix

Table 5: Evaluation metrics.

Structure	Logistics	School
Original	tenant, order, shipment, package, role (sender/recipient)	school, campus, class, student, role (student/guardian)
Shallow–wide	tenant, order	school, student
Deep–narrow	tenant, order, shipment, package, address	school, campus, grade, class, student

Appendix

Table 6: Grouping fields by structure and domain, from the top level down. Each field has a corresponding plural container, such as tenants .

Condition	Input	IDs	Output
Baseline	Shuffled	Repeated	Nested
Aligned	Target	Repeated	Nested
Unique IDs	Shuffled	Unique	Nested
Leaf-level paths	Shuffled	Repeated	Leaf paths
Record-level paths	Shuffled	Repeated	Record paths

Appendix

Table 7: What each condition changes. All conditions keep the records and tree shape; only unique identifiers change the target paths.

	Original		Shallow + Deep
Model	Logistics	School	Logistics	School
Qwen 9B	88/96	84/96	176/192	180/192
Gemma 12B	94/96	91/96	181/192	184/192
Ministral 8B	95/96	95/96	188/192	–
OLMo 7B	–	–	73/192	–
Total	1,529/1,728

Appendix

Table 8: Scorable outputs in the pooled audit. Each cell covers four conditions with 24 instances per structure.

Model	Cond.	Scor.	VPA (%)	EM	Whole
Qwen 9B	$96\times 2$	15/24	62.43	0/24	67/67
	$48\times 4$	24/24	64.08	0/24	114/114
	$24\times 8$	24/24	71.68	0/24	82/83
	General	66/72	63.40	0/72	310/311
	Path	64/72	66.92	0/72	252/252
Gemma 12B	$96\times 2$	24/24	67.06	0/24	189/189

Appendix

Table 9: Field-count and checking controls.

r\times f

: records per instance

\times

fields per record. Whole: records moved whole / records with at least one misplaced field. General and Path are the checking instructions, pooled over the three configurations.

	Path similarity		Input position
	Occupied	One ID (%)	Loss (%)	Ratio
Qwen	2,122/2,340	86.1 / 11.8	60.2 / 14.1	4.3 [3.7, 5.0]
Gemma	787/1,616	91.2 / 10.6	30.0 / 10.3	2.9 [2.5, 3.4]

Appendix

Table 10: Path similarity and input position. Occupied: absent records whose target is taken by another record. One ID: occupants differing from the absent record by one identifier, observed / reference, over 3,136 (Qwen 9B) and 1,850 (Gemma 12B) pairs. Loss: absence rate of the first 20% of input records / the rest, with the ratio and 95% interval.

				Record errors
Struct.	IDs	EM	VPA (%)	Other	Outside	Omitted
Qwen3.5-9B
Original	Base	0/24	80.82	111	0	110
	Unique	3/24	89.93	8	81	27
Deep	Base	0/24	60.33	121	3	286
	Unique	0/24	90.08	0	64	36

Appendix

Table 11: Unique-identifier diagnostic on new instances. Columns as in Table 3 ; Base rows repeat its baseline. Deep: Deep–narrow.

Gemma 4 12B IT
Structure	Cond.	EM	PC	UC	Omit.	Outside
Original	U	10/24	14	3	0	336
	J	8/24	16	4	0	384
	T	11/24	0	0	13	0
	T+C	24/24	0	0	0	0
Deep–narrow	U	1/24	21	7	0	504

Appendix

Table 12: Constrained-decoding diagnostic. U: unconstrained; J: JSON only; T: tree shape; T+C: tree shape plus tenant count. PC, UC: outputs flagged for premature closure and under-closure. Omit.: outputs missing a whole tenant. Outside: records outside the target tree.

Comparison	Records / fields	Base instances	Outputs	Pairing and reuse
Fresh 2 $\times$ 2	48 / 4	48	576	2 structures $\times$ 24; 3 models $\times$ 4 arms.
Local shuffling	48 / 4	24	144	Earlier Original contents; 2 models $\times$ 3 orders.
Large-model gradient	48, 64, 96 / 4	72	576	3 levels $\times$ 24; 2 models $\times$ 4 arms. The 48-record contents overlap fresh validation.
PDB	48–66 / 8	48	576	47 eligible entries plus 1 supplement; 3 models $\times$ 4 arms.
MBTA	48 / 4	24	192	2 paired trip-ID variants $\times$ 2 models $\times$ 2 ordering arms.

Appendix

Table 13: Intervention cohorts and exact reporting scope. The appendix keeps distinct validation purposes and reused contents explicit.

Model	Structure	Arm	EM	S	I/T	VP	VPA	Prec.	Tokens
Qwen 9B	Original	Base	0/24	24	0/0	90.45	80.82	81.02	3480
		Order	24/24	24	0/0	100.00	100.00	100.00	3480
		Path	0/24	20	0/4	89.17	80.42	84.70	3565
		Both	24/24	24	0/0	100.00	100.00	100.00	3640
Qwen 9B	Deep–narrow	Base	0/24	23	0/1	75.36	60.33	41.56	6416
		Order	23/24	24	0/0	100.00	99.48	99.48	3732

Appendix

Table 14: Fresh 2

\times

2 validation: accuracy, scoring coverage, and output length. Each row attempts 24 outputs from 48-record instances.

Model	Structure	Arm	N	Other	Outside	Absent	Conflict	Copies
Qwen 9B	Original	Base	1152	111	0	110	0	107
		Order	1152	0	0	0	0	0
		Path	960	84	0	104	0	60
		Both	1152	0	0	0	0	0
Qwen 9B	Deep–narrow	Base	1104	121	3	286	28	1085
		Order	1152	6	0	0	0	0

Appendix

Table 15: Fresh 2

\times

2 validation: complete record diagnostics. The three record-error columns in the main-text table are supplemented by target conflicts and extra copies.

Model	Order	EM	VP	VPA	Prec.	Other	Outside	Absent	Copies	Tokens
Qwen 9B	Shuffled	0/24	92.27	83.85	83.85	97	0	89	89	3482
	Local	10/24	98.18	96.79	97.14	16	0	21	17	3481
	Order	24/24	100.00	100.00	100.00	0	0	0	0	3481
Gemma 12B	Shuffled	2/24	97.31	55.73	56.98	4	475	31	4	3484
	Local	3/24	99.91	56.16	56.21	0	504	1	0	3540
	Order	7/24	100.00	64.58	64.58	0	408	0	0	3541

Appendix

Table 16: Local-shuffle check on reused Original instances. Local permutes records within contiguous target-rank blocks of eight.

Model	Records	Arm	EM	S	I/T	VP	VPA	Prec.	Tokens
Qwen 27B	48	Base	23/24	24	0/0	99.91	99.91	99.91	3480
		Order	24/24	24	0/0	100.00	100.00	100.00	3480
		Path	17/24	24	0/0	99.39	99.22	99.74	3638
		Both	24/24	24	0/0	100.00	100.00	100.00	3640
Qwen 27B	64	Base	18/24	24	0/0	99.35	99.15	99.74	4635
		Order	24/24	24	0/0	100.00	100.00	100.00	4635

Appendix

Table 17: Large-model record-count gradient: all four conditions. Each record contains four payload fields.

Model	Records	Arm	N	Other	Absent	Copies
Qwen 27B	48	Base	1152	0	1	1
		Path	1152	2	7	1
Qwen 27B	64	Base	1536	3	10	1
		Path	1536	3	5	0
Qwen 27B	96	Base	2304	6	5	1
		Path	2304	3	14	3

Appendix

Table 18: Large-model record diagnostics by difficulty level. Base and Path are displayed; the two uniformly correct arms are specified in the notes.

Model	Arm	EM	S	I/T	VP	VPA	Prec.	Tokens
Qwen 9B	Base	18/48	46	0/2	98.22	96.52	92.68	6216
	Order	48/48	48	0/0	100.00	100.00	100.00	5986
	Path	11/48	40	0/8	98.81	97.36	96.95	6644
	Both	48/48	48	0/0	100.00	100.00	100.00	6620
Gemma 12B	Base	22/48	47	1/0	99.02	97.68	98.44	6482
	Order	48/48	48	0/0	100.00	100.00	100.00	6452

Appendix

Table 19: PDB validation: all model–condition results. Each source entry supplies 48–66 atom records.

Model	Arm	N	Other	Outside	Absent	Conflict	Copies
Qwen 9B	Base	2467	8	14	22	56	221
	Order	2574	0	0	0	0	0
	Path	2139	14	10	25	17	40
	Both	2574	0	0	0	0	0
Gemma 12B	Base	2522	0	29	26	2	7
	Order	2574	0	0	0	0	0

Appendix

Table 20: PDB record diagnostics. N explicitly records the denominator lost through invalid or truncated outputs.

Trip IDs	Model	Arm	EM	S	I/T	VP	VPA	Prec.	Tokens
Native	Qwen 9B	Base	8/24	23	0/1	99.55	98.53	92.90	3466
		Order	24/24	24	0/0	100.00	100.00	100.00	3346
	Gemma 12B	Base	24/24	24	0/0	100.00	100.00	100.00	3443
		Order	24/24	24	0/0	100.00	100.00	100.00	3443
Local trip	Qwen 9B	Base	3/24	24	0/0	98.31	95.42	86.02	3432
		Order	24/24	24	0/0	100.00	100.00	100.00	3298

Appendix

Table 21: MBTA ordering validation with native and local trip IDs. Only Base and Order belong to this comparison.

Trip IDs	Model	Arm	N	Other	Outside	Absent	Conflict	Copies
Native	Qwen 9B	Base	1104	9	4	2	8	77
		Order	1152	0	0	0	0	0
	Gemma 12B	Base	1152	0	0	0	0	0
		Order	1152	0	0	0	0	0
Local trip	Qwen 9B	Base	1152	3	23	23	10	261
		Order	1152	0	0	0	0	0

Appendix

Table 22: MBTA record diagnostics. Copies remain visible even when almost every target value is correctly bound.

Structured Output Generation

Structured output generation studies how language models produce machine-readable, schema-compliant responses without sacrificing task accuracy.

See all 65 papers

Structure Tax: How Structured Output affects LLMs Performance
Oct 8, 2026Vineet Kumar, Kanishka, Bhuvanesh MandoraStructured Output Generation Structured Reasoning in Language Models
PayPal Artificial Intelligence PayPal, Bengaluru India
Constrained Decoding Eliminates Structural Failures in Small LLMs but Reveals a Scale-Dependent Semantic Gap
Sep 20, 2026Akash ChavanConstrained Decoding LLM Evaluation
Real-Time Trustworthiness Scoring for LLM Structured Outputs and Data Extraction
Feb 24, 2026Hui Wen Goh, Jonas MuellerLLM Answer Verification LLM Uncertainty Estimation
Cleanlab

Oct 8, 2026J/K move · Enter open · S save

Structure Tax: How Structured Output affects LLMs Performance

Vineet Kumar, Kanishka, Bhuvanesh Mandora

PayPal Artificial Intelligence PayPal, Bengaluru India

Structured Output Generation Structured Reasoning in Language Models LLM Reliability

Open paper AskPDF arXiv

Deploying large language models in production often requires constraining outputs to structured formats such as JSON or XML, and prior work treats the resulting accuracy loss as an inherent

structure tax'. We re-examine this claim by evaluating a battery of models, datasets and schemas, measuring task accuracy, confidence calibration, and hidden-state geometry. The tax turns out to depend on schema design rather than on structure per se: reasoning-first field ordering matches or exceeds free-form accuracy, while answer-first ordering causes steep drops, particularly in smaller models. Format sensitivity scales inversely with a task's own structural constraints, and schemas that preserve reasoning order also improve calibration with CKA showing greater separability between correct and incorrect representations in middle transformer layers. Our findings indicate that properly designed structured formats can match or exceed free-form performance, reframing the critical question from

whether to structure' to `how to structure' for optimal reasoning preservation.

Abstract

Structured Output Generation

Explore similar work

Structure Tax: How Structured Output affects LLMs Performance

Constrained Decoding Eliminates Structural Failures in Small LLMs but Reveals a Scale-Dependent Semantic Gap

Real-Time Trustworthiness Scoring for LLM Structured Outputs and Data Extraction