Reels-RSI: Program-Level Self-Improvement for Code-Rendered Explainer Videos

Wei Lin

cs.AIOct 10, 2026

Reels-RSI: Program-Level Self-Improvement for Code-Rendered Explainer Videos

Authors: Wei Lin

Abstract

Explainer videos -- a worked geometry problem, why the sky is blue, how a dynasty fell -- are usually one more target for pixel video generation. We argue they are a different kind of artifact: a program (script, storyboard and per-frame animation code) that a deterministic renderer turns into frames. A program can be read, diffed, tested and audited; pixels can only be watched. Reels-RSI is an open-source reel director for any agent harness; its self-improvement happens at the program level rather than in weights or prompts. Every revision pass is logged with a code snapshot; a retrospective pairs each check finding with the diff that fixed it and proposes lessons; a lesson that a static check can catch is compiled into a rule with a must-fail and a must-pass example, so the memory is unit-tested; and a rule learned today runs over everything made before it. A bounded, benchmark-gated step lets an agent rewrite the director's playbook, kept only when blind pairwise judgments on held-out topics prefer it; nothing enters memory without a human decision. Across 111 reels -- a field study, two playbook evolutions, a 2x2 memory ablation, a weaker director, a 48-reel learning curve and a judge-reliability study -- we find that the rule learned in the field caught four latent defects in frames authored before it existed and none after; that rules are an audit for a strong director (eight unguarded reels, no violation) and a guard for a weaker one (thirty violations unguarded versus five); that lessons shorten revision loops but do not compound or move the judge; and that the first rewrite's 11--1 verdict was a position bias in our vote design (judges pick the second-shown reel 62--81% of the time), masking scripts that traded thirteen facts for craft. We document four failure modes of the preference signal, which check caught each, and the fixes. Code, rules, lessons and data are public.

Figures & tables

	Pixel generation	Program rendering (Reels-RSI, Manim-based agents)
Text and numbers	sampled; may be wrong or illegible	exact strings in the program
Correctness check	watch the video (human or VLM)	read the program; the render is deterministic
Targeted edit	regenerate	change one value, re-render one frame
Learning a mistake	fine-tune or re-prompt	write a check over the program text
Applying it to old output	impossible	run the check over the corpus
Portability across models	none	the check is code; the model is irrelevant

Table 1: Two routes to an explainer video.

Panel of Figure 1 — Figure 1: Reels-RSI. The creative half (script, storyboard, frame code, review) belongs to the agent; the deterministic half is a CLI the agent calls. Memory sits beside both and is written only by checks and by human-gated acceptance.

Group	Reels	Frames	Length	Lang. / aspect	Made by
Pre-existing	2 (Yuan dynasty; why the sky is blue)	19	66–96 s	zh, 16:9	predecessor skill, no rules
Reviewed	2 (meeting problem; textbook geometry from a photo)	16	55–84 s	zh, 9:16	Opus, reviewer at 3 gates
Headless	17 (8 topics $\times$ 2 skills, +1)	83	26–33 s	zh/en, 16:9	Sonnet, one sentence in
Dubbed	2 (the geometry reel in en, ja)	18	84 s	en, ja, 9:16	reels dub
Total	23	136		zh, en, ja

Table 2: The corpus. Dubs of the geometry reel into Spanish, French and Korean exist but were not rendered at the time of writing and are excluded.

Reels	Frames	Authored	Findings	Where
2 pre-existing	19	before the rule	4	Yuan: frames 06, 09, 11; sky: frame 04
21 later	117	with the rule active	0

Table 3: Retroactive audit with visible-from-state . The other twelve rules were ported from the predecessor’s checker, under which the pre-existing reels were made, and produced no findings on any reel; the signal here is specific to the rule learned later.

	Absolute composite (0–100)		Blind pairwise votes
	baseline	candidate	baseline	candidate
Train (4 topics $\times$ 3 votes)	75.3	74.9	1	11
Holdout (4 topics $\times$ 3 votes)	72.8	73.5	4	8

Table 4: The first evolve round. Composite

=0.6\cdot

judge rubric

+\,0.4\cdot

deterministic score; the deterministic part was at or near 100 for both versions, so the absolute columns are an affine rescaling of the judge’s 1–5 rubric. Every held-out topic split 2–1.

Figure 2: Why do we have seasons? under the baseline skill (left) and the evolved skill (right): same topic, same model; each row is one frame sampled at 30, 60 and 92% of its duration. The judge preferred the right 3–0.

Failure	What the signal produced	Gate that caught it
(a) Judge misreading	a proposed lesson asserting a non-existent render bug	human accept ; then sample sheets sized per aspect, evidence ranked
(b) Writing for the sampler	a playbook edit about “the 30/60/92 % samples”	human apply ; then the proposer prompt forbids sampling-directed text
(c) Craft bought with facts	preferred reels whose narration drops 13 baseline facts over 6 topics	none in the loop; found by reading the scripts
(d) Position bias in our own vote design	a 3-vote alternation that gave the candidate the second slot twice; judges pick the second-shown reel 62–81 % of the time ( $p<10^{-5}$ for the round’s judge)	the reliability study (§ 5.9 ); the gate now judges each side from its disadvantaged slot

Table 5: Four failure modes of the judge signal observed around one round, and where each stopped.

Topic	Stated by the baseline, absent from the candidate	Added by the candidate	Votes
sci-rainbow	white light split into seven colours; violet at $\sim$ 40 ∘ ; the viewer at the apex of a cone	—	0–3
hist-chibi	Cao Cao had just unified the north; the Sun–Liu alliance held the river; soldiers unacclimatised	chained ships became the fire attack’s target	1–2
solve-chicken-rabbit	—	the general rule (assume all chickens; halve the missing feet)	0–3
en-seasons	Earth is closest to the Sun in early January; in winter light spreads thin, days shrink	—	0–3
sci-tides	the two high tides are $\sim$ 12 h 25 min apart because the Moon moves on	the Moon’s pull differs across the Earth	1–2
hist-silk-road	Emperor Wu of Han; Zhang Qian as envoy; Buddhism travelling east	Yumen Pass; porcelain west, glass east	1–2

Table 6: Fact coverage of the two scripts per topic, read by hand (a fact = a number, name, date or causal step). Votes are baseline–candidate from the round. The candidate drops thirteen baseline facts over six topics and adds six over five; the judge preferred it on all eight.

Figure 3: Red Cliffs, baseline (left) and evolved (right). The baseline’s fourth row shows the Sun–Liu alliance holding the river; the evolved reel has no such frame, and its narration never names the alliance. The judge preferred the right 2–1 for its larger, animated fire scene.

	memory		rule violations	finalize	renderer warn.	judge	pairwise, 4 topics $\times$ 4 votes
Condition	rules	lessons	in the output	passes	at last pass	rubric	vs lessons off	vs rules off
A	on	on	0	17	0	58.7	8–8 (vs C)	5–11 (vs B)
B	off	on	0 †	10	0	60.5	7–9 (vs D)	—
C	on	off	0	11	3	59.8	—	5–11 (vs D)
D	off	off	0 †	12	3	58.0	—	—

Table 7: The 2

\times

2 memory ablation, four training topics each, Sonnet directing. † The offline audit of the eight unguarded reels raised one finding, a counter-clockwise ring in a Red Cliffs map; the region renders correctly because that kit projects points into a plain SVG path, where ring direction is irrelevant, so it was a false positive of polygon-winding , which now fires only where a ring becomes GeoJSON and ships this kit’s pattern as a must-pass example.

Director	rules	firings	violations left	passes	renderer errors left	det. score	cost
haiku	on	18	5 (1 reel)	7	48	45	$2.58
haiku	off	0	30 (2 reels)	4	67	58	$2.00
sonnet (Table 7 )	on	0	0	28	0	100	$6.32
sonnet (Table 7 )	off	—	0	22	0	100	$7.29

Table 8: The same four topics under a weaker director. “Firings” are rule findings raised at finalize while authoring; “violations left” are what all thirteen rules find in the delivered reels. Sonnet rows pool the two lessons conditions.

Figure 4: Per reel, in order: renderer findings at the first finalize pass, finalize passes to a clean build, and rule firings; lessons accumulating (blue) against none (orange). Topics cycle science, history, problem, English, which is the period in the orange spikes.

	candidate wins (sonnet, 3 repeats)			other judges, 1 run
Topic	shown first	shown second	balanced	haiku	opus
sci-rainbow	0/3	6/6	0.50	C	C
hist-chibi	3/3	6/6	1.00	C	C
solve-chicken-rabbit	2/3	6/6	0.83	C	C
en-seasons	3/3	6/6	1.00	C	C
sci-tides	3/3	6/6	1.00	C	C

Table 9: Pairwise judging by position. “Balanced” is the mean of the two position-specific win rates. C/B: topic winner by majority for the other judges. Topic-winner agreement between the five judge runs is 0.82.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Rule	Severity	What it catches
timeline-registration	error	a sub-composition that does not register its paused timeline under its composition id
nondeterminism	error	Math.random , Date.now , requestAnimationFrame , timers or self-driving animation in a seek-rendered frame
cjk-font	error	CJK text without an @font-face for a shipped font (the render browser has no CJK system fonts)
dasharray-attr	error	a stroke-dasharray attribute in the markup of a path that a tween draws on
svgorigin-one-sided	error	svgOrigin given only in a fromTo ’s to-vars
negative-zindex	error	a negative z-index, which puts the layer behind the composition root

Appendix

Table 10: Rules shipped with Reels-RSI 0.1. A user’s own rules live beside them and may override a built-in of the same name.

Self-Improving Agents

Self-improving agents use experience and feedback to update their models, memories, skills, or workflows for better future task performance.

See all 171 papers

Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution
Sep 29, 2026Bingjun Luo, Jialin Guo, Siqi LiVideo Understanding Agent Harness Optimization
Tsinghua University · Harbin Engineering University
PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoning
May 19, 2026Qiran Zhang, Yuheng Wang, Runde Yang +9Spatial Reasoning Code Generation Benchmarks
School of Artificial Intelligence, Shanghai Jiao Tong University
VideoHarness-SRSI: Structured Recursive Self-Improvement for Video Understanding Harness
Oct 14, 2026Zihao Lin, Haoyu Dong, Haobo Xu +10LLM Agent Harnesses Recursive Self-Improvement
University of California, Davis · Meta MRS Research · Duke University +3

Sep 29, 2026J/K move · Enter open · S save

Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution

Bingjun Luo, Jialin Guo, Siqi Li

Tsinghua University · Harbin Engineering University

Video Understanding Agent Harness Optimization LLM Agent Self-Improvement Self-Improving Agents

Open paper AskPDF arXiv

Video understanding agents acquire evidence through an executable harness that controls what they observe and how they use those observations. However, execution traces contain only the evidence acquired by the current harness, leaving competing explanations for failure unresolved and limiting the basis for self-improvement. We introduce Video-RSI, a framework for recursive self-improvement in which a video understanding agent uses its own language model to revise its harness. Through active video investigation, the model revisits the original training videos to test competing failure explanations with additional observations, grounding proposed changes in evidence beyond the existing trace. Cost-aware harness evolution turns these diagnoses into reusable revisions and determines which revisions to retain by considering both answer accuracy and visual cost. Across our evaluation settings on video understanding benchmarks, the evolved agent improves accuracy while processing fewer frames and achieves competitive accuracy-efficiency trade-offs against existing video understanding agents. These results demonstrate the potential for video understanding agents to improve their own evidence acquisition and use through harness evolution. Code is available at https://github.com/bingjunluo/Video-RSI .

Abstract

Self-Improving Agents

Explore similar work

Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution

PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoning

VideoHarness-SRSI: Structured Recursive Self-Improvement for Video Understanding Harness