cs.AIOct 10, 2026

Reels-RSI: Program-Level Self-Improvement for Code-Rendered Explainer Videos

Authors: Wei Lin

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Explainer videos -- a worked geometry problem, why the sky is blue, how a dynasty fell -- are usually one more target for pixel video generation. We argue they are a different kind of artifact: a program (script, storyboard and per-frame animation code) that a deterministic renderer turns into frames. A program can be read, diffed, tested and audited; pixels can only be watched. Reels-RSI is an open-source reel director for any agent harness; its self-improvement happens at the program level rather than in weights or prompts. Every revision pass is logged with a code snapshot; a retrospective pairs each check finding with the diff that fixed it and proposes lessons; a lesson that a static check can catch is compiled into a rule with a must-fail and a must-pass example, so the memory is unit-tested; and a rule learned today runs over everything made before it. A bounded, benchmark-gated step lets an agent rewrite the director's playbook, kept only when blind pairwise judgments on held-out topics prefer it; nothing enters memory without a human decision. Across 111 reels -- a field study, two playbook evolutions, a 2x2 memory ablation, a weaker director, a 48-reel learning curve and a judge-reliability study -- we find that the rule learned in the field caught four latent defects in frames authored before it existed and none after; that rules are an audit for a strong director (eight unguarded reels, no violation) and a guard for a weaker one (thirty violations unguarded versus five); that lessons shorten revision loops but do not compound or move the judge; and that the first rewrite's 11--1 verdict was a position bias in our vote design (judges pick the second-shown reel 62--81% of the time), masking scripts that traded thirteen facts for craft. We document four failure modes of the preference signal, which check caught each, and the fixes. Code, rules, lessons and data are public.

Figures & tables

 Pixel generationProgram rendering (Reels-RSI, Manim-based agents)
Text and numberssampled; may be wrong or illegibleexact strings in the program
Correctness checkwatch the video (human or VLM)read the program; the render is deterministic
Targeted editregeneratechange one value, re-render one frame
Learning a mistakefine-tune or re-promptwrite a check over the program text
Applying it to old outputimpossiblerun the check over the corpus
Portability across modelsnonethe check is code; the model is irrelevant
Table 1: Two routes to an explainer video.
Figure 1: Reels-RSI. The creative half (script, storyboard, frame code, review) belongs to the agent; the deterministic half is a CLI the agent calls. Memory sits beside both and is written only by checks and by human-gated acceptance.
GroupReelsFramesLengthLang. / aspectMade by
Pre-existing2 (Yuan dynasty; why the sky is blue)1966–96 szh, 16:9predecessor skill, no rules
Reviewed2 (meeting problem; textbook geometry from a photo)1655–84 szh, 9:16Opus, reviewer at 3 gates
Headless17 (8 topics ×\times 2 skills, +1)8326–33 szh/en, 16:9Sonnet, one sentence in
Dubbed2 (the geometry reel in en, ja)1884 sen, ja, 9:16reels dub
Total23136 zh, en, ja 
Table 2: The corpus. Dubs of the geometry reel into Spanish, French and Korean exist but were not rendered at the time of writing and are excluded.
ReelsFramesAuthoredFindingsWhere
2 pre-existing19before the rule4Yuan: frames 06, 09, 11; sky: frame 04
21 later117with the rule active0 
Table 3: Retroactive audit with visible-from-state . The other twelve rules were ported from the predecessor’s checker, under which the pre-existing reels were made, and produced no findings on any reel; the signal here is specific to the rule learned later.
 Absolute composite (0–100)Blind pairwise votes
 baselinecandidatebaselinecandidate
Train (4 topics ×\times 3 votes)75.374.9111
Holdout (4 topics ×\times 3 votes)72.873.548
Table 4: The first evolve round. Composite =0.6⋅=0.6\cdot judge rubric + 0.4⋅+\,0.4\cdot deterministic score; the deterministic part was at or near 100 for both versions, so the absolute columns are an affine rescaling of the judge’s 1–5 rubric. Every held-out topic split 2–1.
Figure 2: Why do we have seasons? under the baseline skill (left) and the evolved skill (right): same topic, same model; each row is one frame sampled at 30, 60 and 92% of its duration. The judge preferred the right 3–0.
FailureWhat the signal producedGate that caught it
(a) Judge misreadinga proposed lesson asserting a non-existent render bughuman accept ; then sample sheets sized per aspect, evidence ranked
(b) Writing for the samplera playbook edit about “the 30/60/92 % samples”human apply ; then the proposer prompt forbids sampling-directed text
(c) Craft bought with factspreferred reels whose narration drops 13 baseline facts over 6 topicsnone in the loop; found by reading the scripts
(d) Position bias in our own vote designa 3-vote alternation that gave the candidate the second slot twice; judges pick the second-shown reel 62–81 % of the time ( p<10−5p<10^{-5} for the round’s judge)the reliability study (§ 5.9 ); the gate now judges each side from its disadvantaged slot
Table 5: Four failure modes of the judge signal observed around one round, and where each stopped.
TopicStated by the baseline, absent from the candidateAdded by the candidateVotes
sci-rainbowwhite light split into seven colours; violet at ∼\sim 40 ∘ ; the viewer at the apex of a cone—0–3
hist-chibiCao Cao had just unified the north; the Sun–Liu alliance held the river; soldiers unacclimatisedchained ships became the fire attack’s target1–2
solve-chicken-rabbit—the general rule (assume all chickens; halve the missing feet)0–3
en-seasonsEarth is closest to the Sun in early January; in winter light spreads thin, days shrink—0–3
sci-tidesthe two high tides are ∼\sim 12 h 25 min apart because the Moon moves onthe Moon’s pull differs across the Earth1–2
hist-silk-roadEmperor Wu of Han; Zhang Qian as envoy; Buddhism travelling eastYumen Pass; porcelain west, glass east1–2
Table 6: Fact coverage of the two scripts per topic, read by hand (a fact = a number, name, date or causal step). Votes are baseline–candidate from the round. The candidate drops thirteen baseline facts over six topics and adds six over five; the judge preferred it on all eight.
Figure 3: Red Cliffs, baseline (left) and evolved (right). The baseline’s fourth row shows the Sun–Liu alliance holding the river; the evolved reel has no such frame, and its narration never names the alliance. The judge preferred the right 2–1 for its larger, animated fire scene.
 memoryrule violationsfinalizerenderer warn.judgepairwise, 4 topics ×\times 4 votes
Conditionruleslessonsin the outputpassesat last passrubricvs lessons offvs rules off
Aonon017058.78–8 (vs C)5–11 (vs B)
Boffon0 †10060.57–9 (vs D)—
Conoff011359.8—5–11 (vs D)
Doffoff0 †12358.0——
Table 7: The 2 ×\times 2 memory ablation, four training topics each, Sonnet directing. † The offline audit of the eight unguarded reels raised one finding, a counter-clockwise ring in a Red Cliffs map; the region renders correctly because that kit projects points into a plain SVG path, where ring direction is irrelevant, so it was a false positive of polygon-winding , which now fires only where a ring becomes GeoJSON and ships this kit’s pattern as a must-pass example.
Directorrulesfiringsviolations leftpassesrenderer errors leftdet. scorecost
haikuon185 (1 reel)74845$2.58
haikuoff030 (2 reels)46758$2.00
sonnet (Table 7 )on00280100$6.32
sonnet (Table 7 )off—0220100$7.29
Table 8: The same four topics under a weaker director. “Firings” are rule findings raised at finalize while authoring; “violations left” are what all thirteen rules find in the delivered reels. Sonnet rows pool the two lessons conditions.
Figure 4: Per reel, in order: renderer findings at the first finalize pass, finalize passes to a clean build, and rule firings; lessons accumulating (blue) against none (orange). Topics cycle science, history, problem, English, which is the period in the orange spikes.
 candidate wins (sonnet, 3 repeats)other judges, 1 run
Topicshown firstshown secondbalancedhaikuopus
sci-rainbow0/36/60.50CC
hist-chibi3/36/61.00CC
solve-chicken-rabbit2/36/60.83CC
en-seasons3/36/61.00CC
sci-tides3/36/61.00CC
Table 9: Pairwise judging by position. “Balanced” is the mean of the two position-specific win rates. C/B: topic winner by majority for the other judges. Topic-winner agreement between the five judge runs is 0.82.
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

RuleSeverityWhat it catches
timeline-registrationerrora sub-composition that does not register its paused timeline under its composition id
nondeterminismerrorMath.random , Date.now , requestAnimationFrame , timers or self-driving animation in a seek-rendered frame
cjk-fonterrorCJK text without an @font-face for a shipped font (the render browser has no CJK system fonts)
dasharray-attrerrora stroke-dasharray attribute in the markup of a path that a tween draws on
svgorigin-one-sidederrorsvgOrigin given only in a fromTo ’s to-vars
negative-zindexerrora negative z-index, which puts the layer behind the composition root
Appendix
Table 10: Rules shipped with Reels-RSI 0.1. A user’s own rules live beside them and may override a built-in of the same name.
More in this topic

Self-Improving Agents

Self-improving agents use experience and feedback to update their models, memories, skills, or workflows for better future task performance.

See all 171 papers

Explore similar work

CardsList
Sep 29, 2026

Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution

Video understanding agents acquire evidence through an executable harness that controls what they observe and how they use those observations. However, execution traces contain only the evidence acquired by the current harness, leaving competing explanations for failure unresolved and limiting the basis for self-improvement. We introduce Video-RSI, a framework for recursive self-improvement in which a video understanding agent uses its own language model to revise its harness. Through active video investigation, the model revisits the original training videos to test competing failure explanations with additional observations, grounding proposed changes in evidence beyond the existing trace. Cost-aware harness evolution turns these diagnoses into reusable revisions and determines which revisions to retain by considering both answer accuracy and visual cost. Across our evaluation settings on video understanding benchmarks, the evolved agent improves accuracy while processing fewer frames and achieves competitive accuracy-efficiency trade-offs against existing video understanding agents. These results demonstrate the potential for video understanding agents to improve their own evidence acquisition and use through harness evolution. Code is available at https://github.com/bingjunluo/Video-RSI .

Bingjun Luo, Jialin Guo, Siqi Li

Tsinghua University · Harbin Engineering University

Video UnderstandingAgent Harness OptimizationLLM Agent Self-ImprovementSelf-Improving Agents
AskPDF
May 19, 2026

PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoning

Programmatic video generation through code offers geometric precision and temporal coherence beyond pixel-level diffusion models, yet rigorously evaluating whether language models can produce spatially correct animated outputs remains an open problem. We introduce PRISM, a large-scale benchmark of 10,372 human-calibrated instruction-code pairs (20 times larger than prior programmatic video generation benchmarks), grounded in real-world knowledge visualization scenarios across English and Chinese and spanning 437 subject categories. We further propose a funnel-style evaluation framework with four complementary metrics: Code-Level Reliability for executability, Spatial Reasoning for layout correctness over full animation sequences, and Prompt-Aware Dynamic Visual Complexity (PADVC) and Temporal Density (TD) for diagnosing dynamic expression and temporal activity. Systematic evaluation of seven mainstream LLMs reveals a striking Execution-Spatial Gap: the average drop from execution success rate to spatial pass rate is approximately 41%, showing that runnable code does not necessarily yield spatially coherent visual output. These findings show that programmatic video generation evaluation should go beyond executability. PRISM provides a principled benchmark for advancing spatially coherent code generation.

Qiran Zhang, Yuheng Wang, Runde Yang +9

School of Artificial Intelligence, Shanghai Jiao Tong University

Spatial ReasoningCode Generation BenchmarksLLM Code GenerationVideo Generation Evaluation
AskPDF
Oct 14, 2026

VideoHarness-SRSI: Structured Recursive Self-Improvement for Video Understanding Harness

LLM system capabilities depend not only on model parameters but also on the executable harnesses that organize information acquisition, tool use, reasoning, and memory. Recursive harness evolution can automate their design, but how it can reliably discover generalizable systems from limited, noisy, and repeatedly reused feedback remains unclear. We study this question in agentic video understanding, where rich multimodal evidence and coupled mechanisms make harness optimization challenging. We first introduce VHarness-BRSI, a recursive self-improvement framework that evolves video harnesses around a frozen vision-language model. Analyzing over 4,000 evolved candidates, we find it competitive with hand-engineered systems but identify three interconnected failure modes: validation overfitting, search-budget dilution, and the early-experience trap. We trace all three to a common root: the same imperfect validation evidence drives both harness revisions and the proposer's search strategy. We therefore propose VHarness-SRSI, an evidence-governed framework in which every search decision rests on evidence that is transferable, sufficient, and gathered before the search commits. It restricts the proposer to content-free feedback, informs exploration with instrumented warm-up experiments, and develops mechanisms in isolated probe-race branches before factorial composition and joint refinement. Across four video understanding benchmarks, VideoHarness-SRSI improves the equally weighted mean held-out score by 6.2% relative to VideoHarness-BRSI and reduces the mean validation-test gap by 37.9%. Reliable harness evolution thus depends on how search is organized and informed, not only on how well it optimizes validation performance.

Zihao Lin, Haoyu Dong, Haobo Xu +10

University of California, Davis · Meta MRS Research · Duke University +3

LLM Agent HarnessesRecursive Self-ImprovementVideo UnderstandingVideo Reasoning
AskPDF