NERFIFY: A Multi-Agent Framework for Turning NeRF Papers into Code

Seemandhar Jain; Keshav Gupta; Kunal Gupta; Manmohan Chandraker

cs.CVFeb 28, 2026

NERFIFY: A Multi-Agent Framework for Turning NeRF Papers into Code

Authors: Seemandhar Jain, Keshav Gupta, Kunal Gupta, Manmohan Chandraker

Organizations: University of California, San Diego

Abstract

The proliferation of neural radiance field (NeRF) research requires significant efforts to reimplement papers before building upon them. We introduce NERFIFY, a multi-agent framework that reliably converts NeRF research papers into trainable Nerfstudio plugins, in contrast to generic paper-to-code methods and frontier models like GPT-5 that usually fail to produce runnable code. NERFIFY achieves domain-specific executability through six key innovations: (1) Context-free grammar (CFG): LLM synthesis is constrained by Nerfstudio formalized as a CFG, ensuring generated code satisfies architectural invariants. (2) Graph-of-Thought code synthesis: Specialized multi-file-agents generate repositories in topological dependency order, validating contracts and errors at each node. (3) Compositional citation recovery: Agents automatically retrieve and integrate components (samplers, encoders, proposal networks) from citation graphs of references. (4) Visual feedback: Artifacts are diagnosed through PSNR-minima ROI analysis, cross-view geometric validation, and VLM-guided patching to iteratively improve quality. (5) Knowledge enhancement: Beyond reproduction, methods can be improved with novel optimizations. (6) Benchmarking: An evaluation framework is designed for NeRF paper-to-code synthesis across 30 diverse papers. On papers without public implementations, NERFIFY achieves visual quality matching expert human code (+/-0.5 dB PSNR, +/-0.2 SSIM) while reducing implementation time from weeks to minutes. NERFIFY demonstrates that a domain-aware design enables code translation for complex vision papers, potentiating accelerated and democratized reproducible research. Code, data and implementations will be publicly released.

Figures & tables

Figure 1: Overview. Manual NeRF implementation requires weeks of specialized effort (left). Existing paper-to-code systems fail to produce trainable code. Nerfify automates this process through grammar-constrained synthesis and compositional citation recovery, generating fully trainable Nerfstudio plugins in minutes (right).

Figure 2: Nerfify converts NeRF papers into code through four stages: (1) Agent parses and summarizes PDFs into simple markdown, CFG from Nerfstudio and curated paper-code pairs as In-Context examples are saved in $\mathcal{K}$ (2) Compositional dependency resolution traverses citation graphs to retrieve missing components from referenced papers, (3) GoT code synthesis generates repository files through specialized agents operating in topological order (4) Visual refinement iteratively patches artifacts until achieving expert-level quality.

Figure 3: NeRF citation dependency graphs. Implementing K-Planes requires retrieving components from 7 direct dependencies (Plenoxels, TensoRF, Instant-NGP, Mip-NeRF 360, DyNeRF, EG3D, NeRF-W) and 12 total papers with transitive dependencies. Our compositional citation recovery automatically traverses such graphs to identify and retrieve all necessary components.

Figure 4: Graph-of-Thought (GoT) Multi-Agent Code Synthesis . The master agent orchestrates specialized file-agents that progressively build a NeRF repository over k steps. Each step shows files being created or modified through four stages: (1) DAG Construction maps papers to Nerfstudio component dependencies, (2) Interface Freeze establishes API contracts in topological order, (3) Implementation generates validated code with shape/gradient checks, (4) Integration Testing runs smoke tests with automated repair. Files evolve from minimal interfaces to complete implementations as agents coordinate through the dependency graph, producing runnable NeRF plugins.

Paper	Reported			Human Impl.			Nerfify (Ours)
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
KeyNeRF [ 38 ]	25.65	0.89	0.11	25.70	0.89	0.12	26.12	0.90	0.09
mi-MLP NerF [ 66 ]	24.70	0.89	0.09	22.64	0.87	0.15	22.85	0.87	0.15
ERS [ 46 ]	27.85	0.94	0.06	26.87	0.90	0.12	27.02	0.90	0.12
TVNeRF [ 61 ]	27.44	0.93	0.08	26.81	0.92	0.12	27.30	0.92	0.10

Table 1: Comparison of Nerfify with paper and human implementations. We evaluate NeRF papers from the Nerfify-Bench set whose code is not publicly available, using SSIM, PSNR, and LPIPS metrics. Note. Other baselines like Paper2Code, AutoP2C, GPT-5 and R1 failed to generate trainable code.

Metric	Paper2Code	AutoP2C	GPT-5	R1	Nerfify
Imports Resolve	✓	×	✓	✓	✓
Compiles/Trainable	×	×	×	×	✓
Training Stability	×	×	×	×	✓
Converges to Paper Results	×	×	×	×	✓

Table 2: Comparison of Nerfify with baselines in terms of executable code. We evaluate ability to produce functional, trainable implementations. All baselines fail to generate trainable code despite some producing syntactically valid Python.

Method	Original Repository			Nerfify
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Vanilla NeRF [ 32 ]	31.36	0.95	0.04	31.36	0.95	0.04
Nerfacto [ 47 ]	20.36	0.82	0.22	20.36	0.82	0.22
$\ell_{0}$ Sampler [ 24 ]	29.21	-	0.04	30.13	0.97	0.03
InfoNeRF [ 20 ]	18.27	0.81	0.23	17.87	0.69	0.44

Table 3: Comparison with existing implementations . Evaluation of Nerfify against original author repositories or gold-standard implementations (

\ell_{0}

Sampler doesn’t report SSIM).

Figure 5: Visual Comparison of Nerfify and Human Implementation . Left : Ground Truth Image, Middle : Expert Implementation, Right : Agent Implementation.

Paper	Paper2Code					AutoP2C					R1					GPT-5					Nerfify
	C $\uparrow$	I $\downarrow$	M $\downarrow$	W $\uparrow$	Score $\uparrow$	C $\uparrow$	I $\downarrow$	M $\downarrow$	W $\uparrow$	Score $\uparrow$	C $\uparrow$	I $\downarrow$	M $\downarrow$	W $\uparrow$	Score $\uparrow$	C $\uparrow$	I $\downarrow$	M $\downarrow$	W $\uparrow$	Score $\uparrow$	C $\uparrow$	I $\downarrow$	M $\downarrow$	W $\uparrow$	Score $\uparrow$
Mip-NeRF [ 2 ]	0.83	0.17	0.00	0.83	0.85	0.17	0.17	0.66	0.25	0.20	0.67	0.17	0.16	0.67	0.58	0.50	0.30	0.20	0.60	0.58	1.00	0.00	0.00	1.00	1.00
BioNeRF [ 39 ]	0.30	0.40	0.30	0.40	0.35	0.10	0.30	0.60	0.10	0.15	0.70	0.20	0.10	0.70	0.75	0.80	0.10	0.10	0.80	0.82	1.00	0.00	0.00	1.00	1.00
PyNeRF [ 48 ]	0.50	0.30	0.20	0.60	0.58	0.00	0.10	0.90	0.10	0.03	0.30	0.60	0.10	0.70	0.68	0.40	0.30	0.30	0.80	0.52	1.00	0.00	0.00	0.90	0.97
TensoRF [ 5 ]	0.20	0.30	0.50	0.30	0.12	0.10	0.20	0.70	0.15	0.28	0.60	0.20	0.20	0.70	0.65	0.70	0.10	0.20	0.75	0.72	1.00	0.00	0.00	0.95	0.98
Tetra-NeRF [ 22 ]	0.13	0.25	0.63	0.20	0.22	0.00	0.13	0.88	0.00	0.08	0.63	0.25	0.13	0.70	0.72	0.50	0.25	0.25	0.60	0.58	1.00	0.00	0.00	1.00	1.00

Table 4: Novelty coverage analysis across Nerfify-Bench papers . For each baseline system, we report: C (Correct implementation rate), I (Incorrect/partial implementation rate), M (Missing component rate), W (Hyperparameter weight match accuracy), and Score

{}_{\text{LLM}}

(overall semantic implementation score on 0-1 scale). Nerfify achieves perfect or near-perfect scores (C=1.00, M=0.00) across all papers, while generic baselines show significant component omissions (M=0.12-0.90 for most methods) and lower implementation fidelity. Metrics are computed over all novel components identified in each paper, with weights derived from paper emphasis and experimental validation.

Configuration	Score	Trainable	Correct Novelties
		(%)	Impl. (C)
Nerfify (Full)	0.98	100	1.00
Knowledge Sources:
w/o In-context Examples (Stage 1)	0.71	90	1.00
w/o citation recovery (Stage 2)	0.68	100	0.65
w/o Both	0.58	90	0.65

Table 5: Component ablation study . We evaluate the impact of each system component on synthesis quality and efficiency. Numbers are averaged over 10 papers from Nerfify-Bench .

Agentic Code Generation

Agentic code generation uses AI agents to plan, write, test, and revise executable programs with tools and feedback.

See all 124 papers

NeRFifyMesh: Optimizing Neural Radiance Fields from Textured Meshes for Robotics Scene Building
Oct 7, 2026Nillan Nimal, Mahboubeh Asadi, Sajad Saeedi3D Scene Representation Neural Radiance Fields
Department of Mechanical and Industrial Engineering, University of Toronto, Canada · Department of Mechanical, Industrial and Mechatronics Engineering, Toronto Metropolitan University, Canada · Department of Computer Science, University College London, UK
Low-Cost Neural Radiance Fields
May 10, 2026Alice Huang, Prathamesh Sonawane, Yashdeep Thorat +1Neural Rendering Sparse-View 3D Reconstruction
University of Illinois Urbana Champaign
SPLATIFY: Reproduce, Discover, Innovate! From Papers and Ideas to Trainable 3DGS Code
Oct 6, 2026Seemandhar Jain, Keshav Gupta, Manmohan ChandrakerCode Generation 3D Gaussian Splatting
University of California, San Diego

Oct 7, 2026J/K move · Enter open · S save

NeRFifyMesh: Optimizing Neural Radiance Fields from Textured Meshes for Robotics Scene Building

Nillan Nimal, Mahboubeh Asadi, Sajad Saeedi

Department of Mechanical and Industrial Engineering, University of Toronto, Canada · Department of Mechanical, Industrial and Mechatronics Engineering, Toronto Metropolitan University, Canada · Department of Computer Science, University College London, UK

3D Scene Representation Neural Radiance Fields

Open paper AskPDF arXiv

In robotics, scene representation plays a pivotal role in understanding and interacting with the environment. The advent of Neural Radiance Fields (NeRF) and its variants, as a novel representation, has opened a new frontier of research. In applications such as semantic mapping and simulation, roboticists aim to build scenes using multiple NeRF models, each representing an object. While extensive datasets of 3D mesh models already exist, there is an urgent need to develop tools to convert these assets to NeRF models for rapid algorithm development and testing. This paper presents a new pipeline for converting existing mesh models to NeRF representations by artificially generating a ground truth point-based radiance field through sampling mesh geometry and texture. This approach alleviates the need for camera-based sampling or rendering multi-view images of the original mesh to train the NeRF model. Extensive benchmarking demonstrates that our method yields comparable rendering quality to the baselines. Additionally, the application of this representation is shown by constructing unified NeRF scenes and performing collision simulations with extracted geometry.

Abstract

Agentic Code Generation

Explore similar work

NeRFifyMesh: Optimizing Neural Radiance Fields from Textured Meshes for Robotics Scene Building

Low-Cost Neural Radiance Fields

SPLATIFY: Reproduce, Discover, Innovate! From Papers and Ideas to Trainable 3DGS Code