RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robots

Soroush Nasiriany; Sepehr Nasiriany; Abhiram Maddukuri; Yuke Zhu

cs.ROMar 4, 2026

RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robots

Authors: Soroush Nasiriany, Sepehr Nasiriany, Abhiram Maddukuri, Yuke Zhu

Organizations: The University of Texas at Austin · NVIDIA Research

Abstract

Recent advances in robot learning have accelerated progress toward generalist robots that can perform everyday tasks in human environments. Yet it remains difficult to gauge how close we are to this vision. The field lacks a reproducible, large-scale benchmark for systematic evaluation. To fill this gap, we present RoboCasa365, a comprehensive simulation benchmark for household mobile manipulation. Built on the RoboCasa platform, RoboCasa365 introduces 365 everyday tasks across 2,500 diverse kitchen environments, with over 600 hours of human demonstration data and over 1600 hours of synthetically generated demonstration data -- making it one of the most diverse and large-scale resources for studying generalist policies. RoboCasa365 is designed to support systematic evaluations for different problem settings, including multi-task learning, robot foundation model training, and lifelong learning. We conduct extensive experiments on this benchmark with state-of-the-art methods and analyze the impacts of task diversity, dataset scale, and environment variation on generalization. Our results provide new insights into what factors most strongly affect the performance of generalist robots and inform strategies for future progress in the field.

Figures & tables

Figure 1: Overview of RoboCasa365. RoboCasa365 is a large-scale simulation framework for training and benchmarking generalist robots. RoboCasa365 includes 365 everyday tasks, 2500 diverse kitchen scenes, over 600 hours of human demonstration data, plus 1600 hours of synthetically generated demonstration data, and systematic benchmarks for training and evaluating generalist robot models.

Figure 2: Kitchen Scenes. Our simulation framework features 2,500 distinct kitchen scenes for pretraining (top, representative samples shown), and 10 distinct target kitchen scenes (bottom, all scenes shown).

Figure 3: Composite Tasks. RoboCasa365 features 300 composite tasks that involve a sequence of skills. We use large language models to generate a set of high-level activities, and for each activity, a set of task blueprints. There are 6 activity families (high-level categories) spanning 60 activities, which organize composite tasks based on shared functional and semantic structure. Representative tasks are shown for selected activities.

Panel of Figure 4 — Figure 4: Distribution of task lengths (by number of subtasks) and dataset episode lengths (by number of seconds). We observe a long tail of tasks and data representing long-horizon behaviors.

Task Split	Diffusion Policy	$\bm{\pi_{0}}$	$\bm{\pi_{0.5}}$	GR00T N1.5
Atomic	15.7	36.3	39.6	43.0
Composite-Seen	0.2	5.2	7.1	9.6
Composite-Unseen	1.25	0.7	1.2	4.4
Average	6.1	15.0	16.9	20.0

Table 1: Multi-task Training Results. We compare state-of-the-art policy learning approaches on our human pretraining data across 300 tasks, and report task success rates (%) across seen and unseen tasks. We see that learning composite tasks is more challenging, and that performance suffers when evaluating on unseen tasks.

Task Type	Pretraining Only	Target Only			Pretraining + Target Post-Training
Task Type		10%	30%	100%	10%	30%	100%
Atomic	41.9	38.7	50.6	60.6	56.9	59.1	68.5
Composite-Seen	0.0	11.0	22.7	35.0	25.4	34.6	40.6
Composite-Unseen	0.2	11.2	27.5	33.3	22.7	30.8	42.1
Average	15.1	21.0	34.3	43.7	35.9	42.2	51.1

Table 2: Foundation Model Training Results. Comparing the impact of training on pretraining and target datasets on learning downstream tasks. The performances are measured by average task success rates (%).

Figure 5: Foundation Model Training Results. Pre-training enables more effective learning of downstream tasks with significant gains in data efficiency.

Phase	Atomic Tasks	2-3 Stage Tasks	4–5 Stage Tasks	6+ Stage Tasks
Phase 1	41.5	-	-	-
Phase 2	13.9	24.5	-	-
Phase 3	13.9	4.8	11.3	-
Phase 4	10.6	1.7	2.7	4.3

Table 3: Lifelong Learning Results. We train across four phases with progressively longer horizon tasks. After each phase, we report task success rates (%) across all tasks seen in the current and previous phases.

Target Data	Pretraining Data
Target Data	No Pretraining	Human50	Human300	Human300 + MG60
Atomic (10%)	38.7	52.0	57.0	56.9
Composite-Seen (10%)	11.0	26.2	28.7	25.4
Composite-Unseen (10%)	11.2	23.8	32.3	22.7
Average (10%)	21.0	34.7	40.0	35.9
Atomic (100%)	60.6	68.1	70.0	68.5

Table 4: Pretraining Task Diversity Results. We report task success rates (%) and compare the downstream effects of training on different mixtures of pretraining data.

Figure 6: Real-Robot Platform. Our real-world setup features a Panda robot arm in a real kitchen.

	CloseElectric KettleLid	PickPlaceToasterOven ToCounter	PickPlaceCounter ToCabinet	PlaceOn DishRack	Avg
Real Only	70	70	52	55	61.8
Sim-and-Real (Ours)	70	100	84	65	79.8

Table 5: Real-world evaluations. Across four real-world tasks, we compare training on real-world data only versus training on a mixture of our simulation and real-world data. By additionally using simulation data, it outperforms training on real-world data only by an average task success rate of 18.1%.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Unique models
Blender	22
Coffee machine	48
Dishwasher	25
Electric kettle	25
Fridge	50
Microwave	50

Appendix

Table 6: Inventory of fixtures and appliances

Activity	Task	# Subtasks	MoMa req.	Description
Atomic	CloseBlenderLid	1	No	Close the lid blender by securely placing the lid on top.
Atomic	CloseFridge	1	No	Close the fridge door(s).
Atomic	CloseToasterOvenDoor	1	No	Close the toaster oven door.
Atomic	CoffeeSetupMug	1	No	Pick the mug from the counter and place it under the coffee machine dispenser.
Atomic	NavigateKitchen	1	Yes	Navigate to the [ kitchen location ].
Atomic	OpenCabinet	1	No	Open the cabinet door(s).

Appendix

Figure 7: Post-training Atomic-Seen Tasks (18)

Activity	Task	# Subtasks	MoMa req.	Description
Serving beverages	DeliverStraw	4	Yes	Take a straw from the drawer in front and place it inside the glass cup on the dining counter.
Toasting bread	GetToastedBread	4	Yes	Start the toaster. Once the lever pops up, take the bread to the plate on the dining counter.
Brewing	KettleBoiling	2	No	Pick the kettle from the counter and place it on a stove burner. Then turn the burner on.
Loading dishwasher	LoadDishwasher	3	No	Pick up the items from the counter, place them in the dishwasher, and close the dishwasher door.
Packing lunches	PackIdenticalLunches	15	Yes	Place two identical items of each object in each tupperware on the nearby counter, to pack two identical lunches.
Washing dishes	PreSoakPan	3	No	Pick the pan and sponge and place them into the sink. Then turn on the water.

Appendix

Figure 8: Post-training Composite-Seen Tasks (16)

Activity	Task	# Subtasks	MoMa req.	Description
Setting the table	ArrangeBreadBasket	5	Yes	Open the cabinet, pick up the item from the cabinet and place it in the basket. Then move the basket to the dining counter.
Brewing	ArrangeTea	3	No	Pick the kettle from the counter and place it on the tray. Then pick the mug from the cabinet and place it on the tray. Then close the cabinet doors.
Making toast	BreadSelection	2	Yes	From the different types of pastries on the counter, select a croissant and place it on the cutting board. Then retrieve a jar of jam from the cabinet and place it alongside the croissant on the cutting board.
Arranging condiments	CategorizeCondiments	2	No	Put the shaker and condiment bottle from the counter next to their counterparts in the cabinet.
Chopping vegetables	CuttingToolSelection	2	No	Place the appropriate cutting tool for cutting the item skin on the cutting board.
Garnishing dishes	GarnishPancake	4	Yes	Take the strawberry from the fridge and place it on top of the pancake, located on the dining counter.

Appendix

Figure 9: Post-training Composite-Unseen Tasks (16)

Setting	Num Tasks	Num Scenes	Demos per Task	Dataset Size (hrs)
Pretraining (Human)	300	2500	100	404
Pretraining (MimicGen)	60	2500	10,000	1615
Target (Human)	50	10	500	208

Appendix

Table 7: Dataset statistics across pretraining and target settings.

Figure 10: Camera Images. Camera images from three views (rendered at $256\times 256$ resolution are fed into the model.)

	Pretraining Data
	5 Scenes	25 Scenes	2500 Scenes
Zero-shot Evaluation	29.6	39.6	44.7
+ Fine-tuning on Atomic Target Data (10%)	53.3	56.7	62.4

Appendix

Table 8: Pretraining scene diversity results. Increasing the composition of scenes in pretraining data improves downstream task performance.

Task Split	No Perturbation	Novel Language	Camera Perturbations	Initial Joint Noise	Initial Base Pose Noise
Composite-Seen	40.6	38.3	28.8	27.9	31.2
Composite-Unseen	42.1	39.2	31.5	32.1	30.2

Appendix

Table 9: Evaluation of robustness under different perturbations.

	Atomic-Seen	Composite-Seen	Composite-Unseen	Average
Full fine-tuning	43.0	9.6	4.4	20.0
LoRA fine-tuning	2.4	0.2	0.8	1.2

Appendix

Table 10: Policy success rates (%) comparing full vs LoRA fine-tuning for GR00T N1.5.

Task	Success Rate (%)	Stages	MoMa Required
Atomic-Seen
TurnOnElectricKettle	93	1	No
OpenStandMixerHead	90	1	No
CloseToasterOvenDoor	87	1	No
OpenCabinet	87	1	No
SlideDishwasherRack	87	1	No

Appendix

Table 11: Foundation Model Training Results.

Robot Manipulation Benchmarks

Robot manipulation benchmarks measure how reliably robots perceive, plan, and execute physical tasks across objects, settings, and embodiments.

See all 291 papers

RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
Apr 10, 2026Jenai Xuning Yang, Rishit Dagli, Alex Zook +5Robotic Policy Evaluation Robot Policy Generalization
NVIDIA · University of Toronto · Work done during internship at NVIDIA +1
RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies
Jul 5, 2026Tianxing Chen, Yue Chen, Zixuan Li +41Long-Horizon Robotic Manipulation Language-Conditioned Robot Manipulation
RoboSynChallenge: Mastering Real-World Dexterity via Generalizing Synthesized Manipulation Skills
Aug 12, 2026Runyi Zhao, Ruixin Wu, Chengkun Li +15Sim-to-Real Transfer Generalization in Robotic Manipulation
Shenzhen Loop Area Institute (SLAI) · The Chinese University of Hong Kong, Shenzhen · DexForce +8

Apr 10, 2026J/K move · Enter open · S save

RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies

Jenai Xuning Yang, Rishit Dagli, Alex Zook, Hugo Hadfield+4

NVIDIA · University of Toronto · Work done during internship at NVIDIA+1

Robotic Policy Evaluation Robot Policy Generalization Robotics Simulation Robot Manipulation Benchmarks Synthetic Data Generation for Robotics

Open paper AskPDF arXiv

The pursuit of general-purpose robotics has yielded impressive foundation models, yet simulation-based benchmarking remains a bottleneck due to rapid performance saturation and a lack of true generalization testing. Existing benchmarks often exhibit significant domain overlap between training and evaluation, trivializing success rates and obscuring insights into robustness. We introduce RoboLab, a simulation benchmarking framework designed to address these challenges. Concretely, our framework is designed to answer two questions: (1) to what extent can we understand the performance of a real-world policy by analyzing its behavior in simulation, and (2) which factor most strongly affect policy behavior. First, RoboLab enables human-authored and LLM-enabled generation of scenes and tasks in a robot- and policy-agnostic manner within a high-fidelity simulation environment. We introduce an accompanying RoboLab-120 benchmark, consisting of 120 tasks categorized into three competency axes: visual, procedural, relational, across three difficulty levels. Second, we introduce a systematic analysis of real-world policies that quantify both their performance and the sensitivity of their behavior to controlled perturbations, exposing significant performance gap in current state-of-the-art models. By providing granular metrics and a scalable toolset, RoboLab offers a scalable framework for evaluating the true generalization capabilities of task-generalist robotic policies. Project website: https://research.nvidia.com/labs/srl/projects/robolab/.

Abstract

Robot Manipulation Benchmarks

Explore similar work

RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies

RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies

RoboSynChallenge: Mastering Real-World Dexterity via Generalizing Synthesized Manipulation Skills