Lifelong Robot Recomposition via Persistent Categorical Modeling for Unified Task-Driven Co-Design, Verification, and Planning

Steven Swanbeck; Mitch Pryor

cs.ROAug 21, 2026

Lifelong Robot Recomposition via Persistent Categorical Modeling for Unified Task-Driven Co-Design, Verification, and Planning

Authors: Steven Swanbeck, Mitch Pryor

Organizations: Texas Robotics and the Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, TX 78712, USA

Abstract

Robotic systems are traditionally designed and deployed in static configurations, with assumptions made at design-time becoming immutable constraints during runtime. This design-then-deploy paradigm produces performant systems under narrow operating conditions, but renders robots brittle when qualities of themselves, their tasks, or their environments unexpectedly change. We address this challenge with a compositional framework that formalizes robotic systems as abstract circuits within a strict symmetric monoidal category, in which design and runtime composition of hardware, software, and behavior are synthesized simultaneously via an SMT-based solver, with monoidal functors projecting the system into lifecycle-specific views and free symbolic variables simultaneously solving for parameters and entire component specifications within larger compositions. This persistent model also supports queries a long-lived system needs beyond plan existence across its entire lifecycle, including mapping Pareto fronts over candidate compositions, diagnosing why a composition has become infeasible, finding its minimal restoration, and reconfiguring with limited change to the deployed system. We evaluate against official implementations of optimal numeric, stream-based, and SMT-based planners all measured onboard a deployed robot and demonstrate the approach end-to-end in a search-and-rescue scenario in which the robot recognizes when it has become unfit and synthesizes and assumes new holistic configurations to restore operation. We release our solver and supporting software open-source.

Figures & tables

Fig. 1: Informal graphical illustration of lifelong robot recomposition. Via unified compositional abstraction, hardware, software, and behavior modules can be consistently reasoned over and composed. This process extends beyond pre-deployment co-design to also support recomposition of the system during runtime in response to unexpected changes in system capabilities, mission requirements, or the environment.

	Components			Lifecycle	Configuration Space	Queries Answered
Thread & Representative Work	HW	SW	Behavior	Lifecycle	Configuration Space	Exist.	Opt.	Diag.	Relax	Stab.	Pareto
Modular/self-reconfigurable robots [ 17 , 18 ]	$\bullet\!\!\!\circ$	$\times$	$\times$	Runtime	Unit lattice	$\bullet\!\!\!\circ$	$\circ$ a	$\times$	$\times$	$\circ$ b	$\times$
Plug-and-produce/reconfigurable manufacturing [ 22 ]	$\bullet\!\!\!\circ$	$\circ$	$\times$	Both	Device catalog	$\bullet\!\!\!\circ$	$\circ$	$\times$	$\times$	$\times$	$\times$
Self-adaptation, [email protected] [ 24 , 25 , 27 ]	$\times$	$\bullet\!\!\!\circ$	$\circ$	Runtime	Enum. variants	$\bullet\!\!\!\circ$	$\circ$ c	$\circ$ d	$\times$	$\circ$ e	$\times$
Categorical/compositional modeling [ 29 , 10 , 33 ]	$\circ$	$\circ$	$\circ$	Both	–	–	–	–	–	–	–
Task-driven co-design (MCDP) [ 11 , 12 ]	$\bullet\!\!\!\circ$	$\bullet\!\!\!\circ$	$\circ$	Design	Design catalog	$\bullet\!\!\!\circ$	$\bullet\!\!\!\circ$	$\times$	$\times$	$\times$	$\bullet\!\!\!\circ$

TABLE I: Positioning of this work against the threads of Section II . Components is what the method has agency over; lifecycle describes the stage during which each tool is used; configuration space is the set from which a configuration is drawn; queries answered are the questions the method can pose over its own model.

\bullet\!\!\!\circ

= supported,

\circ

= supported in a restricted form (see notes),

\times

= not supported, – = not applicable. The separation is structural, as the component and query blocks are satisfied by disjoint sets of rows. Diagnosis, relaxation, and stability are mature capabilities over plans and planning domains, not over deployed systems, while methods that do reconfigure heterogeneous systems support no queries beyond the configurations they produce.

Fig. 2: Development of the combined assessment stack, including hypothetical STT and VLM software components (top left). Solving for the free parameters within the subcomposition yields actionable design targets (top center). A combination of five STT candidates (Tiny, Base, Small, Medium, and Large variants of Whisper [ 57 ] ) and eleven VLM candidates (E2B, E4B, 12B, 26B, and 31B variants of Gemma4 [ 58 ] and 0.8B, 2B, 4B, 9B, 27B, and 35B variants of Qwen3.5 [ 59 ] ) were evaluated to generate the Pareto front (bottom). Of the 55 total compositions, 20 non-dominated compositions form the front. Based on these results, the implemented software components use the Small variant of Whisper and the 26B parameter variant of Gemma4, respectively, with both adhering to the standard of [ 61 ] (top right).

[-0.2em] Transcribe

\theta_{t}

PromptVLM

\theta_{p}

Verbalize

\theta_{v}

+\;\;

[-0.2em]

\mu

\theta_{\mu}

\;=\;\;

[-0.2em] Transcribe

\theta_{t}

\mu

\theta_{\mu}

PromptVLM

\theta_{p}

Verbalize

\theta_{v}

=

[-0.2em] VLMInteraction

\theta

\;\to\kern-15.00002pt

⬇ <BehaviorTree ID =" VLMInteraction " > <Sequence> <Transcribe transcription =

\,\theta_{t,1}\to

"{ text }" /> <

\mu\to

impl( TakePhoto ) > image_topic =

\,\theta_{\mu,1}\to

"/ camera / color / image_raw " image =

\,\theta_{\mu,2}\to

"{ image }" /> <PromptVLM image =

\,\theta_{p,1}\to

"{ image }" text =

\,\theta_{p,2}\to

"{ text }" response =

\,\theta_{p,3}\to

"{ response }" /> <Verbalize text =

\,\theta_{v,1}\to

"{ response }" /> </Sequence> </BehaviorTree>

Fig. 3: A simple behavior tree to interact verbally using the STT and VLM models optimized in Figure 2 plus TTS capabilities is developed. An initial composition is solved with a free structural variable

\mu

. The conditions of

\mu

are solved such that the composition is valid. This behavior tree is stored as a new VLMInteraction behavior component, which is reasoned over in Section V-B . Once a real behavior matching the solved specification for

\mu

has been implemented, during runtime, the remaining free parametric variables

\theta

are solved and the morphism application of VLMInteraction is grounded to the executable behavior tree shown right, translating the symbolic action into real robot behavior.

[-0.2em] LiDAR

\theta

\;\to

⬇ lidar: mount: +: mount to

\!\!\theta_{1}

\!\!\to

" payload_plate " communication: +: plug into ethernet port

\!\theta_{2}

\!\!\!\to

2 power: +: connect to 12 V power at usbc power port

\!\theta_{3}

\!\!\!\to

1 …

[-0.2em] [-0.2em] YOLOE

\,\theta

\!\to

⬇ services: yoloe: image: yoloe :

\theta_{1}

\theta_{2}\to

humble , arm64 … ⬇ yoloe: ros__parameters: classes: [

\theta_{3}

]

\to

[" person "] image_topic:

\!\!\!\theta_{4}\to

"/ camera / color / image_raw " …

Fig. 4: Runtime configurations for a LiDAR hardware component and a YOLOE [ 56 ] software component adherent to the standard of [ 61 ] . Because the robot in our demonstration ( Figure 5 ) cannot reconfigure its own hardware, morphism applications associated with hardware are grounded to human-readable instructions (left) which must be carried out manually by a human teammate. In our demonstration, these instructions are provided via an augmented reality headset. Morphism applications associated with software are grounded into the compose and parameter YAML file formats required by [ 61 ] (right), and all reconfiguration and management of software is performed autonomously.

Fig. 5: Search and Rescue Demonstration: Counter-clockwise from top left: I) A quadruped robot and human teammate with an AR headset are tasked with performing a search-and-rescue mission. The robot cannot perform the task in its base composition ( $\mathrm{UNSAT}$ ), and solves for a new composition to achieve the task. Software and behavior recomposition are autonomous and instructions for hardware recomposition are provided to the teammate via the AR headset. Once reintegrated, the robot is $\mathrm{SAT}$ . II) During deployment, the robot exceeds a $10\text{\,}\mathrm{m}$ range constraint from the teammate. The robot tries to solve for a new composition that would render it $\mathrm{SAT}$ , and generates and executes a new behavior tree to move itself back within communication range. To continue exploring further into the facility, the teammate attaches a long-range radio component, which significantly extends the range constraint. III) Upon encountering a person, the robot approaches, interacts with the person, and shares their location and an assessment of their condition with the teammate. IV) During exploration, the robot enters a dark room and is no longer able to detect people within its RGB image stream, rendering it $\mathrm{UNSAT}$ . The robot solves for an updated composition to become $\mathrm{SAT}$ , which requires a physical swap of the RGB camera for a thermal one, corresponding software changes, and modifications to its behavior tree. In the updated composition, the robot is once again $\mathrm{SAT}$ . V) Using the thermal camera, the robot locates and interacts with a person in darkness, providing the same location and assessment information to the remote teammate.

Scenario	PDDL (ENHSP)	PS ${}_{\text{s}}$ (FD)	PS ${}_{\text{o}}$ (FD)	SMTP	Ours
Search and rescue demonstration ( Figure 5 ):
Figure 5 -I ( $\mathrm{SAT}$ )	✓ (1.01 s) SAT	n/e	n/e	✓ (2.63 s) SAT	✓ (6.46 s) SAT
Figure 5 -II ( $\mathrm{OPT}$ )	$\times$ (0.58 s) SUBOPT	n/e	n/e	$\times$ (0.21 s) SUBOPT	✓ (4.85 s) OPT
Figure 5 -IV ( $\mathrm{SAT}$ )	✓ (1.02 s) SAT	n/e	n/e	✓ (15.9 s) SAT	✓ (6.66 s) SAT
Microbenchmark:
GPU alloc. ( $\mathrm{OPT}$ )	✓ (0.51 s) OPT	✓ (13.1 s) OPT	✓ (0.56 s) OPT	$\times$ (0.05 s) SUBOPT	✓ (0.12 s) OPT

TABLE II: Solve times (seconds) on the search-and-rescue deployment ( Figure 5 -I/II/IV) and three microbenchmarks, all measured onboard the robot’s NVIDIA Jetson AGX Orin (median of 10 trials).

\mathrm{SAT}

rows seek any feasible plan; OPT rows seek an optimum. PDDL with ENHSP [ 64 ] ; PS

{}_{\text{s}}

/PS

{}_{\text{o}}

: PDDLStream sampler/optimizer [ 65 ] with Fast Downward (FD) [ 66 ] ; SMTP: SMTPlan+ [ 44 ] . ✓ = optimal (

\mathrm{OPT}

) or feasible (

\mathrm{SAT}

) plan found;

\times

= suboptimal (

\mathrm{SUBOPT}

) or infeasible (

\mathrm{UNSAT}

); — = optimizer inapplicable (feasibility-only row, no numeric objective, so the adaptive config reduces to the sampler); n/e = a valid encoding exists in principle but we did not author it for this problem: PDDLStream on the full-scale search-and-rescue model, whose hand-written stream/optimizer bodies are a substantial modeling effort we instead demonstrate on the microbenchmarks that isolate the same capabilities.

Mission-time event	PDDL (ENHSP)	PS ${}_{\text{s}}$ (FD)	PS ${}_{\text{o}}$ (FD)	SMTP	Ours
GPU upgrade: GPU pool $+$ $1\text{\,}\mathrm{GB}$ ( $\mathrm{OPT}$ )	0 L $\to$ $\times$ † (0.51 s) SUBOPT	0 L $\to$ ✓ (15.3 s) OPT	0 L $\to$ ✓ (0.55 s) OPT	0 L $\to$ $\times$ (0.05 s) SUBOPT	0 L $\to$ ✓ (0.12 s) OPT
Re-tasking: objective changed ( $\mathrm{OPT}$ )	34 L $\to$ ✓ (0.51 s) OPT	30 L $\to$ ✓ (13.1 s) OPT	32 L $\to$ ✓ (0.55 s) OPT	34 L $\to$ $\times$ (0.06 s) SUBOPT	0 L $\to$ ✓ (0.12 s) OPT
Increased precision: exact real measurement (7.73) ( $\mathrm{SAT}$ )	0 L $\to$ $\times$ (0.46 s) UNSAT	24 L $\to$ $\times$ (3.90 s) UNSAT	22 L $\to$ ✓ (0.53 s) SAT	0 L $\to$ ✓ (0.02 s) SAT	0 L $\to$ ✓ (0.02 s) SAT
Unmodeled request: out-of-vocabulary class ( $\mathrm{SAT}$ )	0 L $\to$ $\times$ (0.45 s) UNSAT	0 L $\to$ $\times$ (3.62 s) UNSAT	—	0 L $\to$ $\times$ (1.68 s) UNSAT	0 L $\to$ ✓ (0.11 s) SAT

TABLE III: Absorbing mission-time change. Each row perturbs a solved problem by one runtime event; cells report the model edits (lines outside the problem file, comments excluded) a human must make before re-solving, and the re-solve outcome with median time over 10 trials on the robot’s Jetson. Model-line counts are per configuration : added

+

removed substantive lines in the .pddl files each config loads–sampler domain

+

stream, optimizer domain

+

optimizer–excluding problem definitions. Outcome markers match Table II : ✓ = optimal/feasible;

\times

= suboptimal/infeasible; — = optimizer inapplicable.

Query	PDDL (ENHSP)	SMTP	Ours
Diagnose: why does the deployed config. fail?	“unsolvable” no cause (0.53 s)	timeout † no certificate	minimal unsat core: 1 of 94 constraints (high LiDAR cost) (0.08 s)
Relax: minimal restoration?	$+$ 19,860 USD after 22 full runs with bisection replan (19.6 s); parameter must be designated	impractical: every $\mathrm{UNSAT}$ probe times out	$+$ 19,860 USD exact, 1 call (11.4 s); all resources relaxed jointly, no per- parameter loop
Recompose: minimal change?	churn 4 (1.22 s) no stability term in the encoding	churn 4 (8.97 s) no stability term in the encoding	churn 0 (5.35 s) components held by Equation 22 ‡

TABLE IV: Lifelong queries on the deployed search-and-rescue model ( Figure 5 -I). Top rows: a budget cut renders the deployment infeasible. Bottom row: a cheaper interchangeable camera enters the catalog while the incumbent remains deployed. Baselines are given their best-faith workaround where one exists. Churn = components added

+

removed relative to the deployed system. PDDLStream is omitted: it has no encoding of the full-scale search-and-rescue model ( Table II , n/e ), so no workaround column can be measured for it.

	PDDL	PS ${}_{\text{s}}$	PS ${}_{\text{o}}$	SMTP	Ours
Modeler must hand-encode:
Enumerated/discretized values	✓	✓	$\times$	$\times$	$\times$
Per-action costs tuned to objective	✓	✓	✓	$\times$ ‡	$\times$
Solution structure in model	$\times$	$\times$	✓	$\times$	$\times$
Supported variable types:
Activations $b$	✓	✓	✓	✓	✓

TABLE V: What each planner supports and what the modeler must supply to reach the optimum on the coupled-parameter benchmarks, grouped as modeling burden, expressible free-variable classes, and outcome. ✓ = yes/required,

\times

= no/not needed; enum. / disc. = expressible only via pre-enumeration or discretization;

\varnothing

= none.

Robotics

Robotics studies the design and control of machines that perceive, move, and act in physical environments.

See all 751 papers

Deployment Is Not Destiny: Robot Recomposition in the Field with Unseen Software, Hardware, and Compute Payloads
Aug 11, 2026Steven Swanbeck, Jonathan Salfity, Jeffery Gunawan +3Modular Robotics Robotics
Texas Robotics and the Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, TX 78712, USA
Context-Aware System Synthesis, Task Assignment, and Routing
Jun 14, 2017Jason Ziglar, Ryan Williams, Alfred WicksCombinatorial Optimization Robotics
Evolving Skill Modules under a Fixed Planner: Versioning, Rollback, and Runtime Governance for Long-Lived Robot Systems
Apr 9, 2026Xue Qin, Simin Luan, Cong Yang +1Robotic Policy Evaluation Robot Safety
School of Software, Harbin Institute of Technology, Harbin, China · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China · School of Future Science and Engineering, Soochow University, Suzhou, China

Aug 11, 2026J/K move · Enter open · S save

Deployment Is Not Destiny: Robot Recomposition in the Field with Unseen Software, Hardware, and Compute Payloads

Steven Swanbeck, Jonathan Salfity, Jeffery Gunawan, Corrie Van Sice+2

Texas Robotics and the Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, TX 78712, USA

Modular Robotics Robotics Search-and-Rescue Robotics

Open paper AskPDF arXiv

The tight coupling of subsystems in most robots, though a natural consequence of their complexity, leads to monolithic designs that are time-consuming and difficult to adapt after initial deployment. To address this challenge, we present a framework and supporting abstractions for recomposition during runtime that enable robots to quickly integrate previously unseen modular software, hardware, and compute payloads. Our approach allows non-expert users to quickly add new capabilities in the field through a true plug-and-play process. Crucially, new resources are not only immediately available to a host robot but are also shared with distributed peers, enabling compute-constrained systems to access powerful new remote capabilities. Our framework reduces reconfiguration time to a matter of minutes with no developer intervention, in stark contrast to the hours of expert effort often required for traditional manual integration. We demonstrate our method in two disaster response scenarios, including radioactive source localization at an operational nuclear reactor facility and a thermal-guided search for people in dark, difficult-to-reach spaces. These demonstrations show how in-field recomposition provides timely, flexible, and accessible adaptation to dynamic requirements, representing a critical step toward creating robots that can quickly evolve alongside the tasks, technologies, and environments they support.

Abstract

Robotics

Explore similar work

Deployment Is Not Destiny: Robot Recomposition in the Field with Unseen Software, Hardware, and Compute Payloads

Context-Aware System Synthesis, Task Assignment, and Routing

Evolving Skill Modules under a Fixed Planner: Versioning, Rollback, and Runtime Governance for Long-Lived Robot Systems