cs.ROAug 21, 2026
SaveModel-Based Reinforcement Learning for Heterogeneous Multi-Robot Task Assignment Under Distribution Shifts
Organizations: John A. Paulson School Of Engineering And Applied Sciences, Harvard University, 150 Western Avenue, Boston, 02134, MA, USA · Harvard Medical School, 77 Avenue Louis Pasteur, Boston, 02215, MA, USA · Beth Israel Deaconess Medical Center, 330 Brookline Avenue, Boston, 02215, MA, USA · Stanford University School of Medicine, 300 Pasteur Drive, Stanford, 94305, CA, USA
Abstract
Heterogeneous multi-robot service systems must assign requests to compatible robots, construct feasible schedules, and adapt as new tasks arrive online. Historical data can help anticipate future demand, but relying too heavily on inaccurate predictions can degrade performance under distribution shifts. We develop a prediction-aware adaptive rollout framework for heterogeneous multi-robot task assignment with scheduled and real-time requests. The problem is formulated as a finite-horizon stochastic dynamic program incorporating robot-task compatibility, ordered service requirements, routing constraints, service windows, and end-of-horizon return requirements. The proposed policy evaluates current assignments using sampled future request scenarios while restricting immediate commitments to requests already observed. To enable online use, the framework combines pruned candidate controls, wait actions, and an interaction-aware base policy for efficient future-cost estimation. Robustness to forecast error is provided by adaptively reweighting predicted requests based on recent prediction mismatch and selectively re-optimizing assigned but unstarted requests. We also introduce a historical-data-driven procedure for selecting the heterogeneous fleet composition before deployment. In a case study using real nursing-task requests from hospital inpatient floors, the proposed approach achieves near-complete service and reduces serviced-request wait times relative to reactive, token-passing, prediction-positioning, and myopic greedy baselines, with the largest improvements in tail-delay metrics.
Figures & tables
Figure 1: Overview of the proposed prediction-aware adaptive rollout framework for heterogeneous multi-robot task assignment. Local temporal point process models generate sampled future request scenarios, which are filtered to remove requests that are already observed or scheduled. Recent forecast errors are used to dynamically reweight predicted requests, producing context-dependent prediction-confidence scores and weighted future-cost estimates. Newly observed urgent requests can trigger re-optimization by returning eligible assigned but unstarted requests to the pending set. The rollout-based assignment module evaluates candidate assignments for observed and re-optimized requests using the weighted future-cost estimates and selects schedules for the heterogeneous robot team.
Figure 2: Schematic of the neural temporal point process used for future-request generation. Past request events are represented as a sequence of timed marks and encoded into a history-dependent representation. This representation parameterizes conditional distributions over future event times and request marks. Samples from these distributions are then converted into graph-grounded request tuples for rollout simulation.
Figure 3: Reservation-table check used by the interaction-aware base policy. A candidate service interval is computed from the robot’s estimated arrival time and the request execution duration. If the interval overlaps an existing reservation at the same service node, it is shifted to the earliest available interval that satisfies the request’s timing constraints. The resulting interval determines the estimated completion time used to compare compatible robot assignments.
Figure 4: Environment-generation pipeline used in the case study. Starting from an architectural floor plan, we construct a simplified three-dimensional representation of the hospital floor and then extract a two-dimensional navigation map for robot simulation and planning. The resulting map defines the traversable workspace used to build the graph representation, compute travel-time heuristics, and assign service locations for scheduled, observed, and predicted requests.
Figure 5: Example traversal graph used in the case-study simulations. Waypoint nodes and feasible local motion edges are overlaid on the hospital navigation map, with zoomed regions showing graph connectivity in representative corridor and intersection areas. The resulting graph supports shortest-path travel-time estimation and graph-constrained routing for task assignment and scheduling.
Figure 6: Example weekly delivery-request arrival rates used to select evaluation periods. The empirical mean is shown by the dotted blue line, and the dashed lines mark the Laney-adjusted upper and lower control limits. Weeks exceeding these limits are identified as abnormal high- or low-demand periods and are used to test the robustness of the proposed planner under demand shifts.
| Floor | High-demand weeks | Medium-demand weeks | Low-demand weeks |
| 2 | 2025-W06, 2025-W08 | 2024-W44, 2025-W14 | 2024-W27, 2024-W28 |
| 3 | 2025-W05, 2025-W10 | 2024-W44, 2025-W14 | 2024-W46, 2025-W02 |
| 7 | 2024-W39, 2024-W40 | 2024-W44, 2025-W14 | 2024-W46, 2025-W26 |
| 9 | 2024-W43, 2025-W06 | 2024-W44, 2025-W14 | 2025-W19, 2025-W21 |
| Total requests | 36,948 | 30,824 | 25,300 |
Table 1: Summary of the hospital-floor test set. The table reports the selected high-, medium-, and low-demand weeks for each floor. The final row gives the total number of requests in each demand level aggregated across all task types, selected weeks, and floors.
Figure 7: Comparison of temporal point process (TPP) models for monitoring-request prediction using the OTD metric. Each stacked bar reports mean OTD for one model configuration, decomposed into timing, request-type, deletion, and insertion components. Error bars indicate variability across evaluation samples. Lower values indicate better agreement between predicted and observed request sequences, while smaller deletion and insertion components correspond to fewer missed or spurious predicted events. TPP model names and references are given at the beginning of Section 5.3 . In the model labels, denotes the use of enhanced marks, denotes conditioning on previous-day information, and denotes the use of both enhanced mark and previous day conditioning.
Figure 8: Comparison of temporal point process (TPP) models for medication-delivery prediction using the OTD metric. Each stacked bar reports mean OTD for one model configuration, decomposed into timing, request-type, deletion, and insertion components. Error bars indicate variability across evaluation samples. Lower values indicate better agreement between predicted and observed request sequences, while smaller deletion and insertion components correspond to fewer missed or spurious predicted events.TPP model names and references are given at the beginning of Section 5.3 . In the model labels, denotes the use of enhanced marks, denotes conditioning on previous-day information, and denotes the use of both enhanced mark and previous day conditioning.
Figure 9: Day-wise robot requirements obtained during team-size initialization. For each historical day, the sizing routine increases the number of monitoring or delivery robots until all requests are feasibly served. The histograms summarize the resulting feasible robot counts across historical days and are used to initialize the heterogeneous team composition at the desired empirical percentile.
| Phase | Monitoring robots | Delivery robots |
| Histogram initialization | 6 | 2 |
| Joint historical verification | 6 | 3 |
Table 2: Selected heterogeneous team composition.
| Policy | High demand | Medium demand | Low demand | |||
| Serviced | Rejected | Serviced | Rejected | Serviced | Rejected | |
| Fleet Manager | 36,948 | 0 | 30,824 | 0 | 25,300 | 0 |
| Token Passing | 36,920 | 28 | 30,824 | 0 | 25,300 | 0 |
| Token Passing with Task Swaps | 36,920 | 28 | 30,824 | 0 | 25,300 | 0 |
| Token Passing with Deadlines | 36,948 | 0 | 30,824 | 0 | 25,300 | 0 |
| Deadline-Aware Token Passing with Task Swaps | 36,948 | 0 | 30,824 | 0 | 25,300 | 0 |
Table 3: Service-level results by demand regime. Each entry reports serviced and rejected requests for the corresponding policy and load level.
Figure 10: Comparison of request wait times across assignment policies during the highest-demand evaluation weeks. The left panel reports the distribution of daily mean wait times for serviced requests, and the right panel reports the distribution of daily -th percentile wait times for serviced requests. Each box summarizes performance across evaluation instances, with lower values indicating shorter delays. Abbreviated policy labels correspond to the assignment policies defined in Section 5.5.1 .
Figure 11: Comparison of request wait times across assignment policies during medium-demand evaluation weeks. The left panel reports the distribution of daily mean wait times for serviced requests, and the right panel reports the distribution of daily -th percentile wait times for serviced requests. Each box summarizes performance across evaluation instances, with lower values indicating shorter delays. Abbreviated policy labels correspond to the assignment policies defined in Section 5.5.1 .
Figure 12: Comparison of request wait times across assignment policies during the lowest-demand evaluation weeks. The left panel reports the distribution of daily mean wait times for serviced requests, and the right panel reports the distribution of daily (95)-th percentile wait times for serviced requests. Each box summarizes performance across evaluation instances, with lower values indicating shorter delays. Abbreviated policy labels correspond to the assignment policies defined in Section 5.5.1 .
Figure 13: Planning time per request-triggered decision across the evaluated assignment policies. Each box plot summarizes the computation time required to process a decision during the evaluation runs. Lower values indicate faster online decision making. The proposed adaptive rollout policy requires more computation than myopic and token-passing baselines because it evaluates candidate assignments through sampled future scenarios, but its median planning time remains within the operational lead time of the case-study setting. Abbreviated policy labels correspond to the assignment policies defined in Section 5.5.1 .
| Policy | High demand | Medium demand | Low demand | |||
| Mean | Std. | Mean | Std. | Mean | Std. | |
| Rollout without adaptation | 49.25 | 90.02 | 34.23 | 78.21 | 25.65 | 68.72 |
| Rollout with re-optimization only | 46.78 | 86.88 | 32.18 | 74.08 | 24.36 | 65.97 |
| Rollout with prediction reweighting only | 49.39 | 90.49 | 33.70 | 77.25 | 25.17 | 67.69 |
| Proposed adaptive rollout | 46.01 | 85.41 | 31.88 | 73.54 | 24.28 | 65.91 |
Table 4: Ablation study of the adaptive rollout components. All variants use the same rollout framework and include future scheduled requests in the future-cost estimate. The table reports the mean and standard deviation of request wait times, in seconds, for each demand level.
Figure 14: Planning time per request-triggered decision across the ablation policies. Each box plot shows the distribution of computation time required to process a decision during the evaluation runs. Lower values indicate faster online decision making.