EBuddy: a workflow orchestrator for industrial human-machine collaboration
Organizations: Laboratory of Automation, Robotics and Machines (ARM), University of Applied Sciences of Southern Switzerland, Lugano-Viganello, Switzerland
Abstract
This paper presents EBuddy, a voice-guided workflow orchestrator for natural human-machine collaboration in industrial environments. EBuddy targets a recurrent bottleneck in tool-intensive workflows: expert know-how is effective but difficult to scale, and execution quality degrades when procedures are reconstructed ad hoc across operators and sessions. EBuddy operationalizes expert practice as a finite state machine (FSM) driven application that provides an interpretable decision frame at runtime (current state and admissible actions), so that spoken requests are interpreted within state-grounded constraints, while the system executes and monitors the corresponding tool interactions. Through modular workflow artifacts, EBuddy coordinates heterogeneous resources, including GUI-driven software and a collaborative robot, leveraging fully voice-based interaction through automatic speech recognition and intent understanding. An industrial pilot on impeller blade inspection and repair preparation for directed energy deposition (DED), realized by human-robot collaboration, shows substantial reductions in end-to-end process duration across onboarding, 3D scanning and processing, and repair program generation, while preserving repeatability and low operator burden.
Figures & tables
| Workflow | Typical operator decisions | EBuddy orchestration (examples) | Shared recovery strategies |
| Components | Choose guidance granularity; decide when to request help; confirm readiness to start | State-linked guidance via Helper App; structured repeat / expand / skip actions | Re-ground & resume : replay last relevant guidance snippet/state,then continue. |
| Preview | Check prerequisites (scanner/software ready); choose autopilot vs step-by-step | Launch/configure 3D Studio App; readiness checks; enforce required confirmations | Block–fix–retry : stop on missing prerequisites, troubleshoot, retry transition. |
| Full Scan | Select scan strategy/parameters; judge completeness; decide proceed vs refine | Guides acquisition order; constrain valid actions; coordinates cobot viewpoint routine | Quality loop : evaluate quality and repeat a view or branch to refinement (Scan slicing) until criteria are met. |
| 3D Processing | Choose basic vs advanced path; accept vs rework based on mesh quality | Drives GUI sequence (merge/registration/fusion/export); keeps active step and valid commands visible | Local rollback : re-run the minimal failed sub-step (e.g., re-register, re-fuse); escalate upstream to re-scan if needed. |
| Scan Slicing | Select region of interest (ROI) to refine; decide sufficiency; iterate vs fallback | Bounded refinement sub-workflow; constrained actions | Iterative micro-loops : repeat ROI scans; fallback to Full Scan if unresolved (e.g., gaps cannot be closed). |
| Part Program Generator | Select parameters (e.g., region, layer thickness); validate output; regenerate vs proceed | Orchestrates external GUI pipeline (alignment/closure/deviation/clustering/export) | Rollback and rerun : adjust parameters and regenerate; route upstream on data issues. |
| Workflow Phase | Manual Op. | EBuddy Op. | ||||
| Users | Expert | Users | Expert | Users | Expert | |
| 1. EBuddy configuration phase | 0 | 0 | 0 | 770 | -% | -% |
| 2. Training/onboarding phase | 289 | 289 | 154 | 30 | -47% | -90% |
| 3. Scan and 3D processing phase | 518 | 518 | 142 | 0 | -73% | -100% |
| 4. Repair program generation phase | 60 | 60 | 24 | 0 | -70% | -100% |
| Total time (min) | 868 | 868 | 320 | 800 | -63% | -8% |