cs.CEApr 8, 2026

XR-CareerAssist: An Immersive Platform for Personalised Career Guidance Leveraging Extended Reality and Multimodal AI

Authors: N. D. Tantaroudas, A. J. McCracken, I. Karachalios, E. Papatheou, V. Pastrikakis

Organizations: Institute of Communications and Computer Systems (ICCS), Iroon Polytechneiou 9, 15773, Zografou, Athens, Greece · DASKALOS-APPS, 183 Rue de l’Abbé Griffon, 01960 Péronnas, France · National Technical University of Athens, Leof. Alimou, Katechaki, Zografou, 15772, Athens, Greece · Exeter Small-Scale Robotics Laboratory, Engineering Department, University of Exeter, Exeter, EX4 4QF, UK · CVCOSMOS Ltd, 1 Barnfield Crescent, Exeter, England, EX1 1QT

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Conventional career guidance platforms rely on static, text-driven interfaces that struggle to engage users or deliver personalised, evidence-based insights. Although Computer-Assisted Career Guidance Systems have evolved since the 1960s, they remain limited in interactivity and pay little attention to the narrative dimensions of career development. We introduce XR-CareerAssist, a platform that unifies Extended Reality (XR) with several Artificial Intelligence (AI) modules to deliver immersive, multilingual career guidance. The system integrates Automatic Speech Recognition for voice-driven interaction, Neural Machine Translation across English, Greek, French, and Italian, a Langchain-based conversational Training Assistant for personalised dialogue, a BLIP-based Vision-Language model for career visualisations, and AWS Polly Text-to-Speech delivered through an interactive 3D avatar. Career trajectories are rendered as dynamic Sankey diagrams derived from a repository of more than 100,000 anonymised professional profiles. The application was built in Unity for Meta Quest 3, with backend services hosted on AWS. A pilot evaluation at the University of Exeter with 23 participants returned 95.6% speech recognition accuracy, 78.3% overall user satisfaction, and 91.3% favourable ratings for system responsiveness, with feedback informing subsequent improvements to motion comfort, audio clarity, and text legibility. XR-CareerAssist demonstrates how the fusion of XR and AI can produce more engaging, accessible, and effective career development tools, with the integration of five AI modules within a single immersive environment yielding a multimodal interaction experience that distinguishes it from existing career guidance platforms.

Figures & tables

Figure 1: XR-CareerAssist architecture diagram illustrating the layered system design: User Interface (UI) Layer with VR/AR devices; Application Layer with the Career Guidance Application; Integration Layer incorporating the Data Integration Framework (APIs) and VOXReality Model Extensions; AI Models and Services Layer hosting ASR, NMT, VL Model, and Conversational Agent (CA); and Data Layer comprising the CVs Database and User Data Repository linked to VOXReality Repositories.
Figure 2: Envisioned scenario of XR-CareerAssist showing the complete user journey and AI integration flow, including voice input processing through ASR, multilingual translation via NMT, career data retrieval, VL model interpretation, dialogue generation, and TTS audio output delivered through the 3D Avatar.
StepUser InputModel/ProcessOutput
Questionnaire completionJob role, experience, skills, target role, industry preferencesCVs database queried via APICareer profiles matched, Sankey diagrams generated
Voice-based navigationVoice queries and navigation commandsASR Model: spoken input converted to textTranscribed text for NMT processing
Multilingual interactionText output from ASRNMT Model: translates to user’s chosen languageMultilingual accessibility enabled
Career map explorationUser-requested insights for Sankey diagramVL Model: interprets Sankey diagramsCareer progression explained in text
Career questioningUser queries about career pathsDialogue System: generates personalised adviceTailored guidance delivered
Listening to responsesContinued interactionTTS: text converted to speech via 3D AvatarAudio responses enhance engagement
Table 1: User journey and AI model integration within XR-CareerAssist.
Figure 3: XR ASR pipeline: English voice input to text transcription and NMT translation to Italian. The display includes the audio input waveform, ASR transcription output, and NMT translation with timing metrics, confidence scores, and language detection indicators.
Figure 4: VL model extended through fine-tuning to respond to Sankey diagram queries. Two examples demonstrate the fine-tuned BLIP model answering questions about career transition patterns and relative frequencies of different career moves.
Figure 5: Dialogue system training manual for XR-CareerAssist, providing structured guidance for interacting with the Training Assistant including step-by-step instructions, best practices for career-related questions, and troubleshooting information.
Figure 6: Local testing of PIPER TTS with XR-CareerAssist backend, demonstrating the initial evaluation before transitioning to AWS Polly for production deployment.
Figure 7: Sample Sankey diagram generated from CVCOSMOS data for a user profile input with 25 years of experience. Flowing paths indicate career transitions from current roles through intermediate stages to potential future roles over a 10-year projection, with flow widths representing relative transition frequencies.
Figure 8: Career map concept note for location shift target showing detailed metrics for career progression tracking.
Figure 9: Job role evolution career map for specified user input showing 10-year progression. The left column displays starting roles (Employee, Manager, Director), middle columns show intermediate career stages at 2 and 5 years, and the right column shows potential roles at 10 years with flow widths indicating transition frequencies.
Figure 10: Industry shift evolution career map for specified user input over 10 years, showing how professionals typically transition between industries with flow widths representing relative transition frequencies across sectors including Computer Manufacturing, Financial Services, IT Services, and Software Products.
Figure 11: Questionnaire flow within the XR environment showing two sequential input screens: role selection (left) and work experience specification (right), rendered in a clean, dark-themed design optimised for VR readability.
Figure 12: XR-CareerAssist user interface within the XR environment as experienced on the Meta Quest 3 headset, showing the 3D avatar (Training Assistant) alongside the Sankey diagram display panel in the virtual workspace.
Figure 13: Load testing results for 10,000 concurrent users. The top graph shows sustained throughput of 900 to 1,000 requests per second, the middle graph displays response time distribution with median latency between 800 and 900 milliseconds and 95th percentile under 2 seconds, and the bottom section shows zero failure rate throughout the test.
Figure 14: Pilot demonstration setup at the University of Exeter showing the equipment arrangement and testing environment.
MetricResultSample
Voice Recognition Accuracy95.6% rated accurate or very accurate22/23 participants
Overall User Satisfaction78.3% satisfied or very satisfied18/23 participants
System Responsiveness91.3% rated good or excellent21/23 participants
Interface Intuitiveness78.2% found intuitive or very intuitive18/23 participants
Platform Stability73.9% experienced no downtime17/23 participants
Career Guidance Value69.6% found valuable16/23 participants
Table 2: Pilot study quantitative results summary.

Explore similar work

CardsList

  1. AI-Driven Modular Services for Accessible Multilingual Education in Immersive Extended Reality Settings: Integrating Speech Processing, Translation, and Sign Language Rendering

    Apr 7, 2026N. D. Tantaroudas, A. J. McCracken, I. Karachalios +1Speech Translation

    Institute of Communications and Computer Systems (ICCS), Iroon Polytechneiou 9, 15773 Zografou, Athens, Greece · DASKALOS-APPS, 183 Rue de l’Abbé Griffon, 01960 Péronnas, France · National Technical University of Athens, Leof. Alimou, Katechaki, Zografou 15772, Athens, Greece +1

  2. INTERACT: An AI-Driven Extended Reality Framework for Accesible Communication Featuring Real-Time Sign Language Interpretation and Emotion Recognition

    Apr 7, 2026Nikolaos D. Tantaroudas, Andrew J. McCracken, Ilias Karachalios +1Sign Language Translation

    Institute of Communications and Computer Systems (ICCS), Iroon Polytechneiou 9, 15773 Zografou, Athens, Greece · DASKALOS-APPS, 183 Rue de l’Abbé Griffon, 01960 Péronnas, France · National Technical University of Athens, Leof. Alimou, Katechaki, Zografou, 15772 Athens, Greece +1

  3. An Integrated System for Real-Time Student Assessment and Career Guidance Using Neural Networks in Computing Disciplines

    Jun 14, 2026Sakir Hossain Faruque, Md. Jubair Hossain, Sharun Akter KhushbuEducational AssessmentAI in Education

    Department of Computer Science and Engineering, Daffodil International University, Dhaka-1216, Bangladesh · Department of Electrical and Electronic Engineering, Barishal Engineering College, Barishal, Bangladesh