Train the Trainers -- An Agentic AI Framework for Peer-Based Mental Health Support in Battlefield Environments
Authors: Atmaram Yarlagadda, Eranga Bandara, Ross Gore, Anita H. Clayton, Preston Samuel, Christopher K. Rhea, Sachin Shetty, Ravi Mukkamala, +3 more
Organizations: McDonald Army Health Center, Newport News, VA, USA · Old Dominion University, Norfolk, VA, USA · Department of Psychiatry and Neurobehavioral Sciences, University of Virginia School of Medicine, Charlottesville, VA, USA · Blanchfield Army Community Hospital, Fort Campbell, KY, USA · Florida International University, FL, USA · Accenture Technology Labs, Arlington, VA, USA · Deloitte & Touche LLP, USA
Modern military operations expose soldiers to sustained psychological stress, leading to acute reactions, post-traumatic stress symptoms, and other mental health issues. Although the U.S. Department of Defense offers evidence-based therapies, access to trained professionals in forward-deployed and contested environments is limited. As a result, soldiers with early-stage distress are often evacuated to rear medical facilities, delaying care, reducing readiness, and increasing long-term risks. This paper proposes a Train-the-Trainers framework in which soldiers who have completed therapy and returned to duty are trained as peer facilitators to provide first-line psychological support in operational settings. To scale and standardize this model under severe resource and connectivity constraints, we introduce an agentic AI-enabled platform that augments these recovered soldiers with specialized AI agents. The recovered soldier acts as a human supervisor, coordinating agents for symptom triage, guided peer-support interventions, operational constraint reasoning, training and simulation, and structured documentation for clinical escalation when needed. The AI agents use consensus-driven decision support in high-stakes environments. The architecture functions in air-gapped and low-connectivity settings, maintaining human oversight and ethical safeguards. A functional prototype was developed with the McDonald U.S. Army Health Center, Newport News, VA, USA. By combining peer-based intervention with consensus-driven agentic AI decision support, the framework seeks to cut response times, prevent symptom escalation, reduce unnecessary evacuations, and improve continuity of care. This work shows how agentic AI can serve as a force multiplier for mental health support in austere environments and identifies pathways for broader evaluation and deployment across defense and humanitarian operations.
Figures & tables
Figure 1: Comparison between direct human–LLM interaction and agentic AI–LLM interaction. In the top panel, a human user directly constructs and submits a prompt to a language model, manually interpreting the response. In the bottom panel, an AI agent autonomously manages the interaction loop by constructing prompts, invoking the language model, interpreting responses, and determining subsequent actions without direct human intervention. This agent-mediated workflow enables iterative reasoning, tool integration, and task automation, forming the basis of agentic AI systems used for structured decision support and multi-step workflows.
Figure 2: Human-supervised agentic AI architecture for the Train-the-Trainers framework. A recovered soldier acting as a peer trainer serves as the central human supervisor, coordinating multiple specialized AI agents that provide advisory decision support for assessment, peer intervention guidance, operational constraint reasoning, escalation and referral, documentation and after-action reporting, and training and simulation. All agent interactions are advisory in nature, with authority and execution remaining strictly under human control.
Figure 3: Overview of the supervised fine-tuning pipeline used to adapt a base LLM to domain-specific tasks. A curated, domain-specific dataset is used to perform supervised fine-tuning of a base LLM, with Low-Rank Adapter (LoRA) modules applied to enable parameter-efficient training. The resulting fine-tuned model retains the general linguistic capabilities of the base model while incorporating domain-specific knowledge, supporting efficient deployment and updates in resource-constrained and air-gapped environments.
Figure 4: Consensus-driven LLM consortium and reasoning workflow used by the agentic AI platform. An LLM agent generates a task-specific prompt, which is forwarded via the agent orchestration layer to multiple fine-tuned LLMs (e.g., Llama-3, Mistral, and Qwen). Each model independently generates a response based on the same prompt and contextual inputs. The resulting outputs are collected and processed by the orchestration layer, which constructs a structured reasoning prompt for a dedicated reasoning-oriented LLM (GPT-OSS). The reasoning LLM evaluates, reconciles, and synthesizes the multiple model outputs to produce a final consolidated response accompanied by an interpretable rationale. This consensus-driven process improves robustness, mitigates single-model bias, and supports explainable, human-supervised decision support.
Figure 5: Structure of the psychiatric assessment fine-tuning dataset. Each sample contains an instruction prompt, a psychiatrist-patient conversation (2.07k–5.07k characters), a DSM-5 diagnosis label, and a condition category. The dataset was constructed from publicly available data and synthetically generated conversations covering five psychiatric conditions.
Table 2: Fine-Tuning Configuration for Psychiatric Assessment Model
Figure 6: Training metrics during fine-tuning: learning rate schedule (left) showing warmup to 1×10−4 followed by linear decay, and training loss (right) decreasing from 1.9 to 0.90 over 180 steps, indicating effective learning of the diagnostic classification task.
Figure 7: Validation metrics during fine-tuning: evaluation loss (left) decreasing from 1.8 to 1.195, closely following the training loss trajectory and indicating good generalization without overfitting, and evaluation runtime (right) stabilizing at approximately 3.91 seconds.
Figure 8: Training progression metrics: epoch count (left) showing linear progression through approximately 3.6 epochs, and gradient norm (right) increasing gradually from 0.28 to 0.71, indicating stable optimization without gradient explosion.
Metric
Value
Training Duration
494.24 seconds (8.24 minutes)
Hardware
NVIDIA A100-SXM4-40GB
GPU Maximum Memory
39.557 GB
Initial Memory Reserved
6.725 GB
Peak Memory Reserved
7.789 GB
Memory for LoRA Training
1.064 GB
Table 3: Training Resource Utilization for Psychiatric Assessment Model
Figure 9: Computational efficiency metrics: total floating-point operations (left) reaching 5.65×1016 FLOPs, and cumulative training loss (right) at 1.203, demonstrating the computational cost of parameter-efficient fine-tuning.
Figure 10: Inference performance metrics during validation: samples per second (left) stabilizing at approximately 12.78, and steps per second (right) at approximately 6.39, indicating consistent evaluation throughput throughout training.
Condition
Correct
Total
Accuracy
Generalized Anxiety Disorder
13
13
100.0%
Bipolar I Disorder
14
14
100.0%
Panic Disorder
8
8
100.0%
Major Depressive Disorder
5
8
62.5%
Post-Traumatic Stress Disorder
4
7
57.1%
Overall
44
50
88.0%
Table 4: Assessment Label Alignment Accuracy by Condition
Figure 11: Representative assessment output of the fine-tuned Llama-3 LLM aligned with structured reference labels (DSM-5 296.21).
Figure 12: Representative assessment output of the fine-tuned Mistral LLM aligned with structured reference labels (DSM-5 300.01).
Figure 13: Representative assessment output of the fine-tuned Qwen2 LLM aligned with structured reference labels (DSM-5 295.90).
Figure 14: Consensus-driven assessment synthesis produced by the reasoning-oriented LLM through multi-model evaluation and reconciliation.
Mental health is a growing global concern, prompting interest in AI-driven solutions to expand access to psychosocial support. Peer support, grounded in lived experience, offers a valuable complement to professional care. However, variability in training, effectiveness, and definitions raises concerns about quality, consistency, and safety. Large Language Models (LLMs) present new opportunities to enhance peer support interactions, particularly in real-time, text-based interactions. We present and evaluate an AI-supported system with an LLM-simulated distressed client (SimClient), context-sensitive LLM-generated suggestions (Suggestions), and real-time emotion visualisations. 2 mixed-methods studies with 12 peer supporters and 6 mental health professionals (i.e., experts) examined the system's effectiveness and implications for practice. Both groups recognised its potential to enhance training and improve interaction quality. However, we found a key tension emerged: while peer supporters engaged meaningfully, experts consistently flagged critical issues in peer supporter responses, such as missed distress cues and premature advice-giving. This misalignment highlights potential limitations in current peer support training, especially in emotionally charged contexts where safety and fidelity to best practices are essential. Our findings underscore the need for standardised, psychologically grounded training, especially as peer support scales globally. They also demonstrate how LLM-supported systems can scaffold this development--if designed with care and guided by expert oversight. This work contributes to emerging conversations on responsible AI integration in mental health and the evolving role of LLMs in augmenting peer-delivered care.
Mental health disorders affect millions worldwide, and healthcare systems are increasingly overwhelmed by the volume of clinical data generated from electronic records, telemedicine platforms, and population-level screening programs. At the same time, the emergence of novel AI-based approaches in healthcare calls for intelligent frameworks capable of processing domain-specific unstructured clinical information while adapting to patient-specific needs. This paper proposes an agentic framework for building robust LLM-based pipelines, where each stage is encapsulated as a LangChain agent governed by explicit policies and proxy-guided evaluation. Stages are incrementally locked once validated, ensuring that later adaptations cannot overwrite configurations without demonstrated improvement. The proposed framework evolves from feature-level exploration, through proxy-based tuning and freeze/rollback mechanisms, to full orchestration by an Orchestrator Agent that coordinates preprocessing, retrieval, selection, diversity, threshold optimization, and decoding. A proof-of-concept in transcript-based depression detection demonstrates that the framework converges to stable configurations, such as cosine similarity, dynamic Top-k, and threshold 0.75, while controlling evaluation costs and avoiding regressions. These results highlight the potential of agentic AI to enable population-level mental health screening over large clinical datasets, addressing critical challenges in trustworthiness, reproducibility, and adaptability required in healthcare environments.
Giuliano Lorenzoni, Paulo Alencar, Donald Cowan
University of Waterloo · Waterloo, Ontario, Canada
Privacy represents one of the most critical yet underaddressed barriers to AI adoption in mental healthcare -- particularly in high-sensitivity operational environments such as military, correctional, and remote healthcare settings, where the risk of patient data exposure can deter help-seeking behavior entirely. Existing AI-enabled psychiatric decision support systems predominantly rely on cloud-based inference pipelines, requiring sensitive patient data to leave the device and traverse external servers, creating unacceptable privacy and security risks in these contexts. In this paper, we propose a zero-egress, on-device AI platform for privacy-preserving psychiatric decision support, deployed as a cross-platform mobile application. The proposed system extends our prior work on fine-tuned LLM consortiums for psychiatric diagnosis standardization by fundamentally re-architecting the inference pipeline for fully local execution -- ensuring that no patient data is transmitted to, processed by, or stored on any external server at any stage. The platform integrates a consortium of three lightweight, fine-tuned, and quantized open-source LLMs -- Gemma, Phi-3.5-mini, and Qwen2 -- selected for their compact architectures and proven efficiency on resource-constrained mobile hardware. An on-device orchestration layer coordinates ensemble inference and consensus-based diagnostic reasoning, producing DSM-5-aligned assessments for conditions. The platform is designed to assist clinicians with differential diagnosis and evidence-linked symptom mapping, as well as to support patient-facing self-screening with appropriate clinical safeguards. Initial evaluation demonstrates that the proposed zero-egress deployment achieves diagnostic accuracy comparable to its server-side predecessor while sustaining real-time inference latency on commodity mobile hardware.
Eranga Bandara, Asanga Gunaratna, Ross Gore +9
Old Dominion University, Norfolk, VA, USA · AI Motion Labs, Melbourne, Australia · Department of Psychiatry and Neurobehavioral Sciences, University of Virginia School of Medicine, Charlottesville, VA, USA +3