Large language models hold promise for delivering safe, scientifically grounded, and personalized exercise guidance at scale, yet the sports-and-health domain still lacks curated training corpora and rigorous evaluation benchmarks. Keepace.ai is a domain-specialized LLM built on Qwen3.5-27B through post-training, currently optimized for Chinese-language exercise guidance across three tasks: workout plan generation, sports-and-health knowledge QA, and personal sports data analysis. In internal rubric-based evaluations designed with domain experts, Keepace.ai outperforms its base model and larger general-purpose baselines, with the most pronounced improvements in exercise safety, boundary compliance, structured output completeness, and scientific accuracy. This report summarizes the current design, data pipeline, safety-first evaluation framework, and evaluation results of Keepace.ai.
Figures & tables
Figure 1: Keepace.ai task overview. Keepace.ai is organized around an exercise-assistance loop: planning safe structured workouts, answering sports-science questions, and interpreting workout logs or wearable metrics for future adjustment.
Figure 2: Keepace.ai evaluation summary. Rubric-based evaluation scores for Keepace.ai and seven general-purpose baselines across workout plan generation, sports-and-health knowledge QA, and personal sports data analysis.
Task
Input
Output
Scope and quality focus
Workout plan generation
User profile, training goal, and practical constraints, including fitness level, training history, reported limitations, equipment, space, and time budget.
Structured session with warm-up, main training, cooldown, exercise parameters, rest intervals, and accompanying session metadata.
Safety-first personalization; non-clinical exercise guidance; completeness and fit to user context.
Sports-and-health knowledge QA
User question about exercise science, nutrition, recovery, or related topics, optionally with user profile context.
Educational answer with accurate terminology, evidence-grounded explanation, and explanation depth adapted to the user.
Coaching-oriented education; no individualized prescription or medical advice; scientific accuracy and usefulness.
Personal sports data analysis
Structured exercise log and user query about the session or training pattern.
Evidence-grounded analysis that cites input metrics, flags anomalies, computes supported derived indicators, and recommends recovery or training adjustments.
Contextual interpretation of noisy wearable and mobile data; no diagnosis; accurate data use and actionable feedback.
Table 1: Task definitions and scope constraints for Keepace.
Risk category
Example
Mitigation approach
Fitness pseudoscience
”Sweating more means burning more fat”
Expert-knowledge filtering
Medical overreach
Generating clinical diagnoses or rehabilitation prescriptions
Domain-boundary filtering
Unsafe exercise programming
Excessive plyometrics for novice users
Rule-based safety constraints
Privacy exposure
Personally identifiable information in exercise logs
Anonymization
Sensor anomaly
Physiologically implausible elevation or pace readings
Anomaly flagging
Table 2: Data risks and mitigation strategies in training-data construction.
Figure 3: Training data creation pipeline. Platform records and expert-reviewed materials are converted into training examples through semantic transformation, safety- and science-constraint injection, and rubric-based quality filtering. DPO preference pairs are constructed on top of the same supervised data infrastructure.
SFT stage
DPO stage
Initialization
Qwen3.5-27B base weights
SFT checkpoint
Training data
∼ 200K supervised examples
∼ 4K preference pairs
Objective
LSFT
LDPO+αLSFT
Learning rate
1×10−6
1×10−6
Batch size
64
32
α
—
0.7
Table 3: Training configuration for the SFT and DPO stages.
Dim.
Dimension Name
# Rubrics
Scope
A
Exercise Safety
2
Acute risk recognition (halting on dangerous symptoms); professional scope compliance (not overstepping into clinical diagnosis or rehabilitation)
Verdict: Unsafe request rejected. Pushes back on rapid weight-loss and overload demands; explains risk; proposes safer pacing and recovery-aware alternatives.
Abnormal wearable signals
Treat anomalies as risk indicators; avoid optimizing around abnormal readings; reduce activity or seek evaluation when warranted.
Verdict: Risk-sensitive handling. Flags abnormal readings; avoids medical diagnosis; recommends stopping or reducing activity; advises professional evaluation for persistent or severe abnormalities.
Table 13: Qualitative safety and boundary stress-test probes.
Despite rapid advances in large language models (LLMs), their deployment in the sports and health domain remains high-risk due to the absence of domain-specific evaluation benchmarks. Existing general-purpose benchmarks (e.g., MMLU and HELM) primarily assess broad knowledge and reasoning capabilities, but fail to capture critical domain requirements such as exercise safety, physiological appropriateness, personalization, and actionable coaching quality. This gap is particularly concerning in fitness and health applications, where incorrect or unsafe recommendations can lead to physical harm. To address this limitation, we introduce MoveBench, the first comprehensive benchmarking framework specifically designed for evaluating LLMs in sports and health coaching. The proposed benchmark integrates heterogeneous task formats, including multiple-choice questions (MCQs) and open-ended question answering (OpenQA), grounded in real-world user scenarios and paired with structured rubrics emphasizing safety compliance, scientific validity, and training logic. Through large-scale evaluation of 15 diverse LLMs on 809 questions with 5,012 rubric-level criteria, our results reveal substantial performance variability, particularly in safety awareness, structured program design, and personalization capability, highlighting a critical gap between knowing and doing. Our work establishes a foundational evaluation standard for sports and health LLM applications and provides a reproducible framework for advancing safe, reliable, and context-aware AI-driven coaching systems. Open resources are available at https://github.com/SatoSakula/MoveBench.
Sports have witnessed growing global enthusiasm in recent years, serving as a vital force for physical health, cultural exchange, social connection, and economic growth. The rapid advancement of large models, particularly (multimodal) large language models (M)LLMs, has demonstrated transformative potential to reshape sports understanding, analysis, and interaction across diverse domains. This paper presents a comprehensive survey of large models in sports, including (i) an overview of tasks and applications across different participant groups; (ii) a detailed analysis of sports-related datasets and benchmarks; and (iii) a critical discussion of current challenges and future directions. Our goal is to establish a foundation for advancing research and practical development of large-model-driven sports intelligence. An open-source GitHub repository is maintained at: https://github.com/Road2Redemption/Awesome_Large_Models_In_Sports1.
Yichen Xu, Jianzhe Ma, Chuhan Wang +4
Renmin University of China · Sichuan University · Beijing University of Posts and Telecommunications
Background: Large language models (LLMs) have been explored as tools for generating personalized exercise prescriptions, yet the consistency of outputs under identical conditions remains insufficiently examined. Objective: This study evaluated the intra-model consistency of LLM-generated exercise prescriptions using a repeated generation design. Methods: Six clinical scenarios were used to generate exercise prescriptions using Gemini 2.5 Flash (20 outputs per scenario; total n = 120). Consistency was assessed across three dimensions: (1) semantic consistency using SBERT-based cosine similarity, (2) structural consistency based on the FITT principle using an AI-as-a-judge approach, and (3) safety expression consistency, including inclusion rates and sentence-level quantification. Results: Semantic similarity was high across scenarios (mean cosine similarity: 0.879-0.939), with greater consistency in clinically constrained cases. Frequency showed consistent patterns, whereas variability was observed in quantitative components, particularly exercise intensity. Unclassifiable intensity expressions were observed in 10-25% of resistance training outputs. Safety-related expressions were included in 100% of outputs; however, safety sentence counts varied significantly across scenarios (H=86.18, p less than 0.001), with clinical cases generating more safety expressions than healthy adult cases. Conclusions: LLM-generated exercise prescriptions demonstrated high semantic consistency but showed variability in key quantitative components. Reliability depends substantially on prompt structure, and additional structural constraints and expert validation are needed before clinical deployment.
Kihyuk Lee
Data Convergence Team, Office of Hospital Information, Seoul National University Bundang Hospital, Seongnam, Republic of Korea