SKINOPATHY AI: Smartphone-Based Ophthalmic Screening and Longitudinal Tracking Using Lightweight Computer Vision
Authors: S. Kalaycioglu, C. Hong, M. Zhu, H. Xie
Organizations: Toronto Metropolitan University · York University · AIMechatroniX Inc. · DrRobot Inc · Skinopathy Inc. · Scarborough Health Network, Centenary Hospital, Scarborough, Ontario, Canada
Early ophthalmic screening in low-resource and remote settings is constrained by access to specialized equipment and trained practitioners. We present SKINOPATHY AI, a smartphone-first web application that delivers five complementary, explainable screening modules entirely through commodity mobile hardware: (1) redness quantification via LAB a* color-space normalization; (2) blink-rate estimation using MediaPipe FaceMesh Eye Aspect Ratio (EAR) with adaptive thresholding; (3) pupil light reflex characterization through Pupil-to-Iris Ratio (PIR) time-series analysis; (4) scleral color indexing foricterus and anemia proxies via LAB/HSV statistics; and (5) iris-landmark-calibrated lesion encroachment measurement with millimeter-scale estimates and longitudinal trend tracking. The system is implemented as a React/FastAPI stack with OpenCV and MediaPipe, MongoDB-backed session persistence, and PDF report generation. All algorithms are fully deterministic, privacy-preserving, and designed for non-diagnostic consumer triage. We detail system architecture, algorithm design, evaluation methodology, clinical context, and ethical boundaries of the platform. SKINOPATHY AI demonstrates that multi-signal ophthalmic screening is feasible on unmodified smartphones without cloud-based AI inference, providing a foundation for future clinically validated mobile ophthalmoscopy tools.
Figures & tables
Tier
Technology
Role
Frontend
React, Chart.js, MediaRecorder API, Axios
UI, consent, media capture, visualization
Backend
FastAPI, OpenCV, MediaPipe FaceMesh, ReportLab
Image/video processing, metrics, PDF export
Database
MongoDB (Motor async client), UUID keys
Session storage, result persistence, trend data
DevOps
Supervisor, Kubernetes ingress, .env variables
Process management, routing, configuration
Table 1: System Technology Stack by Tier
Figure 1: Schematic overview of SKINOPATHY AI three-tier architecture.
Binary mask; M(x,y)=1 if pixel belongs to region of interest
EM[⋅]
—
Spatial expectation over mask: (1/NM)∑(x,y):M(x,y)=1f(x,y)
p i
R2
Unnormalized pixel-space landmark coordinate (x i , y i )
Table 2: Unified Notation for All Module Algorithms
Figure 2: SKINOPATHY AI — Red-Eye Meter module. The user captures or uploads an eye photograph; the system computes a redness score (0–10) via LAB a* analysis and displays triage guidance. Illustrated output: 3.66/10 (mild, monitor). Camera device selection and live preview are provided to guide optimal capture. Screening only; not diagnostic.
Module
Primary Metric
Pilot Value
Target (Full Study)
Status
Redness
Spearman r s
0.86
> 0.80 vs. Efron grade
Pilot complete
Blink Rate
MAE (blinks/min)
2.1 ± 0.9
< 2.5 blinks/min MAE
In progress
Pupil Reflex
Latency error (ms)
42 ± 18 ms
< 50 ms vs. pupillometer
Pilot complete
Color Screen
Icterus AUC
0.79
> 0.80 vs. clinical grade
In progress
Lesion Track.
MAE (mm)
0.31 ± 0.12 mm
< 0.5 mm vs. slit-lamp
Ongoing
Table 3: Pilot Performance Summary by Module
Figure 3: SKINOPATHY AI — Blink Coach module. The user records a 10-second selfie video; MediaPipe FaceMesh and adaptive EAR thresholding detect blink events. Illustrated output: 17 blinks detected, rate = 102.05 blinks/min (elevated; re-recording with stable head position recommended). The module provides blink count, rate, and normative guidance (typical adult range: 15–20 blinks/min).
Figure 4: SKINOPATHY AI — Pupil Reflex Test module. The user records a 10-second video while briefly exposing the eye to a bright light source at approximately t = 3 s. MediaPipe FaceMesh with iris refinement tracks the Pupil-to-Iris Ratio (PIR) time-series. Key metrics extracted include PLR amplitude ( Δrel ), latency (L ms ), mean constriction velocity (v mean ), and a recording quality score Q. The module targets neurological screening signals; absent or markedly delayed constriction warrants clinical follow-up. This figure shows the baseline without the application of a bright light source.
Figure 5: SKINOPATHY AI — Pupil Reflex Test module (additional view). The user records a 10-second video while briefly exposing the eye to a bright light source at approximately t = 3 s. MediaPipe FaceMesh with iris refinement tracks the Pupil-to-Iris Ratio (PIR) time-series.
Figure 6: SKINOPATHY AI — Color Health Screen module. The user uploads or captures an eye photograph; the system computes the Yellow Index (icterus proxy from LAB b*) and Pallor Index (anemia proxy from LAB L* and a*) over a luminance-gated scleral mask. Gray-world white balance correction is applied before color analysis. Outputs include both indices on [0,1] with uncertainty bands and conservative triage messaging. Screening only; not diagnostic.
Figure 7: SKINOPATHY AI — Lesion Detection & Tracking module. MediaPipe FaceMesh iris landmarks calibrate the pixel-to-millimeter conversion factor using the population mean HVID (11.8 mm). A multi-space color filter (LAB + HSV) segments candidate lesion pixels within the near-limbus annular analysis band (0.65–0.98 ρiris ). Illustrated output: iris diameter assumed 11.8 mm; distance from limbus = 0 mm (no lesion detected). The OLS trend estimator flags progressive lesions when v^growth > 0.005 mm/day. Heuristic MVP; trained segmentation model planned for future release.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Endpoint
Description
POST
/api/sessions
Create new session; accepts consent and intake JSON; returns session_id (UUID)
GET
/api/sessions
List all sessions for the current user context; returns array of session summaries
GET
/api/sessions/{id}
Retrieve full session document including all module results and intake data
GET
/api/sessions/{id}/report.pdf
Generate and download PDF report for session; includes all intake and results
Anemia affects over one billion people globally and remains severely under-diagnosed in low-resource regions where laboratory blood tests are inaccessible. This paper presents AnemiaVision, an end-to-end web-based system for non-invasive anemia screening from smartphone photographs of the palpebral conjunctiva and fingernail beds. The proposed pipeline fine-tunes a pre-trained EfficientNet-B3 backbone with a redesigned three-layer classifier head incorporating BatchNorm, GELU activations, and high-rate Dropout (0.45/0.35). Training employs four orthogonal accuracy-boosting techniques: TrivialAugmentWide for policy-free image augmentation, RandomErasing for spatial regularisation, Mixup (alpha=0.2) for inter-class smoothing, and cosine-annealing scheduling with linear warmup. Early stopping is governed by peak validation accuracy rather than validation loss to prevent premature termination on high-variance epochs. The deployed Flask application integrates persistent patient-history management backed by PostgreSQL on Render, with an automated database-migration entrypoint ensuring zero data loss across redeploys. Ablation experiments demonstrate that accuracy-first early stopping contributes +1.6% and Mixup contributes +2.8% to final validation accuracy. Overall, the proposed system achieves a validation accuracy of 96.2% and AUC-ROC of 0.98, compared with 44.9% validation accuracy and AUC-ROC of 0.58 from the three-epoch CPU-only baseline. Sensitivity for the anemic class reaches 0.96, making the system suitable as a first-line screening tool for community health workers in rural settings. The system is publicly accessible and source code is openly available.
Rahul Patel
Department of Electronics and Communication Engineering Indian Institute of Information Technology, Surat India
Purpose: To develop and evaluate an interpretable artificial intelligence (AI) framework for glaucoma screening from low-cost portable, handheld retinal fundus photographs in a West African population and to compare its performance with clinical tabletop fundus imaging. Methods: We used data from a community-based study of 681 participants (1,362 eyes) in Nigeria, comprising 414 glaucoma, 478 glaucoma suspect, and 470 non-glaucoma eyes. Fundus photographs were acquired using the low-cost handheld, portable Volk Viva retinal camera and the Canon CR-2-AF tabletop camera. We fine-tuned component models separately to each device to perform vessel segmentation, cup and disc boundary segmentation, and feature extraction to detect optic nerve head features. A final classification model combined these components to classify scans as glaucoma, glaucoma suspect or non-glaucoma. Feature-weight analysis and Gradient-weighted Class Activation Mapping were used for interpretation. Results: The models performed well on both Volk Viva and Canon CR-2-AF images: Vessel segmentation: 0.98 Dice Coefficient (DC) (Volk) and 0.94 DC (Canon); Cup and disc segmentation: 0.95 DC (Volk) and 0.96 DC (Canon); Optic nerve head feature detection: area under the receiver operating characteristic curve (AUCs) of 0.83±0.03 (Volk) and 0.87±0.04 (Canon); Classification model: AUCs of 0.85±0.01 (Volk) and 0.93±0.01 (Canon). Reports for each image, present model decision confidence scores and decision-rationale visualizations to support clinical interpretation. Conclusions: Volk Viva results were reasonably comparable to Canon CR-2-AF in the component models and not far behind in classification. This shows that interpretable AI combined with low-cost, portable imaging may enhance community-level glaucoma screening, especially in settings with limited specialist access and resources.
Charis Y. N. Chiang, Tarela Sarimiye, Adeyinka Ashaye +4
Department of Ophthalmology, Emory University, Atlanta, Georgia, United States · Department of Biomedical Engineering, National University of Singapore, Singapore · Singapore Eye Research Institute, Singapore National Eye Centre, Singapore +5
Diabetic Retinopathy (DR) is a major cause of vision impairment worldwide. However, manual diagnosis is often time-consuming and prone to errors, leading to delays in screening. This paper presents a lightweight automated deep learning framework for efficient assessment of DR severity from digital fundus images. We use a MobileNetV3 architecture with a Consistent Rank Logits (CORAL) head to model the ordered progression of disease while maintaining computational efficiency for resource-constrained environments. The model is trained and validated on a combined dataset of APTOS 2019 and IDRiD images using a preprocessing pipeline including circular cropping and illumination normalization. Extensive experiments including 3-fold cross-validation and ablation studies demonstrate strong performance. The model achieves a Quadratic Weighted Kappa (QWK) score of 0.9019 and an accuracy of 80.03 percent. Additionally, we address real-world deployment challenges through model calibration to reduce overconfidence and optimization for mobile devices. The proposed system provides a scalable and practical tool for early-stage diabetic retinopathy screening.
Aadi Joshi, Manav S. Sharma, Vijay Uttam Rathod +3
Vishwakarma Institute Of Technology, Computer Engineering Department, Pune , India · Vishwakarma Institute Of Technology, Computer Science and Engineering (AI & ML ) Department, Pune , India