SKINOPATHY AI: Smartphone-Based Ophthalmic Screening and Longitudinal Tracking Using Lightweight Computer Vision

S. Kalaycioglu; C. Hong; M. Zhu; H. Xie

cs.CVFeb 26, 2026

SKINOPATHY AI: Smartphone-Based Ophthalmic Screening and Longitudinal Tracking Using Lightweight Computer Vision

Authors: S. Kalaycioglu, C. Hong, M. Zhu, H. Xie

Organizations: Toronto Metropolitan University · York University · AIMechatroniX Inc. · DrRobot Inc ·

Abstract

Early ophthalmic screening in low-resource and remote settings is constrained by access to specialized equipment and trained practitioners. We present SKINOPATHY AI, a smartphone-first web application that delivers five complementary, explainable screening modules entirely through commodity mobile hardware: (1) redness quantification via LAB a* color-space normalization; (2) blink-rate estimation using MediaPipe FaceMesh Eye Aspect Ratio (EAR) with adaptive thresholding; (3) pupil light reflex characterization through Pupil-to-Iris Ratio (PIR) time-series analysis; (4) scleral color indexing foricterus and anemia proxies via LAB/HSV statistics; and (5) iris-landmark-calibrated lesion encroachment measurement with millimeter-scale estimates and longitudinal trend tracking. The system is implemented as a React/FastAPI stack with OpenCV and MediaPipe, MongoDB-backed session persistence, and PDF report generation. All algorithms are fully deterministic, privacy-preserving, and designed for non-diagnostic consumer triage. We detail system architecture, algorithm design, evaluation methodology, clinical context, and ethical boundaries of the platform. SKINOPATHY AI demonstrates that multi-signal ophthalmic screening is feasible on unmodified smartphones without cloud-based AI inference, providing a foundation for future clinically validated mobile ophthalmoscopy tools.

Figures & tables

Tier	Technology	Role
Frontend	React, Chart.js, MediaRecorder API, Axios	UI, consent, media capture, visualization
Backend	FastAPI, OpenCV, MediaPipe FaceMesh, ReportLab	Image/video processing, metrics, PDF export
Database	MongoDB (Motor async client), UUID keys	Session storage, result persistence, trend data
DevOps	Supervisor, Kubernetes ingress, .env variables	Process management, routing, configuration

Table 1: System Technology Stack by Tier

Figure 1: Schematic overview of SKINOPATHY AI three-tier architecture.

Symbol	Domain / Units	Definition
I	$H\times W\times 3$ , uint8	Input BGR image from smartphone camera
I ${}_{\text{LAB}}$	$H\times W\times 3$ , float	CIE LAB image; OpenCV scaling: $L\in[0,255]$ , $a^{}\in[0,255]$ , $b^{}\in[0,255]$
I ${}_{\text{HSV}}$	$H\times W\times 3$ , uint8	HSV image; $H\in[0,179]$ , $S\in[0,255]$ , $V\in[0,255]$
M	$H\times W$ , bool	Binary mask; M(x,y)=1 if pixel belongs to region of interest
$\mathbb{E}_{M}[\cdot]$	—	Spatial expectation over mask: $(1/N_{M})\sum_{(x,y):M(x,y)=1}f(x,y)$
p i	$\mathbb{R}^{2}$	Unnormalized pixel-space landmark coordinate (x i , y i )

Table 2: Unified Notation for All Module Algorithms

Figure 2: SKINOPATHY AI — Red-Eye Meter module. The user captures or uploads an eye photograph; the system computes a redness score (0–10) via LAB a* analysis and displays triage guidance. Illustrated output: 3.66/10 (mild, monitor). Camera device selection and live preview are provided to guide optimal capture. Screening only; not diagnostic.

Module	Primary Metric	Pilot Value	Target (Full Study)	Status
Redness	Spearman r s	0.86	> 0.80 vs. Efron grade	Pilot complete
Blink Rate	MAE (blinks/min)	2.1 $\pm$ 0.9	< 2.5 blinks/min MAE	In progress
Pupil Reflex	Latency error (ms)	42 $\pm$ 18 ms	< 50 ms vs. pupillometer	Pilot complete
Color Screen	Icterus AUC	0.79	> 0.80 vs. clinical grade	In progress
Lesion Track.	MAE (mm)	0.31 $\pm$ 0.12 mm	< 0.5 mm vs. slit-lamp	Ongoing

Table 3: Pilot Performance Summary by Module

Figure 3: SKINOPATHY AI — Blink Coach module. The user records a 10-second selfie video; MediaPipe FaceMesh and adaptive EAR thresholding detect blink events. Illustrated output: 17 blinks detected, rate = 102.05 blinks/min (elevated; re-recording with stable head position recommended). The module provides blink count, rate, and normative guidance (typical adult range: 15–20 blinks/min).

Figure 4: SKINOPATHY AI — Pupil Reflex Test module. The user records a 10-second video while briefly exposing the eye to a bright light source at approximately t = 3 s. MediaPipe FaceMesh with iris refinement tracks the Pupil-to-Iris Ratio (PIR) time-series. Key metrics extracted include PLR amplitude ( $\Delta_{\text{rel}}$ ), latency (L ${}_{\text{ms}}$ ), mean constriction velocity (v ${}_{\text{mean}}$ ), and a recording quality score Q. The module targets neurological screening signals; absent or markedly delayed constriction warrants clinical follow-up. This figure shows the baseline without the application of a bright light source.

Figure 5: SKINOPATHY AI — Pupil Reflex Test module (additional view). The user records a 10-second video while briefly exposing the eye to a bright light source at approximately t = 3 s. MediaPipe FaceMesh with iris refinement tracks the Pupil-to-Iris Ratio (PIR) time-series.

Figure 6: SKINOPATHY AI — Color Health Screen module. The user uploads or captures an eye photograph; the system computes the Yellow Index (icterus proxy from LAB b*) and Pallor Index (anemia proxy from LAB L* and a*) over a luminance-gated scleral mask. Gray-world white balance correction is applied before color analysis. Outputs include both indices on [0,1] with uncertainty bands and conservative triage messaging. Screening only; not diagnostic.

Figure 7: SKINOPATHY AI — Lesion Detection & Tracking module. MediaPipe FaceMesh iris landmarks calibrate the pixel-to-millimeter conversion factor using the population mean HVID (11.8 mm). A multi-space color filter (LAB + HSV) segments candidate lesion pixels within the near-limbus annular analysis band (0.65–0.98 $\rho_{\text{iris}}$ ). Illustrated output: iris diameter assumed 11.8 mm; distance from limbus = 0 mm (no lesion detected). The OLS trend estimator flags progressive lesions when $\hat{v}_{\text{growth}}$ > 0.005 mm/day. Heuristic MVP; trained segmentation model planned for future release.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Endpoint	Description
POST	/api/sessions	Create new session; accepts consent and intake JSON; returns session_id (UUID)
GET	/api/sessions	List all sessions for the current user context; returns array of session summaries
GET	/api/sessions/{id}	Retrieve full session document including all module results and intake data
GET	/api/sessions/{id}/report.pdf	Generate and download PDF report for session; includes all intake and results
POST	/api/analyze/redness	Upload eye photo; returns redness_score (0-10), guidance, LAB a* statistics
POST	/api/analyze/blink	Upload 10s video; returns blink_count, blink_rate_per_min, EAR time-series

Appendix

Table 4: REST API Endpoint Reference

Field	Type	Description
id	UUID string	Primary key; universally unique session identifier
created_at	ISO 8601 datetime	Session creation timestamp (UTC)
intake.consent	Boolean	Explicit user consent flag; must be true to proceed
intake.name	String	User-provided name for PDF report header
intake.email	String (optional)	Email for session retrieval link; not shared
intake.phone	String (optional)	Phone number; included in PDF report only

Appendix

Table 5: MongoDB Session Document Schema

Ophthalmic Image Analysis

See all 42 papers

AnemiaVision: Non-Invasive Anemia Detection via Smartphone Imagery Using EfficientNet-B3 with TrivialAugmentWide, Mixup Augmentation, and Persistent Patient History Management
Apr 24, 2026Rahul PatelMedical Imaging Medical Image Classification
Department of Electronics and Communication Engineering Indian Institute of Information Technology, Surat India
Interpretable AI plus Handheld, Portable Retinal Photographs: A Low-Cost Glaucoma Screening Solution for West Africa
Sep 22, 2026Charis Y. N. Chiang, Tarela Sarimiye, Adeyinka Ashaye +4Fundus Imaging Medical Diagnosis
Department of Ophthalmology, Emory University, Atlanta, Georgia, United States · Department of Biomedical Engineering, National University of Singapore, Singapore · Singapore Eye Research Institute, Singapore National Eye Centre, Singapore +5
Mobile-Ready Automated Triage of Diabetic Retinopathy Using Digital Fundus Images
Feb 25, 2026Aadi Joshi, Manav S. Sharma, Vijay Uttam Rathod +3Diabetic Retinopathy Grading Medical Image Classification
Vishwakarma Institute Of Technology, Computer Engineering Department, Pune , India · Vishwakarma Institute Of Technology, Computer Science and Engineering (AI & ML ) Department, Pune , India

Apr 24, 2026J/K move · Enter open · S save

AnemiaVision: Non-Invasive Anemia Detection via Smartphone Imagery Using EfficientNet-B3 with TrivialAugmentWide, Mixup Augmentation, and Persistent Patient History Management

Rahul Patel

Department of Electronics and Communication Engineering Indian Institute of Information Technology, Surat India

Medical Imaging Medical Image Classification Mixup

Open paper AskPDF arXiv

Anemia affects over one billion people globally and remains severely under-diagnosed in low-resource regions where laboratory blood tests are inaccessible. This paper presents AnemiaVision, an end-to-end web-based system for non-invasive anemia screening from smartphone photographs of the palpebral conjunctiva and fingernail beds. The proposed pipeline fine-tunes a pre-trained EfficientNet-B3 backbone with a redesigned three-layer classifier head incorporating BatchNorm, GELU activations, and high-rate Dropout (0.45/0.35). Training employs four orthogonal accuracy-boosting techniques: TrivialAugmentWide for policy-free image augmentation, RandomErasing for spatial regularisation, Mixup (alpha=0.2) for inter-class smoothing, and cosine-annealing scheduling with linear warmup. Early stopping is governed by peak validation accuracy rather than validation loss to prevent premature termination on high-variance epochs. The deployed Flask application integrates persistent patient-history management backed by PostgreSQL on Render, with an automated database-migration entrypoint ensuring zero data loss across redeploys. Ablation experiments demonstrate that accuracy-first early stopping contributes +1.6% and Mixup contributes +2.8% to final validation accuracy. Overall, the proposed system achieves a validation accuracy of 96.2% and AUC-ROC of 0.98, compared with 44.9% validation accuracy and AUC-ROC of 0.58 from the three-epoch CPU-only baseline. Sensitivity for the anemic class reaches 0.96, making the system suitable as a first-line screening tool for community health workers in rural settings. The system is publicly accessible and source code is openly available.

Abstract

Ophthalmic Image Analysis

Explore similar work

AnemiaVision: Non-Invasive Anemia Detection via Smartphone Imagery Using EfficientNet-B3 with TrivialAugmentWide, Mixup Augmentation, and Persistent Patient History Management

Interpretable AI plus Handheld, Portable Retinal Photographs: A Low-Cost Glaucoma Screening Solution for West Africa

Mobile-Ready Automated Triage of Diabetic Retinopathy Using Digital Fundus Images