MeFEm: Medical Face Embedding model

Yury Borets; Stepan Botman

cs.CVFeb 16, 2026

MeFEm: Medical Face Embedding model

Authors: Yury Borets, Stepan Botman

Organizations: Sber AI Lab

Abstract

We present MeFEm, a vision model based on a modified Joint Embedding Predictive Architecture (JEPA) for biometric and medical analysis from facial images. Key modifications include an axial stripe masking strategy to focus learning on semantically relevant regions, a circular loss weighting scheme, and the probabilistic reassignment of the CLS token for high quality linear probing. Trained on a consolidated dataset of curated images, MeFEm outperforms strong baselines like FaRL and Franca on core anthropometric tasks despite using significantly less data. It also shows promising results on Body Mass Index (BMI) estimation, evaluated on a novel, consolidated closed-source dataset that addresses the domain bias prevalent in existing data. Model weights are available at https://huggingface.co/boretsyury/MeFEm , offering a strong baseline for future work in this domain.

Figures & tables

Dataset name	Original Size	Media type	Usable size
FaceCaption-15M	15 078 935	images	4 572 807
AVSpeech	0 2 621 845	videos	1 446 778
SFHQ	00 425 258	images	0 425 258
Total			6 444 843

Table 1: Composition of training dataset. The ’usable’ refers to samples that were both accessible online and met face-cropping and resolution criteria.

	SHFQ	AVSpeech	FaceCaption
Good samples
Bad samples

Figure 1: Visual examples from the datasets comprising the training set.

Panel of Figure 2 — Figure 2: Schematic representation of different masking strategies: multiblock (a), quadrant (b) and, axial stripes (c). Colored regions represent source the mask; remaining part is the target.

Visual Representation Learning

Visual representation learning develops image and video features that capture useful structure for recognition, retrieval, spatial reasoning, and action.

See all 224 papers

Unmasking Face Embeddings: Reading, Rendering and Naming with Foundation Models
Aug 31, 2026Fizza Rubab, Yiying Tong, Arun RossCross-Modal Alignment Face Recognition
Michigan State University
DIME: Scaling Facial Representation Learning via Differential Masked Autoencoding
Oct 12, 2026Hao Yu, Haoyu Chen, Hui Wei +3Visual Representation Learning Self-Supervised Visual Representation Learning
ELLIS Institute Finland and CMVS, University of Oulu, Finland · DISI, University of Trento, Italy
MEGC2026: Micro-Expression Grand Challenge on Visual Question Answering
Mar 9, 2026Xinqi Fan, Jingting Li, John See +3Micro-Expression Recognition Video QA
Department of Computing and Mathematics, Manchester Metropolitan University · State Key Laboratory of Cognitive Science and Mental Health, Institute of Psychology, CAS & Department of Psychology, University of the Chinese Academy of Sciences · School of Mathematical and Computer Sciences, Heriot-Watt University Malaysia

Aug 31, 2026J/K move · Enter open · S save

Unmasking Face Embeddings: Reading, Rendering and Naming with Foundation Models

Fizza Rubab, Yiying Tong, Arun Ross

Michigan State University

Cross-Modal Alignment Face Recognition

Open paper AskPDF arXiv

Modern face recognition (FR) owes much of its success to deep neural networks that learn to extract compact identity embeddings from face images. These models are typically trained for identity discrimination, producing embeddings that are highly effective for biometric matching but largely opaque to semantic interpretation. In contrast, foundation models, pretrained on broad visual or vision--language tasks, provide rich interfaces for describing, retrieving, generating, and organizing visual content. This contrast raises a natural question: what capabilities become available when face embeddings from domain-specific FR models are made interoperable with foundation models? Building on recent work on embedding compatibility across models, we use simple pre-computed linear transformations, estimated from paired embeddings alone, to connect existing FR models with off-the-shelf foundation models. Once aligned with a foundation model, a face embedding can be 'unmasked' in multiple ways, without training or modifying either model: it can be read in natural language, enabling free-form text queries over a gallery of FR embeddings; rendered into a face image that recovers a person's appearance, using an unmodified diffusion decoder; and converted to a name, enabling identification even in the absence of an enrolled face gallery. In effect, one linear transformation turns an identity embedding into a rich embedding for web-scale foundation models. This interoperability exposes face embeddings as semantically and visually rich biometric representations, with direct implications for interpretability, retrieval, reconstruction, and template security.

Abstract

Visual Representation Learning

Explore similar work

Unmasking Face Embeddings: Reading, Rendering and Naming with Foundation Models

DIME: Scaling Facial Representation Learning via Differential Masked Autoencoding

MEGC2026: Micro-Expression Grand Challenge on Visual Question Answering