Maintaining Difficulty: A Margin Scheduler for Triplet Loss in Siamese Networks Training

Roberto Sprengel Minozzo Tomchak; Oge Marques; Lucas Garcia Pedroso; Luiz Eduardo Oliveira; Paulo Lisboa de Almeida

cs.LGMar 27, 2026

Maintaining Difficulty: A Margin Scheduler for Triplet Loss in Siamese Networks Training

Authors: Roberto Sprengel Minozzo Tomchak, Oge Marques, Lucas Garcia Pedroso, Luiz Eduardo Oliveira, Paulo Lisboa de Almeida

Organizations: Departamento de Informática, Universidade Federal do Paraná, Curitiba, Brazil · College of Engineering and Computer Science, Florida Atlantic University, Boca Raton, FL, U.S.A · Departamento de Matemática, Universidade Federal do Paraná, Curitiba, Brazil

Abstract

The Triplet Margin Ranking Loss is one of the most widely used loss functions in Siamese Networks for solving Distance Metric Learning (DML) problems. This loss function depends on a margin parameter μ, which defines the minimum distance that should separate positive and negative pairs during training. In this work, we show that, during training, the effective margin of many triplets often exceeds the predefined value of μ, provided that a sufficient number of triplets violating this margin is observed. This behavior indicates that fixing the margin throughout training may limit the learning process. Based on this observation, we propose a margin scheduler that adjusts the value of μ according to the proportion of easy triplets observed at each epoch, with the goal of maintaining training difficulty over time. We show that the proposed strategy leads to improved performance when compared to both a constant margin and a monotonically increasing margin scheme. Experimental results on four different datasets show consistent gains in verification performance.

Figures & tables

Fig. 1: To identify the person in the query image (left), the image is compared with all samples in a database, and the identity corresponding to the smallest distance is selected (right) as the answer. Images from LFW [ 4 ] .

Panel of Figure 2 — Fig. 2: Median effective margin of triplets at each training epoch on the LFW dataset.

Panel of Figure 3 — Fig. 3: Histogram of effective margins across training epochs using the LFW dataset. Values below $\mu=0.3$ correspond to hard triplets (red), while values above correspond to easy triplets (green).

Panel of Figure 4 — Fig. 4: Proportion of easy triplets per epoch on the LFW dataset.

Dataset	Train		Test
Dataset	# Classes	# Images	# Classes	# Images
LFW	1,184	6,671	496	2,493
CelebA	7,094	141,859	3,039	60,696
CUB-200-2011	100	5,899	100	5,897
Stanford Cars	98	8,109	98	8,076

TABLE I: Train and test splits of the datasets used in the experiments.

Dataset	Margin	AUC	Recall@1	Recall@2	Recall@4	Recall@8
LFW	Constant	0.956 $\pm$ 0.01	0.273 $\pm$ 0.04	0.385 $\pm$ 0.04	0.500 $\pm$ 0.04	0.620 $\pm$ 0.04
	DAMS	0.966 $\pm$ 0.00	0.395 $\pm$ 0.02	0.500 $\pm$ 0.02	0.605 $\pm$ 0.01	0.703 $\pm$ 0.01
	Linear	0.963 $\pm$ 0.00	0.364 $\pm$ 0.02	0.476 $\pm$ 0.02	0.578 $\pm$ 0.02	0.679 $\pm$ 0.02
CelebA	Constant	0.953 $\pm$ 0.00	0.137 $\pm$ 0.00	0.211 $\pm$ 0.01	0.303 $\pm$ 0.01	0.411 $\pm$ 0.01
	DAMS	0.956 $\pm$ 0.00	0.217 $\pm$ 0.02	0.309 $\pm$ 0.02	0.411 $\pm$ 0.02	0.519 $\pm$ 0.02
	Linear	0.940 $\pm$ 0.00	0.101 $\pm$ 0.00	0.161 $\pm$ 0.00	0.243 $\pm$ 0.00	0.347 $\pm$ 0.00

TABLE II: Metrics Results with each Method in the Datasets

Panel of Figure 7 — Fig. 5: Margin evolution for each method across datasets.

$\mu_{0}$	$t$	$s_{a}$	Recall@1	AUC
Best Configurations
0.0	99%	0.02	0.411 $\pm$ 0.02	0.967 $\pm$ 0.00
0.0	95%	0.01	0.403 $\pm$ 0.01	0.963 $\pm$ 0.01
0.0	99%	0.05	0.387 $\pm$ 0.03	0.965 $\pm$ 0.01
0.0	95%	0.02	0.383 $\pm$ 0.00	0.964 $\pm$ 0.00
0.0	99%	0.01	0.380 $\pm$ 0.02	0.966 $\pm$ 0.00

TABLE III: Results for Different Parameters in the Adaptive Method

Metric Learning

Metric learning trains distances or embedding geometries so task-relevant examples are close and irrelevant ones far apart for retrieval, clustering, and recognition.

See all 79 papers

Task-Guided Multi-Annotation Triplet Learning for Remote Sensing Representations
Apr 4, 2026Meilun Zhou, Alina ZareRepresentation Learning Multi-Task Learning
Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL, USA, 32611 · Artificial Intelligence and Informatics Research Institute, University of Florida, Gainesville, FL, USA, 32611
Minor First, Major Last: A Depth-Induced Implicit Bias of Sharpness-Aware Minimization
Mar 9, 2026Chaewon Moon, Dongkuk Si, Chulhee YunSharpness-Aware Minimization Implicit Bias
Kim Jaechul Graduate School of AI, KAIST · Mobilint, Inc.
SMAT: Simple and Efficient Merge-Aware Training
Sep 27, 2026Yanggan Gu, Yuanyi Wang, Zhen Li +5Language Model Merging Model Merging
The Hong Kong Polytechnic University (PolyU) · The Hong Kong University of Science and Technology (Guangzhou) · The Chinese University of Hong Kong +1

Apr 4, 2026J/K move · Enter open · S save

Task-Guided Multi-Annotation Triplet Learning for Remote Sensing Representations

Meilun Zhou, Alina Zare

Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL, USA, 32611 · Artificial Intelligence and Informatics Research Institute, University of Florida, Gainesville, FL, USA, 32611

Representation Learning Multi-Task Learning Geospatial Representation Learning Metric Learning Multi-Task Optimization

Open paper AskPDF arXiv

Prior multi-task triplet loss methods relied on static weights to balance supervision between various types of annotation. However, static weighting requires tuning and does not account for how tasks interact when shaping a shared representation. To address this, the proposed task-guided multi-annotation triplet loss removes this dependency by selecting triplets through a mutual-information criteria that identifies triplets most informative across tasks. This strategy modifies which samples influence the representation rather than adjusting loss magnitudes. Experiments on an aerial wildlife dataset compare the proposed task-guided selection against several triplet loss setups for shaping a representation in an effective multi-task manner. The results show improved classification and regression performance and demonstrate that task-aware triplet selection produces a more effective shared representation for downstream tasks.

Abstract

Metric Learning

Explore similar work

Task-Guided Multi-Annotation Triplet Learning for Remote Sensing Representations

Minor First, Major Last: A Depth-Induced Implicit Bias of Sharpness-Aware Minimization

SMAT: Simple and Efficient Merge-Aware Training