cs.CVOct 27, 2026
SaveGradCO: Behavior-Aware Gradient-Conflict Offloading for Text-Based Person Anomaly Search
Organizations: Shandong University · National University of Singapore
Abstract
Text-based Person Anomaly Search (TPAS) retrieves a target pedestrian using natural-language descriptions of appearance and normal or anomalous behavior. Unlike conventional text-based person retrieval, TPAS must preserve identity cues while distinguishing fine-grained behaviors. It presents two challenges: same-identity, different-behavior hard negatives cause gradient conflict during granular alignment, while strong appearance cues can obscure weak behavioral evidence during aggregation. We propose GradCO (behavior-aware Gradient-Conflict Offloading), comprising three complementary components. Behavior-Aware Mixture-of-Experts (B-MoE) forms appearance, normal-behavior, and anomalous-behavior streams whose roles are learned from retrieval objectives without expert-specific semantic labels. Conflict Offload Learning (COL) uses pair-conditioned offload anchors in similarity computation to reduce the conflicting-gradient burden on the main text representations. Hierarchical Intent Alignment (HIA) aggregates expert similarities and features using query-conditioned weights. Experiments on five TPAS/TIPR benchmarks show GradCO significantly surpasses SOTA methods and validate our design.
Figures & tables
Figure 1: (a) illustrates examples of TIPR and TPAS tasks. (b) displays the loss value curves for the InfoNCE loss and our COL-enhanced (detailed in Sec. 3.3 and 3.4 ) on the PAB dataset for TPAS, illustrating the loss oscillation phenomenon caused by gradient conflict. (c) illustrates that within global features, the feature strength of the appearance signal is typically significantly greater than that of the behavior signal, which readily leads to the intent mismatch phenomenon.
Figure 2: Overview of GradCO. B-MoE learns three input-dependent streams whose names denote post-training routing preferences rather than supervised semantic labels. COL predicts pair-conditioned offload anchors for expert-pair similarities, while HIA aggregates expert similarities and features with query-conditioned weights.
| Method | #Data | R@1 | R@5 | R@10 | mAP |
| CLIP (ICML’21) | - | 47.57 | 81.55 | 89.03 | 62.73 |
| X-VLM (ICML’22) | - | 71.94 | 97.78 | 98.99 | 83.96 |
| RaSa (IJCAI’23) | - | 21.74 | 27.30 | 27.96 | 24.35 |
| APTM (MM’23) | - | 22.90 | 45.80 | 52.38 | 33.56 |
| IRRA (CVPR’23) | - | 30.59 | 59.61 | 68.91 | 44.41 |
| MRA (arXiv’25) | - | 9.91 | 23.66 | 31.45 | 17.15 |
Table 1: Performance comparison on PAB dataset. Best results are in bold , and sub-optimal is underlined .
| Method | Normal | Wind | Rain | Snow | Rain+Snow | Dark | Dark+Wind | Dark+Rain | Dark+Snow | Over-exposure | Mean | |||||||||||
| R@1 | mAP | R@1 | mAP | R@1 | mAP | R@1 | mAP | R@1 | mAP | R@1 | mAP | R@1 | mAP | R@1 | mAP | R@1 | mAP | R@1 | mAP | R@1 | mAP | |
| X-VLM (ICML’22) | 83.47 | 90.94 | 79.02 | 88.10 | 54.40 | 67.40 | 59.10 | 72.34 | 49.95 | 62.09 | 79.58 | 88.24 | 75.53 | 85.47 | 34.93 | 45.98 | 50.25 | 63.32 | 74.87 | 84.82 | 64.11 | 74.87 |
| CMP (ICCV’25) | 84.93 | 91.66 | 81.24 | 89.34 | 60.06 | 72.53 | 63.40 | 75.74 | 54.85 | 67.31 | 80.89 | 89.00 | 77.20 | 86.55 | 39.03 | 50.58 | 53.49 | 66.12 | 76.09 | 85.56 | 67.12 | 77.44 |
| GradCO (Ours) | 86.60 | 92.88 | 83.69 | 91.25 | 68.00 | 78.85 | 69.55 | 80.68 | 61.77 | 69.20 | 83.24 | 90.08 | 82.08 | 89.33 | 47.38 | 55.22 | 60.31 | 70.21 | 79.32 | 88.95 | 72.20 | 80.67 |
Table 2: Performance comparison on the PAB dataset of multi-weather test setting, measured by R@1 and mAP for each weather. Our model is highlighted in light purple, and the overall best results are indicated in bold .
| Method | R@1 | R@5 | R@10 | mAP |
| CLIP (ICML’21) | 51.60 | 68.31 | 76.43 | 43.05 |
| X-VL (ICML’22) | 52.33 | 66.73 | 72.54 | 40.87 |
| APTM (MM’23) | 27.86 | 40.41 | 46.77 | 22.61 |
| IRRA (CVPR’23) | 40.28 | 57.24 | 65.98 | 33.53 |
| RaSa (IJCAI’23) | 54.12 | 70.32 | 75.96 | 39.71 |
| CMP (ICCV’25) | 54.12 | 71.07 | 77.90 | 43.13 |
Table 3: Comparisons with existing methods in OOD setting. Best results are indicated in bold , and the overall second-best results are underlined .
| Method | CUHK | ICFG | RSTP | UFineBench | |
| Avg | Avg | Avg | 6926-Avg. | 3C-Avg. | |
| NAFS (ECCV’18) | - | - | - | 76.49 | 58.25 |
| SSAN (arXiv’21) | - | - | - | 85.52 | 67.32 |
| CMKA (TIP’21) | 70.07 | - | - | - | - |
| LapsCore (ICCV’21) | 75.60 | - | - | - | - |
| LGUR (MM’22) | - | - | - | 81.72 | 65.42 |
Table 4: Performance comparison on TIPR task datasets. Best results are shown in bold, and the second-best results are underlined .
| Method | Parameters(M) | GPU Memory(MiB) | Test time(s/sample) | Train Time(s/iteration) | PAB-Avg | MultiWeather-Avg | UCC(OOD)-Avg |
| CMP | 226.13 | 14575 (bs=22) | 0.0015 | 0.5326 (bs=22) | 94.59 | 67.12 | 68.30 |
| GradCO (Ours) | 247.36 | 18413 (bs=22) | 0.0023 | 0.7213 (bs=22) | 95.32 | 72.19 | 76.08 |
Table 5: Comparison of computational efficiency and performance between GradCO and the baseline method CMP. GradCO is highlighted in light purple.
Figure 3: Ablation results for the B-MoE module. (a) Single-expert removal, (b) branch and full-expert removal, and (c) UCC performance ranking across ablation variants. Variant definitions are listed below the panels.
Figure 4: Ablation results for COL and HIA. (a) Effects of removing COL components, with arrows indicating score changes relative to Ours. (b) Performance changes after removing HIA aggregation components. (c) Retrieval performance after removing HIA training objectives. Variant definitions are listed below the panels.
Figure 5: Sensitivity to the hyper-parameters (a) number of sub-experts , and (b) on PAB datasets.
| Method | PAB-Normal | PAB-Anomaly | ||
| R@1 | mAP | R@1 | mAP | |
| CMP | 85.04 | 92.04 | 83.22 | 90.24 |
| GradCO (Ours) | 86.55 | 92.82 | 86.65 | 92.94 |
Table 6: Performance comparison between GradCO and the baseline model CMP on the two subsets (normal and anomaly) of PAB. GradCO is highlighted in light purple, and the overall best results are shown in bold .
Figure 6: Evolution of the offload anchor norm ( ) in the COL module during training. The curves exhibit a trend of initial decline followed by a rise and stabilization. Notably, the anomaly behavior stream (anom) consistently maintains a higher magnitude compared to the appearance (app) and normal behavior (norm) streams.
Figure 7: Trend of cosine similarity between offload anchors and the main feature semantic difference. The similarities for all streams stabilize near 0, indicating that remains approximately orthogonal to the main feature subspace. The anomaly stream (anom) shows a consistent, slight negative deviation (approximately -0.1) compared to the other streams.
Figure 8: Mechanistic analysis of how COL resolves identity–behavior conflicts. (a) Retrieval performance comparison among different COL variants. (b) Gradient cosine similarity between identity-preserving and behavior-discriminative optimization signals. (c) Optimization stability measured by normalized loss volatility. (d) Directional diversity of learned offload anchors.
| Method | ||||
| w/o anchors | 87.0% | 0.072 | 0.18 | – |
| w/o | 45.4% | 0.043 | 0.46 | 0.681 |
| Full COL | 12.2% | 0.019 | 0.78 | 0.129 |
Table 7: Optimization diagnostics of COL variants. Gradient conflict rate and loss volatility measure optimization interference, while retrieval margin and anchor directional similarity characterize retrieval separation and offload geometry, respectively.
Figure 9: Instance-level correction of hard negatives by COL. The scatter plot compares appearance-expert similarity scores before and after applying COL on the same hard negative pairs. Most samples fall below the diagonal , indicating that COL suppresses confusing similarity responses.
Figure 10: Feature alignment and embedding geometry analysis of COL. Each row corresponds to (a) w/o COL, (b) COL w/o , and (c) Full COL. The left column shows positive-pair text-image feature distances. The middle column presents the global feature distribution and angular density. The right columns show the projected distributions of Appearance Expert, Normal Behavior, Abnormal Behavior, and fixed hard negatives.
Figure 11: Retrieval separation and adaptive expert aggregation analysis. (a) KDE of query-level retrieval margins under different COL variants. (b) Joint density of the highest positive and non-positive scores, illustrating the score redistribution induced by COL. (c) Query-level AP distributions of individual experts, uniform averaging, and HIA-weighted aggregation.
Figure 12: Query-conditioned expert activation analysis of HIA. (a–c) KDE distributions of appearance, normal-behavior, and abnormal-behavior gate weights for official Normal and Anomaly queries. Dashed lines indicate group means. (d) Joint density of paired gate variations under controlled text interventions, including appearance masking, behavior masking, and behavior replacement.
Figure 13: Representative visualization of B-MoE gating weights for selected anomaly examples. The anomaly-behavior stream receives relatively higher weights in the shown cases, while the appearance stream remains actively weighted. These examples illustrate input-dependent routing preferences learned by the gating network rather than category-level expert assignment.
Figure 14: Representative visualization of B-MoE gating weights for selected normal examples. The normal-behavior stream receives relatively higher weights in the shown controlled or skillful activities, while the anomaly-behavior stream receives lower preferences. The visualization reflects the adaptive routing behavior of B-MoE without implying a predefined semantic boundary among experts.
Figure 15: Visualization of hard-negative feature separation. Image-to-image cosine similarity matrices of ten selected hard-negative pairs from the PAB test set. (a) X2VLM baseline shows a relatively diffuse similarity pattern with higher cross-identity similarity. (b) GradCO produces a clearer block-like structure, maintaining higher within-identity similarity while reducing similarity among different identities. The visualization qualitatively illustrates improved feature organization under identity–behavior conflicts.
Figure 16: Radar comparison across the reported TPAS and TIPR metrics. GradCO achieves the strongest performance on the displayed TPAS metrics and obtains competitive results on TIPR benchmarks.
Figure 17: Case study on PAB. Green dots • represent complete matches, blue dots • represent identical IDs but mismatched behavior, and orange dots • represent complete mismatches.
Figure 18: Qualitative retrieval comparison on PAB. We select twelve representative TPAS cases involving challenging identity–behavior variations. Green dots • indicate complete matches, blue dots • indicate identical identities with mismatched behaviors, and orange dots • indicate complete mismatches.