Known Intents, New Combinations: Clause-Factorized Decoding for Compositional Multi-Intent Detection
Organizations: Microsoft Research India
Abstract
Multi-intent detection papers usually ask whether a model can recover multiple intents from one utterance. We ask a harder and, for deployment, more useful question: can it recover new combinations of familiar intents? Existing benchmarks only weakly test this, because train and test often share the same broad co-occurrence patterns. We introduce CoMIX-Shift, a controlled benchmark built to stress compositional generalization in multi-intent detection through held-out intent pairs, discourse-pattern shift, longer and noisier wrappers, held-out clause templates, and zero-shot triples. We also present ClauseCompose, a lightweight decoder trained only on singleton intents, and compare it to whole-utterance baselines including a fine-tuned tiny BERT model. Across three random seeds, ClauseCompose reaches 95.7 exact match on unseen intent pairs, 93.9 on discourse-shifted pairs, 62.5 on longer/noisier pairs, 49.8 on held-out templates, and 91.1 on unseen triples. WholeMultiLabel reaches 81.4, 55.7, 18.8, 15.5, and 0.0; the BERT baseline reaches 91.5, 77.6, 48.9, 11.0, and 0.0. We also add a 240-example manually authored SNIPS-style compositional set with five held-out pairs; there, ClauseCompose reaches 97.5 exact match on unseen pairs and 86.7 under connector shift, compared with 41.3 and 10.4 for WholeMultiLabel. The results suggest that multi-intent detection needs more compositional evaluation, and that simple factorization goes surprisingly far once evaluation asks for it.
Figures & tables
| Split | Examples | Notes |
|---|---|---|
| Train single | 2,800 | 280 per intent |
| Train pair seen | 6,480 | 27 seen pairs |
| Test pair seen | 1,080 | familiar pairs |
| Test pair unseen | 1,080 | held-out pairs |
| Test pair shift | 1,080 | held-out pairs + new discourse |
| Test pair long shift | 1,080 | held-out pairs + longer/noisier wrappers |
| Model | Seen | Unseen | Pair shift | Long shift | Template holdout | Triple shift |
|---|---|---|---|---|---|---|
| AtomicSet | 100.0 0.0 | 0.0 0.0 | 0.0 0.0 | 0.0 0.0 | 0.0 0.0 | 0.0 0.0 |
| WholeMultiLabel | 100.0 0.0 | 81.4 7.6 | 55.7 11.0 | 18.8 15.3 | 15.5 8.6 | 0.0 0.0 |
| BERTTinyMultiLabel | 100.0 0.0 | 91.5 10.4 | 77.6 20.0 | 48.9 23.4 | 11.0 9.2 | 0.0 0.0 |
| ClauseCompose | 95.2 1.9 | 95.7 2.9 | 93.9 3.3 | 62.5 9.2 | 49.8 17.2 | 91.1 2.6 |
| Model | Manual unseen pairs | Manual pair shift |
|---|---|---|
| WholeMultiLabel | 41.3 3.1 | 10.4 1.2 |
| BERTTinyMultiLabel | 16.7 7.7 | 10.8 6.8 |
| ClauseCompose | 97.5 0.0 | 86.7 7.4 |
Intent Classification
Intent classification assigns user utterances or behaviors to goals, supporting dialogue systems, search, safety decisions, and proactive assistance.
See all 60 papers