cs.LGAug 16, 2026

Quantum Models with Multi-Stage Training for Compositional Concept Generalization

Authors: Mina Abbaszadeh, Matilda Karabina Moore, Mehrnoosh Sadrzadeh, Martha Lewis

Organizations: Department of Computer Science University College London London, United Kingdom · Institute for Logic, Language and Computation University of Amsterdam Amsterdam, Netherlands

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Compositional Concept Generalization (CoCoGen), the ability to systematically recombine learned primitives in novel contexts, is a key challenge for multimodal learning. In this work, we provide a solution using a compositional model of meaning that separates nouns from relations and uses tensors and variational quantum circuits to train them on data. This model enables us to employ a multi stage training paradigm, one that first learns object representations from single-object image-caption pairs, then subsequently transfers these to the relational stage where object parameters are frozen and optimisation is only applied to relational components. This design explicitly enforces compositional factorisation at the circuit, ensuring that relations are learned as transformations over stable primitives. The training paradigm is tested on the CLEVR dataset developed specificially for CoCoGen. For text, we work with vector representations of nouns and higher order tensor representations of relations using a set of different ansatz. For images, we work with quantum encodings of image embeddings dervied from Open AI's Vision Language tool CLIP and contrast amplitude encoding, which preserves the original embedding geometry, with angle encoding, which introduces nonlinear feature transformations. Our results show that multi-staged training combined with structured encodings significantly improves out of distribution relational generalisation, while using orders of magnitude fewer trainable parameters than classical baselines. We find that performance gains arise from the interaction between representation and encoding, with nonlinear quantum encodings enhancing the separability of compositional structure. These findings demonstrate that structured quantum representations and staged learning provide an effective framework for compositional generalisation in multimodal quantum machine learning.

Figures & tables

Fig. 1 : A multi-stage procedure for compositional image–caption alignment. After optimizing the noun parameters during single-object training, the learned noun parameters are transferred to relational training. In the collage image encoding, relational image circuits are constructed by combining the quantum circuits of single-object images.
DatasetTrainOOD ValOOD Test
Single-Object13604001100
Two-Object74406003700
Relational440250400
TABLE I : Dataset splits for the compositional generalisation benchmark [ 4 ] .
Fig. 2 : Example from the relational dataset with the correct caption “cube left cone” and the relation-swapped distractor “cube right cone”. The corresponding DisCoCat diagram is shown on the left.
TABLE II : MHE vectors for sentences shape.
Fig. 3 : Collage construction of relational image representations from single-object components learned in Stage 1.
Fig. 4 : Quantum circuit used for sentence encoding during relational training. Noun parameters are transferred from single-object training and connected to the relational operator. The qubits corresponding to the noun wires are contracted with the relation (via cups) and post-selected, leaving the ss -wire qubits as the output sentence representation used for similarity computation.
StructureTrainValidTestparametersText-qubitsImage-qubits
Image_sim4 & Text_sim424.76%25%33.33%9633
Image_sim4 & Text_IQP27.18%25%33.33%8033
Image_IQP & Text_IQP80.54%45%53.34%32099
Image_IQP & Text_sim478.66%75.00%66.70%31299
TABLE IV : Performance of different ansatz choices for image and text encoding circuits in the single-object training stage. Validation and test splits are out-of-distribution (OOD), and results are averaged over five random seeds.
Fig. 5 : The left circuit depicts the quantum circuit used for sentence encoding, while the right circuit depicts the quantum circuit used for image encoding during single-object training.
ModelsMethodTrainOOD-ValidOOD-TestBest TestParameters
OHE 92.20%95.00%93.34%100.00%168
Q-CLIPAngle Enc.78.66%75.00%66.70%70.31%312
 Amplitude Enc.87.37%79.71%80.97%100.00%312
Classical Baseline      
Classical-CLIPCTP95.63%90.45%91.00%95.18%63,428,097
TABLE V : Results of single object training. Columns Train, OOD-Valid, and OOD-Test reported average accuracy over 5 random seeds. Best Test reports the best accuracy achieved across the 5 seeds. Parameters reports number of parameters of each model.
ModelsMethodTrainOOD-ValidOOD-TestBest TestParameters
MHEBinary99.00%73.00%85.88%91.66279
Collage Encoding      
 Angle Enc. with Entangl.82.20%61.05%72.32%75.00426
 Angle Enc. No Entangl.82.11%64.00%71.66%75.00426
 Ampl. Enc. with Entangl.62.28%40.00%48.58%50.00426
 Ampl. Enc. No Entangl.60.14%40.00%43.33%50.00426
TABLE VI : Results of relational learning.Columns Train, OOD-Valid, and OOD-Test reported average accuracy over 5 random seeds. Best Test reports the best accuracy achieved across the 5 seeds. Parameters reports number of parameters of each model.
Fig. 6 : Amplitude vs. angle encoding state fidelity on the test images. The desired pattern is block diagonal, indicating high within-class similarity and low between-class similarity. Labels use the abbreviations: cu (cube), cy (cylinder), co (cone), s (sphere), L (left), and R (right).
More in this topic

Compositional Generalization

Compositional generalization is the ability to combine learned parts and rules in ways that transfer to unseen combinations.

See all 112 papers

Explore similar work

CardsList
Oct 6, 2026

Mu-DisCoCat: A Variational Pipeline for Compositional Generalization on Quantum Processors

Achieving compositional concept generalization (CoCoGen), the ability to understand novel situations by recombining learned primitives, remains a fundamental challenge in artificial intelligence. Compositional semantic models such as Compositional Distributional Semantics (DisCoCat) offer solutions by generalising vectors to tensors, but suffer from scaling bottlenecks when learning the tensors. Mapping DisCoCat onto Variational Quantum Circuits (VQCs) resolves this limitation for text, yet the methodology has not been expanded to multimodal situations such as the ones involved in CoCoGen. This paper introduces Mu-DisCoCat: a multimodal variational quantum learning framework for DisCoCat that achieves CoCoGen. The framework first learns stable object representations from single-object image-text pairs, then fixes these and uses them to learn the relations between them in multi-object situations. In classical simulations, the model used Uhlmann state fidelity to compute the overlap between the multimodal circuit representations and achieved higher relational OOD accuracy than the evaluated CLIP baseline. Its deployment was evaluated using the destructive SWAP test across noisy quantum emulators, including a range of IBM fake backends, IQM FakeAphrodite, and the IBM Marrakesh quantum processor. Despite real-world device noise, the hardware-executed models maintained a strong positive correlation with simulated fidelities, reliably distinguishing unseen similar and dissimilar pairs. Our work establishes a framework for executing CoCoGen on VQCs, demonstrating a viable use case for near-term quantum hardware.

Mina Abbaszadeh, Matilda Karabina Moore, Raem Haq +2

Department of Computer Science University College London London, United Kingdom · Institute for Logic, Language and Computation University of Amsterdam Amsterdam, Netherlands

Quantum Machine LearningVariational Quantum CircuitsMultimodal LearningCompositional Generalization
AskPDF
Oct 6, 2026

Quantum Entangled Multimodal Fusion Networks (QEMFN): Resource-Aware Hybrid Vision-Language Fusion via Trainable Entanglement

Multimodal vision-language systems typically fuse image and text embeddings through classical operators such as concatenation, attention, bilinear pooling, or tensor interactions. We propose Quantum Entangled Multimodal Fusion Networks (QEMFN), a hybrid quantum-classical framework that introduces parameterized entanglement as a structured inductive bias for multimodal fusion. Pretrained visual and textual features are projected into compact latent spaces, encoded as angle-parameterized quantum states, processed through intra-modal and paired cross-modal entangling circuits, and measured to produce fused representations for retrieval. Under matched parameter budgets and identical frozen CLIP backbones, QEMFN outperforms classical fusion baselines on COCO-5k and Flickr30k, including multilayer perceptron, tensor fusion, FiLM, cross-attention, compact transformer, and a dequantized paired-topology analogue. An ablation suite isolates the quantum module's contribution from the surrounding classical projections, and quantum-centric analyses report Meyer-Wallach entangling capability, expressibility, gradient variance against barren-plateau bounds, and entropy-performance correlation under controls for training progress alongside an intervention study on the entangling component. QEMFN is executed under shot-based estimation, a noise-modeled fake backend, and a real superconducting device with zero-noise extrapolation. This work does not claim quantum computational advantage; the contribution is the framework together with a controlled empirical and quantum-centric evaluation that positions trainable entanglement as an interpretable, hardware-executable fusion mechanism at scales accessible on contemporary devices.

Srikar Alla, Ali Shiri Sichani, Chi-Ren Shyu

Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO 65211, USA · Institute for Data Science and Informatics, University of Missouri, Columbia, MO 65211, USA

Multimodal FusionHybrid Quantum-Classical MLImage-Text RetrievalQuantum Neural Networks
AskPDF
Jul 15, 2026

PQFA: Parallel Quantum Feature Augmentation of Fused Representations for Multimodal Classification

Most multimodal learning methods improve how heterogeneous representations are aligned and fused, while post-fusion enhancement remains less explored. We propose Parallel Quantum Feature Augmentation (PQFA), a hybrid quantum-classical framework that applies multiple shallow variational quantum circuits to fused multimodal features. Text and image representations extracted by frozen RoBERTa and ViT encoders are processed through bidirectional cross-attention, attentive pooling, and adaptive gated fusion. The fused feature is then amplitude-encoded into parallel quantum circuits, whose measurement readouts are concatenated with the classical representation for prediction. We evaluate PQFA on MM-IMDb and N24News through controlled comparisons using the same encoders, fusion backbone, data splits, projection dimension, and augmentation output width. PQFA consistently outperforms both the fusion backbone without quantum augmentation and a width-matched MLP augmentation baseline, while using approximately 2.2K augmentation parameters compared with 24.0K for the MLP branch. Missing-modality experiments further show improved robustness when textual or visual inputs are incomplete, with particularly clear gains when the more informative textual modality is severely degraded. Controlled ablations and feature-space analyses indicate that the improvement cannot be reproduced by random feature mappings, increased classical width, or untrained quantum transformations. Quantum-state diagnostics additionally show stable predictive performance across the tested simulated noise levels and distinct branch-specific transformations of the encoded states. These results establish PQFA as an effective and parameter-efficient strategy for post-fusion augmentation in hybrid quantum-classical multimodal learning.

Mingzhu Wang, Yun Shang

Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 100190, China · School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, 100049, China · State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 100190, China

Quantum Feature MapsQuantum Machine LearningHybrid Quantum-Classical MLMultimodal ClassificationMultimodal Fusion
AskPDF