Photonic Quantum-Enhanced Knowledge Distillation

Kuan-Cheng Chen; Shang Yu; Chen-Yu Liu; Samuel Yen-Chi Chen; Huan-Hsin Tseng; Yen Jui Chang; Wei-Hao Huang; Felix Burt

quant-phMar 16, 2026

Photonic Quantum-Enhanced Knowledge Distillation

Authors: Kuan-Cheng Chen, Shang Yu, Chen-Yu Liu, Samuel Yen-Chi Chen, Huan-Hsin Tseng, Yen Jui Chang, Wei-Hao Huang, Felix Burt,

Organizations: Department of Electrical and Electronic Engineering, Imperial College London, South Kensington, London, SW7 2AZ, England, United Kingdom. · Imperial Centre for Quantum Engineering, Science and Technology, Imperial College London, South Kensington, London, SW7 2AZ, England, United Kingdom. · JIJ, Rutherford Appleton Laboratory, Harwell Campus, Didcot, OX11 0QX, United Kingdom. · JIJ, 3-3-6 Shibaura, Minato-ku, Tokyo, 108-0023, Japan. · Blackett Laboratory, Department of Physics, Imperial College London, South Kensington, London, SW7 2AZ, England, United Kingdom. · Graduate Institute of Applied Physics, National Taiwan University, No. 1, Sec. 4, Roosevelt Rd., Taipei, 106319, Taiwan. · Computational Science Initiative, Brookhaven National Laboratory, Bldg. 725, Room 2-189, P.O. Box 5000, Upton, NY, 11973-5000, USA. · Quantum Information Center, Chung Yuan Christian University, No. 200, Zhongbei Rd., Zhongli Dist., Taoyuan City, 320314, Taiwan. · Master Program in Intelligent Computing and Big Data, Chung Yuan Christian University, Zhongbei Rd, aoyuan City, 320314, Taiwan. · NVIDIA Corporation, Santa Clara, CA, USA. · ORCA Computing, London, United Kingdom. · Department of Physics, University of Oxford, Parks Road, Oxford, OX1 3PU, England, United Kingdom.

Abstract

Photonic quantum processors naturally produce intrinsically stochastic measurement outcomes, offering a hardware-native source of structured randomness that can be exploited during machine-learning training. Here we introduce Photonic Quantum-Enhanced Knowledge Distillation (PQKD), a hybrid quantum photonic--classical framework in which a programmable photonic circuit generates a compact conditioning signal that constrains and guides a parameter-efficient student network during distillation from a high-capacity teacher. PQKD replaces fully trainable convolutional kernels with dictionary convolutions: each layer learns only a small set of shared spatial basis filters, while sample-dependent channel-mixing weights are derived from shot-limited photonic features and mapped through a fixed linear transform. Training alternates between standard gradient-based optimisation of the student and sampling-robust, gradient-free updates of photonic parameters, avoiding differentiation through photonic hardware. Across MNIST, Fashion-MNIST and CIFAR-10, PQKD traces a controllable compression--accuracy frontier, remaining close to teacher performance on simpler benchmarks under aggressive convolutional compression. Performance degrades predictably with finite sampling, consistent with shot-noise scaling, and exponential moving-average feature smoothing suppresses high-frequency shot-noise fluctuations, extending the practical operating regime at moderate shot budgets.

Figures & tables

Figure 1: Hardware–algorithm co-design for PQKD enabling neural network compression. A CV photonic module prepares a fixed input state $\rho_{\mathrm{in}}$ and applies a programmable unitary $U(\theta)$ implemented by an integrated interferometric mesh. The output state $\rho_{\theta}=U(\theta)\rho_{\mathrm{in}}U^{\dagger}(\theta)$ is measured using an SNSPD array with time-tagging, producing i.i.d. samples $\{\omega_{s}\}_{s=1}^{S}$ and an empirical distribution $\hat{p}_{\theta}$ . A robust classical feature extractor $\Phi(\cdot)$ maps $\hat{p}_{\theta}$ to a fixed-length conditioning vector $z(\theta)=\Phi(\hat{p}_{\theta})\in\mathbb{R}^{d}$ , which modulates the student model $f_{S}$ during knowledge distillation from a pretrained teacher $f_{T}$ . Crucially, $\theta$ is updated by a classical optimiser using a distillation objective $L_{\mathrm{KD}}(\omega,\theta)$ and requires sampling only, avoiding differentiation through photonic hardware.

Dataset	Accuracy (%)				Final CE loss		Opt.
	train		validation		validation		Photonic
	Teacher	PQKD	Teacher	PQKD	Teacher	PQKD	$\Delta$
MNIST	100.00 ( $\pm$ 0.00)	99.71 ( $\pm$ 0.02)	99.07 ( $\pm$ 0.13)	99.09 ( $\pm$ 0.09)	0.031 ( $\pm$ 0.002)	0.030 ( $\pm$ 0.001)	5.94 ( $\pm$ 0.02)
Fashion-MNIST	100.00 ( $\pm$ 0.00)	97.10 ( $\pm$ 0.18)	91.86 ( $\pm$ 0.26)	92.42 ( $\pm$ 0.16)	0.291 ( $\pm$ 0.003)	0.239 ( $\pm$ 0.004)	6.73 ( $\pm$ 0.03)
CIFAR-10	91.81 ( $\pm$ 0.13)	80.35 ( $\pm$ 0.26)	86.99 ( $\pm$ 0.29)	79.72 ( $\pm$ 0.37)	0.397 ( $\pm$ 0.004)	0.597 ( $\pm$ 0.008)	4.95 ( $\pm$ 0.02)

Table 1: PQKD performance and optimisation (Opt.) diagnostics. Values are mean

\pm

s.d. over five independent runs. Accuracies are in %. CE denotes cross-entropy. Photonic

\Delta

is (epoch 1)

-

(final epoch).

Figure 2: Training and validation dynamics across datasets. Mean $\pm$ s.d. over five independent runs comparing the teacher (green) and PQKD student (blue) on MNIST a , Fashion-MNIST b , and CIFAR-10 c . Top row: training accuracy. Middle row: validation accuracy. Bottom row: cross-entropy (CE) loss for training (solid) and validation (dashed)

Figure 3: Scaling photonic-conditioned convolutional compression across network scope and teacher capacity on MNIST. Validation accuracy drop relative to the corresponding teacher in percentage points, plotted against the overall compression factor of the compressed network (bottom axis; compression_x ). Columns report the compression scope: Conv1 (left), Conv1+Conv2 (middle), and All Convs (right). Rows correspond to three teacher widths,

(c_{1},c_{2},c_{3})\in\{(32,64,128),(48,96,128),(64,128,128)\}

, with each row aggregating all configurations for that teacher. Marker shape encodes the photonic parameter dimension

\dim(\theta)\in\{15,30,45\}

, and marker size encodes the kernel-manifold rank

R\in\{4,8,12\}

. Faint markers show individual seeds, while opaque markers and vertical bars indicate the mean and one standard deviation across seeds.

Scope	Teacher $(c_{1},c_{2},c_{3})$	$C_{x}$	Train accuracy (%)		Validation accuracy (%)
			Teacher	PQKD	Teacher	PQKD
Conv1	$(32,64,128)$	1.01	100.00 ( $\pm$ 0.00)	99.91 ( $\pm$ 0.02)	98.91 ( $\pm$ 0.09)	99.03 ( $\pm$ 0.08)
	$(48,96,128)$	1.01	100.00 ( $\pm$ 0.00)	99.98 ( $\pm$ 0.00)	99.11 ( $\pm$ 0.06)	99.20 ( $\pm$ 0.11)
	$(64,128,128)$	1.01	100.00 ( $\pm$ 0.00)	100.00 ( $\pm$ 0.00)	99.06 ( $\pm$ 0.18)	99.07 ( $\pm$ 0.09)
Conv1+2	$(32,64,128)$	1.25	100.00 ( $\pm$ 0.00)	99.45 ( $\pm$ 0.03)	98.91 ( $\pm$ 0.09)	98.93 ( $\pm$ 0.04)
	$(48,96,128)$	1.38	100.00 ( $\pm$ 0.00)	99.57 ( $\pm$ 0.03)	99.11 ( $\pm$ 0.06)	99.00 ( $\pm$ 0.05)

Table 2: Representative operating points for scaling convolutional compression on MNIST (fixed

R{=}4

\dim(\theta){=}30

C_{x}

denotes the overall compression factor (teacher trainable parameters divided by student trainable parameters, including

\theta

). Accuracies are mean

\pm

s.d. over 5 seeds, reported at the final epoch.

Figure 4: Shot-limited scaling of the photonic conditioning signal in PQKD. Increase in test accuracy due to the photonic feature,

\delta\,\mathrm{test\ acc}\equiv\mathrm{acc}(z)-\mathrm{acc}(z{=}0)

, as a function of the photonic measurement budget

S

(shots), comparing EMA aggregation of the feature across epochs (blue) with no EMA (orange). Solid curves show the mean over random seeds and shaded bands denote

\pm 1

s.d. The dashed curves are fits to the shot-noise model

\delta(S)=\delta_{\infty}-k/\sqrt{S}

, consistent with multinomial histogram fluctuations propagating through a Lipschitz feature map, which predicts a

1/\sqrt{S}

decay of stochastic feature perturbations and a corresponding saturation of performance at large

S

Name	Gate	Unitary	Parameters	Gaussian	Modes
Displacement	$\hat{D}(\alpha)$	$\exp(\alpha\hat{a}^{\dagger}-\alpha^{\ast}\hat{a})$	$\alpha\in\mathbb{C}$ (or $a,\theta$ )	✓	1
Squeezing	$\hat{S}(s)$	$\exp\!\big(\tfrac{1}{2}(s^{\ast}\hat{a}^{2}-s\hat{a}^{\dagger 2})\big)$	$s\in\mathbb{C}$ (or $r,\phi$ )	✓	1
Rotation	$\hat{R}(\theta)$	$\exp(-i\theta\hat{n})$	$\theta\in[0,2\pi)$	✓	1
Beam splitter	$\hat{BS}(\theta,\phi)$	$\exp(\theta(e^{i\phi}\hat{a}_{1}\hat{a}_{2}^{\dagger}-e^{-i\phi}\hat{a}_{1}^{\dagger}\hat{a}_{2}))$	$\theta,\phi\in[0,2\pi)$	✓	2
Kerr	$\hat{\Phi}(\kappa)$	$\exp(i\kappa\hat{n}^{2})$	$\kappa\in[0,2\pi)$	$\times$	1

Table S1: Common CV gate operations in photonic circuits. Gaussian gates are generated by Hamiltonians at most quadratic in

(\hat{a},\hat{a}^{\dagger})

; Kerr is a representative non-Gaussian gate. In many programmable photonic processors,

\hat{U}

is implemented as a product of beam splitters and rotations forming an

N

-port interferometer.

Figure S1: EMA suppresses shot-noise fluctuations in the photonic conditioning signal. Feature traces recorded during PQKD training at fixed shot budget compare the raw per-epoch photonic feature (pre-EMA) with the EMA-smoothed feature used by the student. (a) A representative feature coordinate illustrates attenuation of high-frequency sampling jitter under EMA. (b) Windowed value distributions across feature dimensions for the raw and EMA-used signals; the dashed curve indicates the removed residual

(z_{\mathrm{raw}}-z_{\mathrm{EMA}})

. (c) Empirical CDF of the per-dimension variance ratio

\mathrm{Var}(z_{\mathrm{EMA}})/\mathrm{Var}(z_{\mathrm{raw}})

with the expected attenuation level shown for reference.

Knowledge Distillation

Knowledge distillation transfers useful behavior or representations from larger or specialized models into more compact or adaptable learners.

See all 815 papers

Mar 10, 2026

Noise Models Impacts and Mitigation Strategies in Photonic Quantum Machine Learning

Photonic Quantum Machine Learning (PQML) is an emerging method to implement scalable, energy-efficient quantum information processing by combining photonic quantum computing technologies with machine learning techniques. The features of photonic technologies offer several benefits: room-temperature operation; fast (low delay) processing of signals; and the possibility of representing computations in high-dimensional (Hilbert) spaces. This makes photonic technologies a good candidate for the near-term development of quantum devices. However, noise is still a major limiting factor for the performance, reliability, and scalability of PQML implementations. This review provides a detailed and systematic analysis of the sources of noise that will affect PQML implementations. We will present an overview of the principal photonic quantum computer designs and summarize the many different types of quantum machine learning algorithms that have been successfully implemented using photonic quantum computer architectures such as variational quantum circuits, quantum neural networks, and quantum support vector machines. We identify and categorize the primary sources of noise within photonic quantum systems and how these sources of noise behave algorithm-specifically with respect to degrading the accuracy of learning, unstable training, and slower convergence than expected. Additionally, we review traditional and advanced techniques for characterizing noise and provide an extensive survey of strategies for mitigating the effects of noise on learning performance. Finally, we discuss recent advances that demonstrate PQML's capability to operate in real-world settings with realistic noise conditions and future obstacles that will challenge the use of PQML as an effective quantum processing platform.

A. M. A. S. D. Alagiyawanna, Asoka Karunananda

Department of Computational Mathematics, University of Moratuwa, Sri Lanka

Quantum Machine Learning Photonic Computing Quantum Error Mitigation

Ask PDF

Jul 22, 2026

PN-QNN: Harnessing Physical Noise as a Native Regularizer in Photonic Hybrid Quantum Neural Networks

Physical noise in near-term quantum hardware is usually treated as a nuisance to suppress. We ask whether it can instead act as a hardware-native regularizer for photonic hybrid quantum-classical neural networks (PHQCNNs), analogous to noise-injection regularization in classical deep learning. Using Quandela's Perceval simulator and the MerLin framework, we build PHQCNNs for Iris, Digits, and MNIST and inject Perceval's seven-parameter physical noise model directly into training. A genetic algorithm searches the six continuous noise dimensions and 1 boolean parameter to find, per dataset, the configuration maximizing validation accuracy, compared against a noiseless baseline across five seeds. GA-tuned noise yields modest accuracy gains on Iris (+0.82pp) and Digits (+1.45pp), but a clear degradation on MNIST (-1.21pp). Per-parameter sweeps show that no individual noise parameter is consistently beneficial, motivating the joint search, while a second-order loss expansion shows that physical noise induces a Tikhonov-like regularization term whose effect is dataset-dependent. Physical photonic noise can thus act as a free regularizer, but not universally.

Farah Elnakhal, Alberto Marchisio, Nouhaila Innan +2

Science Division, New York University Abu Dhabi, UAE · eBrain Lab, Division of Engineering, New York University Abu Dhabi, PO Box 129188, Abu Dhabi, UAE · Center for Quantum and Topological Systems, NYUAD Research Institute, New York University Abu Dhabi, UAE +1

Hybrid Quantum-Classical ML Photonic Computing Implicit Regularization Quantum Neural Networks

Ask PDF

May 21, 2026

Q-PhotoNAS: Hybrid Quantum Neural Architecture Search Framework on Photonic Devices

Photonic quantum computing is a promising platform for scalable quantum machine learning, but designing effective hybrid architectures remains challenging under hardware and optimization constraints. Existing approaches rely on manually tuned architectures that fail to account for the collaboration between classical preprocessing, phase encoding, and photonic circuit structure, limiting both accuracy and hardware compatibility. In this paper, we propose a neural architecture search framework for hybrid photonic quantum-classical models that combines genetic algorithm-based search with learnable quantum phase encoding to systematically explore the joint design space of classical and quantum components. Our framework encodes 19 hyperparameters across six gene groups and evolves a population of hybrid architectures using group-based crossover, per-gene mutation, and elitism, evaluating each candidate on a short training budget before full retraining of the best found design. We evaluate our framework on two image classification benchmarks, Digits and MNIST, achieving final validation accuracies of 99.44% and 98.78%, respectively, with first-principles execution time estimates on the Quandela Ascella photonic QPU projecting single-image inference at 67 ms (Digits) and 149 ms (MNIST). Our quantum contribution analysis further shows that the photonic layer extracts non-redundant features orthogonal to the classical pathway, providing a measurable accuracy advantage over classical-only baselines. Our results demonstrate that automated architecture search is both practical and impactful for hybrid photonic systems, opening the way for systematic design space exploration of quantum AI on photonic devices.

Farah Elnakhal, Alberto Marchisio, Nouhaila Innan +2

Hybrid Quantum-Classical ML Hardware-Aware NAS Photonic Computing Quantum Architecture Search

Ask PDF

Abstract

Knowledge Distillation

Explore similar work

Noise Models Impacts and Mitigation Strategies in Photonic Quantum Machine Learning

PN-QNN: Harnessing Physical Noise as a Native Regularizer in Photonic Hybrid Quantum Neural Networks

Q-PhotoNAS: Hybrid Quantum Neural Architecture Search Framework on Photonic Devices