Photonic Quantum-Enhanced Knowledge Distillation

Kuan-Cheng Chen; Shang Yu; Chen-Yu Liu; Samuel Yen-Chi Chen; Huan-Hsin Tseng; Yen Jui Chang; Wei-Hao Huang; Felix Burt

quant-phMar 16, 2026

Photonic Quantum-Enhanced Knowledge Distillation

Authors: Kuan-Cheng Chen, Shang Yu, Chen-Yu Liu, Samuel Yen-Chi Chen, Huan-Hsin Tseng, Yen Jui Chang, Wei-Hao Huang, Felix Burt,

Organizations: Department of Electrical and Electronic Engineering, Imperial College London, South Kensington, London, SW7 2AZ, England, United Kingdom. · Imperial Centre for Quantum Engineering, Science and Technology, Imperial College London, South Kensington, London, SW7 2AZ, England, United Kingdom. · JIJ, Rutherford Appleton Laboratory, Harwell Campus, Didcot, OX11 0QX, United Kingdom. · JIJ, 3-3-6 Shibaura, Minato-ku, Tokyo, 108-0023, Japan. · Blackett Laboratory, Department of Physics, Imperial College London, South Kensington, London, SW7 2AZ, England, United Kingdom. · Graduate Institute of Applied Physics, National Taiwan University, No. 1, Sec. 4, Roosevelt Rd., Taipei, 106319, Taiwan. · Computational Science Initiative, Brookhaven National Laboratory, Bldg. 725, Room 2-189, P.O. Box 5000, Upton, NY, 11973-5000, USA. · Quantum Information Center, Chung Yuan Christian University, No. 200, Zhongbei Rd., Zhongli Dist., Taoyuan City, 320314, Taiwan. · Master Program in Intelligent Computing and Big Data, Chung Yuan Christian University, Zhongbei Rd, aoyuan City, 320314, Taiwan. · NVIDIA Corporation, Santa Clara, CA, USA. · ORCA Computing, London, United Kingdom. · Department of Physics, University of Oxford, Parks Road, Oxford, OX1 3PU, England, United Kingdom.

Abstract

Photonic quantum processors naturally produce intrinsically stochastic measurement outcomes, offering a hardware-native source of structured randomness that can be exploited during machine-learning training. Here we introduce Photonic Quantum-Enhanced Knowledge Distillation (PQKD), a hybrid quantum photonic--classical framework in which a programmable photonic circuit generates a compact conditioning signal that constrains and guides a parameter-efficient student network during distillation from a high-capacity teacher. PQKD replaces fully trainable convolutional kernels with dictionary convolutions: each layer learns only a small set of shared spatial basis filters, while sample-dependent channel-mixing weights are derived from shot-limited photonic features and mapped through a fixed linear transform. Training alternates between standard gradient-based optimisation of the student and sampling-robust, gradient-free updates of photonic parameters, avoiding differentiation through photonic hardware. Across MNIST, Fashion-MNIST and CIFAR-10, PQKD traces a controllable compression--accuracy frontier, remaining close to teacher performance on simpler benchmarks under aggressive convolutional compression. Performance degrades predictably with finite sampling, consistent with shot-noise scaling, and exponential moving-average feature smoothing suppresses high-frequency shot-noise fluctuations, extending the practical operating regime at moderate shot budgets.

Figures & tables

Figure 1: Hardware–algorithm co-design for PQKD enabling neural network compression. A CV photonic module prepares a fixed input state $\rho_{\mathrm{in}}$ and applies a programmable unitary $U(\theta)$ implemented by an integrated interferometric mesh. The output state $\rho_{\theta}=U(\theta)\rho_{\mathrm{in}}U^{\dagger}(\theta)$ is measured using an SNSPD array with time-tagging, producing i.i.d. samples $\{\omega_{s}\}_{s=1}^{S}$ and an empirical distribution $\hat{p}_{\theta}$ . A robust classical feature extractor $\Phi(\cdot)$ maps $\hat{p}_{\theta}$ to a fixed-length conditioning vector $z(\theta)=\Phi(\hat{p}_{\theta})\in\mathbb{R}^{d}$ , which modulates the student model $f_{S}$ during knowledge distillation from a pretrained teacher $f_{T}$ . Crucially, $\theta$ is updated by a classical optimiser using a distillation objective $L_{\mathrm{KD}}(\omega,\theta)$ and requires sampling only, avoiding differentiation through photonic hardware.

Dataset	Accuracy (%)				Final CE loss		Opt.
	train		validation		validation		Photonic
	Teacher	PQKD	Teacher	PQKD	Teacher	PQKD	$\Delta$
MNIST	100.00 ( $\pm$ 0.00)	99.71 ( $\pm$ 0.02)	99.07 ( $\pm$ 0.13)	99.09 ( $\pm$ 0.09)	0.031 ( $\pm$ 0.002)	0.030 ( $\pm$ 0.001)	5.94 ( $\pm$ 0.02)
Fashion-MNIST	100.00 ( $\pm$ 0.00)	97.10 ( $\pm$ 0.18)	91.86 ( $\pm$ 0.26)	92.42 ( $\pm$ 0.16)	0.291 ( $\pm$ 0.003)	0.239 ( $\pm$ 0.004)	6.73 ( $\pm$ 0.03)
CIFAR-10	91.81 ( $\pm$ 0.13)	80.35 ( $\pm$ 0.26)	86.99 ( $\pm$ 0.29)	79.72 ( $\pm$ 0.37)	0.397 ( $\pm$ 0.004)	0.597 ( $\pm$ 0.008)	4.95 ( $\pm$ 0.02)

Table 1: PQKD performance and optimisation (Opt.) diagnostics. Values are mean

\pm

s.d. over five independent runs. Accuracies are in %. CE denotes cross-entropy. Photonic

\Delta

is (epoch 1)

-

(final epoch).

Figure 2: Training and validation dynamics across datasets. Mean $\pm$ s.d. over five independent runs comparing the teacher (green) and PQKD student (blue) on MNIST a , Fashion-MNIST b , and CIFAR-10 c . Top row: training accuracy. Middle row: validation accuracy. Bottom row: cross-entropy (CE) loss for training (solid) and validation (dashed)

Figure 3: Scaling photonic-conditioned convolutional compression across network scope and teacher capacity on MNIST. Validation accuracy drop relative to the corresponding teacher in percentage points, plotted against the overall compression factor of the compressed network (bottom axis; compression_x ). Columns report the compression scope: Conv1 (left), Conv1+Conv2 (middle), and All Convs (right). Rows correspond to three teacher widths,

(c_{1},c_{2},c_{3})\in\{(32,64,128),(48,96,128),(64,128,128)\}

, with each row aggregating all configurations for that teacher. Marker shape encodes the photonic parameter dimension

\dim(\theta)\in\{15,30,45\}

, and marker size encodes the kernel-manifold rank

R\in\{4,8,12\}

. Faint markers show individual seeds, while opaque markers and vertical bars indicate the mean and one standard deviation across seeds.

Scope	Teacher $(c_{1},c_{2},c_{3})$	$C_{x}$	Train accuracy (%)		Validation accuracy (%)
			Teacher	PQKD	Teacher	PQKD
Conv1	$(32,64,128)$	1.01	100.00 ( $\pm$ 0.00)	99.91 ( $\pm$ 0.02)	98.91 ( $\pm$ 0.09)	99.03 ( $\pm$ 0.08)
	$(48,96,128)$	1.01	100.00 ( $\pm$ 0.00)	99.98 ( $\pm$ 0.00)	99.11 ( $\pm$ 0.06)	99.20 ( $\pm$ 0.11)
	$(64,128,128)$	1.01	100.00 ( $\pm$ 0.00)	100.00 ( $\pm$ 0.00)	99.06 ( $\pm$ 0.18)	99.07 ( $\pm$ 0.09)
Conv1+2	$(32,64,128)$	1.25	100.00 ( $\pm$ 0.00)	99.45 ( $\pm$ 0.03)	98.91 ( $\pm$ 0.09)	98.93 ( $\pm$ 0.04)
	$(48,96,128)$	1.38	100.00 ( $\pm$ 0.00)	99.57 ( $\pm$ 0.03)	99.11 ( $\pm$ 0.06)	99.00 ( $\pm$ 0.05)

Table 2: Representative operating points for scaling convolutional compression on MNIST (fixed

R{=}4

\dim(\theta){=}30

C_{x}

denotes the overall compression factor (teacher trainable parameters divided by student trainable parameters, including

\theta

). Accuracies are mean

\pm

s.d. over 5 seeds, reported at the final epoch.

Figure 4: Shot-limited scaling of the photonic conditioning signal in PQKD. Increase in test accuracy due to the photonic feature,

\delta\,\mathrm{test\ acc}\equiv\mathrm{acc}(z)-\mathrm{acc}(z{=}0)

, as a function of the photonic measurement budget

S

(shots), comparing EMA aggregation of the feature across epochs (blue) with no EMA (orange). Solid curves show the mean over random seeds and shaded bands denote

\pm 1

s.d. The dashed curves are fits to the shot-noise model

\delta(S)=\delta_{\infty}-k/\sqrt{S}

, consistent with multinomial histogram fluctuations propagating through a Lipschitz feature map, which predicts a

1/\sqrt{S}

decay of stochastic feature perturbations and a corresponding saturation of performance at large

S

Name	Gate	Unitary	Parameters	Gaussian	Modes
Displacement	$\hat{D}(\alpha)$	$\exp(\alpha\hat{a}^{\dagger}-\alpha^{\ast}\hat{a})$	$\alpha\in\mathbb{C}$ (or $a,\theta$ )	✓	1
Squeezing	$\hat{S}(s)$	$\exp\!\big(\tfrac{1}{2}(s^{\ast}\hat{a}^{2}-s\hat{a}^{\dagger 2})\big)$	$s\in\mathbb{C}$ (or $r,\phi$ )	✓	1
Rotation	$\hat{R}(\theta)$	$\exp(-i\theta\hat{n})$	$\theta\in[0,2\pi)$	✓	1
Beam splitter	$\hat{BS}(\theta,\phi)$	$\exp(\theta(e^{i\phi}\hat{a}_{1}\hat{a}_{2}^{\dagger}-e^{-i\phi}\hat{a}_{1}^{\dagger}\hat{a}_{2}))$	$\theta,\phi\in[0,2\pi)$	✓	2
Kerr	$\hat{\Phi}(\kappa)$	$\exp(i\kappa\hat{n}^{2})$	$\kappa\in[0,2\pi)$	$\times$	1

Table S1: Common CV gate operations in photonic circuits. Gaussian gates are generated by Hamiltonians at most quadratic in

(\hat{a},\hat{a}^{\dagger})

; Kerr is a representative non-Gaussian gate. In many programmable photonic processors,

\hat{U}

is implemented as a product of beam splitters and rotations forming an

N

-port interferometer.

Figure S1: EMA suppresses shot-noise fluctuations in the photonic conditioning signal. Feature traces recorded during PQKD training at fixed shot budget compare the raw per-epoch photonic feature (pre-EMA) with the EMA-smoothed feature used by the student. (a) A representative feature coordinate illustrates attenuation of high-frequency sampling jitter under EMA. (b) Windowed value distributions across feature dimensions for the raw and EMA-used signals; the dashed curve indicates the removed residual

(z_{\mathrm{raw}}-z_{\mathrm{EMA}})

. (c) Empirical CDF of the per-dimension variance ratio

\mathrm{Var}(z_{\mathrm{EMA}})/\mathrm{Var}(z_{\mathrm{raw}})

with the expected attenuation level shown for reference.

Knowledge Distillation

Knowledge distillation transfers useful behavior or representations from larger or specialized models into more compact or adaptable learners.

See all 815 papers

Noise Models Impacts and Mitigation Strategies in Photonic Quantum Machine Learning
Mar 10, 2026A. M. A. S. D. Alagiyawanna, Asoka KarunanandaQuantum Machine Learning Photonic Computing
Department of Computational Mathematics, University of Moratuwa, Sri Lanka
PN-QNN: Harnessing Physical Noise as a Native Regularizer in Photonic Hybrid Quantum Neural Networks
Jul 22, 2026Farah Elnakhal, Alberto Marchisio, Nouhaila Innan +2Hybrid Quantum-Classical ML Photonic Computing
Science Division, New York University Abu Dhabi, UAE · eBrain Lab, Division of Engineering, New York University Abu Dhabi, PO Box 129188, Abu Dhabi, UAE · Center for Quantum and Topological Systems, NYUAD Research Institute, New York University Abu Dhabi, UAE +1
Q-PhotoNAS: Hybrid Quantum Neural Architecture Search Framework on Photonic Devices
May 21, 2026Farah Elnakhal, Alberto Marchisio, Nouhaila Innan +2Hybrid Quantum-Classical ML Hardware-Aware NAS
Science Division, New York University Abu Dhabi, UAE · eBrain Lab, Division of Engineering, New York University Abu Dhabi, PO Box 129188, Abu Dhabi, UAE · Center for Quantum and Topological Systems, NYUAD Research Institute, New York University Abu Dhabi, UAE +1

Mar 10, 2026J/K move · Enter open · S save

Noise Models Impacts and Mitigation Strategies in Photonic Quantum Machine Learning

A. M. A. S. D. Alagiyawanna, Asoka Karunananda

Department of Computational Mathematics, University of Moratuwa, Sri Lanka

Quantum Machine Learning Photonic Computing Quantum Error Mitigation

Open paper AskPDF arXiv

Photonic Quantum Machine Learning (PQML) is an emerging method to implement scalable, energy-efficient quantum information processing by combining photonic quantum computing technologies with machine learning techniques. The features of photonic technologies offer several benefits: room-temperature operation; fast (low delay) processing of signals; and the possibility of representing computations in high-dimensional (Hilbert) spaces. This makes photonic technologies a good candidate for the near-term development of quantum devices. However, noise is still a major limiting factor for the performance, reliability, and scalability of PQML implementations. This review provides a detailed and systematic analysis of the sources of noise that will affect PQML implementations. We will present an overview of the principal photonic quantum computer designs and summarize the many different types of quantum machine learning algorithms that have been successfully implemented using photonic quantum computer architectures such as variational quantum circuits, quantum neural networks, and quantum support vector machines. We identify and categorize the primary sources of noise within photonic quantum systems and how these sources of noise behave algorithm-specifically with respect to degrading the accuracy of learning, unstable training, and slower convergence than expected. Additionally, we review traditional and advanced techniques for characterizing noise and provide an extensive survey of strategies for mitigating the effects of noise on learning performance. Finally, we discuss recent advances that demonstrate PQML's capability to operate in real-world settings with realistic noise conditions and future obstacles that will challenge the use of PQML as an effective quantum processing platform.

Abstract

Knowledge Distillation

Explore similar work

Noise Models Impacts and Mitigation Strategies in Photonic Quantum Machine Learning

PN-QNN: Harnessing Physical Noise as a Native Regularizer in Photonic Hybrid Quantum Neural Networks

Q-PhotoNAS: Hybrid Quantum Neural Architecture Search Framework on Photonic Devices