A Comprehensive Review of Large Language Models for Nanophotonics: From Surrogate Modeling to Autonomous Design
Authors: Huanshu Zhang, Kegeng Tang, Lei Kang, Sawyer D. Campbell, Zihao Wang, Douglas H. Werner
Organizations: The Pennsylvania State University, Department of Electrical Engineering, University Park, PA 16802, USA · University of Tennessee at Chattanooga, Department of Computer Science and Engineering, Chattanooga, TN 37403, USA
Metasurfaces have revolutionized the development of photonic devices by enabling unprecedented precision in light manipulation. However, their design processes are often constrained by computationally expensive simulations and complex high-dimensional design spaces. Although deep learning has accelerated the design process by serving as a surrogate model, it remains constrained by task-specific architectures and lacks universal reasoning capabilities. This review surveys how Large Language Models (LLMs) are adding semantic interfaces, code generation, and tool orchestration to established numerical nanophotonic workflows. We first outline the development from classical neural networks to transformer-based models and their applications in nanophotonic design. We then review the emergence of LLM-related methods in nanophotonics and organize them into two operational modes: surrogate models that treat structure-spectrum mapping as a language task, and agentic systems that have been demonstrated to generate code, orchestrate selected simulation steps, and support closed-loop optimization. Furthermore, to identify future cross-disciplinary opportunities, we briefly explore applications of LLMs in research fields such as materials science and wireless communications. This review concludes by looking ahead to the next generation of multimodal foundation models with physical perception capabilities. In this vision, artificial intelligence is evolving from passive tools into active collaborators, participating in autonomous scientific discovery.
Figures & tables
Review
Primary scope
41
Broad two-way interaction: classical DNNs for photonic design, imaging, communication, and materials
42
Classical DNNs for metasurface prediction/design and adaptive metadevices
47
Optimization, adjoint methods, discriminative/generative DL, devices, tools, and foundries
48
Classical AI methods across metasurface elements and optical systems
AI across accelerated electromagnetic modeling and inverse design, optical-data characterization, end-to-end imaging, and autonomous metasurface systems
Table 1: Recent Related Reviews
Fig. 1: The paradigm shift in the design of nanophotonic devices.
Fig. 2: Several preliminary and foundational works of DL-enabled nanophotonics. (a) The general workflow of DL-enabled nanophotonics. (b) Forward prediction and inverse design of multilayer-particle scattering. Reprinted with permission from [ 72 ] , Copyright 2018, The American Association for the Advancement of Science. (c) Tandem network for addressing nonuniqueness and data inconsistency of inverse design. Reproduced with permission from [ 73 ] . Copyright 2018 American Chemical Society. (d) A generative network for producing candidate patterns conditioned on target optical responses. Reprinted with permission from [ 78 ] . Copyright 2018, American Chemical Society. (e) MaxwellNet. Reprinted with permission from [ 80 ] . This article is distributed under a CC BY license. (f) Neural operator-based surrogate solver for free-form metasurface inverse design. Reprinted with permission from [ 83 ] , Copyright 2023 American Chemical Society.
Fig. 3: Transformers and LLMs. (a) Transformer architecture. Reproduced from [ 51 ] with permission from Google, which grants reproduction of tables and figures for scholarly works provided that proper attribution is given. (b) Encoder-only model BERT and decoder-only model GPT. (c) Pre-training of LLMs. (d) PEFT methods. Reproduced from [ 105 , 121 , 122 ] , all licensed under CC BY 4.0. (e) ICL and RAG. Reprinted with permission from [ 123 , 124 ] . Ref. [ 123 ] is licensed under CC BY 4.0, [ 124 ] is licensed under CC BY-NC-ND 4.0.
Fig. 4: Transformers for nanophotonics. (a) General workflow of transformers for nanophotonics. (b) Encoder-only transformers for the design of broadband solar metamaterial absorbers. Reproduced from [ 127 ] , licensed under CC BY 4.0. (c) Metaformer for development of metasurface sensors. Reproduced from [ 132 ] , licensed under CC BY 4.0. (d) MetasurfaceViT. Reproduced from [ 138 ] , licensed under CC BY 4.0. (e) OptoGPT. Reproduced from [ 150 ] , licensed under CC BY 4.0. (f) Vision transformer for the design of absorption and phase-cancellation metasurfaces. Reproduced from [ 153 ] . Copyright 2026 IEEE.
Fig. 5: LLM surrogates. (a) Comparison of classical DNN and LLM surrogates. (b) Fine-tuning GPT-3.5 for dielectric metasurface. Reproduced from [ 163 ] , licensed under CC BY 4.0. (c) Fine-tuned Llama-3.1-8B as a surrogate predictor/inverse designer. Reproduced from [ 162 ] , licensed under CC BY 4.0. (d) Fine-tuned Llama-3.1-8B to map 4 × 4 control-point grids to 31-point transmission spectra. Reproduced from [ 161 ] , licensed under CC BY 4.0. (e) Fine-tuned Llama-3-7B to predict and inversely design silicon power beam splitter geometries. Reproduced from [ 167 ] . Copyright 2025 Elsevier. (f) CoSP. Reproduced with permission from [ 168 ] .
Fig. 6: LLM agents. (a) Agent-driven design workflow. (b) MetaChat. Reproduced from [ 164 ] , licensed under CC BY 4.0. (c) An agentic framework for metamaterial inverse design. Reproduced from [ 171 ] . Copyright 2025 American Chemical Society. (d) A human-AI co-design paradigm for photonic-crystal surface-emitting lasers. Reproduced from [ 172 ] , licensed under CC BY-ND 4.0. (e) MCP-enabled LLM for meta-optics inverse design. Reproduced from [ 175 ] , licensed under CC BY-ND 4.0. (f) Electromagnetic metamaterial agent. Reproduced from [ 177 ] , licensed under CC BY-ND 4.0.
Fig. 7: Beyond nanophotonics. (a) LLM for acoustic metasurface design. Reproduced from [ 183 ] , licensed under CC BY 4.0. (b) WirelessAgent. Reproduced with permission from [ 184 ] . (c) CrossMatAgent. Reproduced from [ 186 ] , licensed under CC BY 4.0. (d) Structured information inference for materials science. Reproduced from [ 192 ] , licensed under CC BY-NC-SA 4.0.
Language models have recently been applied to nanophotonic design, but it remains unclear whether they can reliably translate optical objectives into simulation-ready designs, execute electromagnetic analysis, and revise decisions from numerical feedback. We introduce HALO, a physics-aware framework that couples language-model planners with typed design specifications, electromagnetic simulation, diagnostic evaluation, and optional reuse of prior failure trajectories in an iterative design loop. We further introduce HALO-Bench, a 52-task benchmark spanning lab-derived, paper-derived, and open-ended nanophotonic design tasks under a shared evaluation protocol. We compare three planner configurations: a Fixed Structured Workflow, an Autonomous Structured Agent using the same simulation interface, and an Autonomous Coding Agent that directly writes and executes simulation code. The Fixed Structured Workflow is the most token-efficient and exhibits no observed code- or path-level failures, while autonomous coding can achieve higher task success with stronger models at the cost of additional operational failures. We also study reuse of prior failed trajectories. On targeted multi-round tasks, retrieved failure feedback reduces both iterations to first success and total token use. These results clarify the tradeoffs between explicit interfaces, autonomous execution, and reusable design experience in scientific agents.
Yubo Zhang, Jinlin Xiang, Zijun Zhao +3
Department of Electrical & Computer Engineering, University of Washington, Seattle, WA, USA
The rapid development of photonic integrated circuits (PICs) is shifting the design flow from traditional graphical user interface (GUI)-based methods to script-based methods for higher flexibility, portability, and maintainability. However, script-based design introduces new challenges, requiring designers to possess additional proficiency in tool application programming interfaces (APIs) and programming. It also demands greater effort and time because it is inherently less intuitive and more complex than GUI-based methods. As PICs grow in scale and complexity, the productivity gap between design needs and manual scripting capabilities continues to widen. To address this gap, we introduce PICopilot, the first large language model (LLM)-based agentic framework that assists in PIC design via automated design script generation from natural language instructions. PICopilot leverages a multi-agent architecture with a feedback mechanism and a specifically designed retrieval-augmented generation (RAG) pipeline, achieving a high success rate and reliability. Experimental results on a benchmark of diverse PIC scripting tasks demonstrate that PICopilot successfully completes all 48 tasks and outperforms other LLM-based approaches without incurring substantial extra latency or cost, even solving 21 more tasks than the advanced GPT-5 model with a general RAG pipeline.
Xiaohan Jiang, Zeyu Li, Wei Zhang +1
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology · 2Microelectronics Thrust, The Hong Kong University of Science and Technology (Guangzhou)
The design of high performing quantum circuits remains largely dependent on human expertise. We introduce an autonomous agentic framework that employs large language models (LLMs) to conduct iterative quantum circuit designs under explicit design constraints. Our system integrates seven components: Exploration, Generation, Discussion, Validation, Storage, Evaluation, and Review. These components form a closed-loop workflow that combines web-based knowledge acquisition, literature-grounded critique, executable code generation, and experimental feedback. We evaluate the framework on two tasks: quantum feature map construction for quantum machine learning and ansatz generation for variational quantum eigensolver applications in quantum chemistry. In image classification benchmarks, the best generated feature map outperforms representative quantum feature maps and, when scaled to larger qubit counts, surpasses the classical radial basis function kernel. In molecular ground state estimation across seven molecules, the generated ansatz attains competitive accuracy with widely used chemically inspired and hardware-efficient constructions while satisfying the imposed scaling constraints. These results establish LLM driven agentic system as a viable paradigm for automated quantum circuit design and illustrate how AI systems can participate in iterative scientific optimization workflows across scientific domains.
Kenya Sakka, Wataru Mizukami, Kosuke Mitarai
Center for Quantum Information and Quantum Biology, The University of Osaka, 1-2 Machikaneyama, Toyonaka 560-0043, Japan · Graduate School of Engineering Science, The University of Osaka 1-3 Machikaneyama, Toyonaka, Osaka 560-8531, Japan