Metamorphic Testing

Follow this topic Ask about this topic

Momentum

0 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 13

All topics

Consistency Has a Computable Blind Spot: A Commutation Theory of Label-Free Reliability for Vision-Language Figure Reading
Aug 6, 2026Rasul Khanbayov, Hasan KurbanVLM Evaluation Vision-Language Models
College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
MetaSpace: Metamorphic Testing for Spatial Cognition in Embodied Agents
Jul 26, 2026Gengyang Xu, Dongwei Xiao, Yiteng Peng +1Metamorphic Testing Embodied AI
Hong Kong University of Science and Technology, China
Metamorphic Testing for Clinical ML Models: A Framework Proposal and Pilot Study
Jul 25, 2026Jie JW Wu, Feiyu E, Bo ChenMetamorphic Testing Clinical Outcome Prediction
Michigan Technological University Houghton, MI, USA
Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
Jul 18, 2026Haifeng Li, Mo HaiMetamorphic Testing Formal Verification
School of Information, Central University of Finance and Economics
Domain-Validity-Gated Metamorphic Testing of Scientific ML Surrogates
Jun 16, 2026Meng Li, Xiaohua Yang, Jie Liu +1Metamorphic Testing Scientific ML
School of Computing, University of South China, Hengyang, China · Hunan Engineering Research Center of Software Evaluation and Testing for Intellectual Equipment, Hengyang, China · CNNC Key Laboratory on High Trusted Computing, Hengyang, China
Metamorphic Testing with the Rashomon Set: Explanation Faithfulness in Machine Learning
Jun 4, 2026Helge Spieker, Jørn Eirik Betten, Arnaud GotliebMetamorphic Testing Explainability Evaluation
Simula Research Laboratory Oslo, Norway
Multi-Agent LLM-based Metamorphic Testing for REST APIs
May 27, 2026Shehroz Khan, Abdullah Mughees, Gaadha Sudheerbabu +2Automated Software Testing Metamorphic Testing
Abo Akademi University Turku, Finland
Multi-Agent Specification-based Metamorphic Testing of FMU-Based Simulations
May 24, 2026Ashir Kulshreshtha, Abdullah Mughees, Gaadha Sudheerbabu +4Automated Software Testing Metamorphic Testing
Abo Akademi University, Finland · University of Turku, Finland · Novia University of Applied Sciences, Finland
MetaRA: Metamorphic Robustness Assessment for Multimodal Large Language Model-based Visual Question Answering Systems
May 19, 2026Quanxing Xu, Yuhao Tian, Ling Zhou +4Visual Question Answering Automated Software Testing
School of Computer Science and Engineering, Macau University of Science and Technology, Macao SAR 999078, China · Hubei Key Laboratory of Transportation Internet of Things, School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan 430070, China · State Key Laboratory of Maritime Technology and Safety, Wuhan University of Technology, Wuhan 430063, China +3
A semantic mutation metric for metamorphic relation adequacy in scientific computing programs
May 17, 2026Meng Li, Xiaohua Yang, Jie Liu +1Automated Software Testing Metamorphic Testing
School of Computing, University of South China, Hengyang, China · Hunan Engineering Research Center of Software Evaluation and Testing for Intellectual Equipment, Hengyang, China · CNNC Key Laboratory on High Trusted Computing, Hengyang, China
NOETHER: A Constructive Framework for Metamorphic Pattern Discovery from Operator Algebras
May 17, 2026Meng Li, Xiaohua Yang, Jie Liu +1Automated Software Testing Metamorphic Testing
School of Computing, University of South China, Hengyang, 421001, China · Hunan Engineering Research Center of Software Evaluation and Testing for Intellectual Equipment, Hengyang, 421001, China · CNNC Key Laboratory on High Trusted Computing, Hengyang, 421001, China +2
LGMT: Logic-Grounded Metamorphic Testing for Evaluating the Reasoning Reliability of LLMs
May 12, 2026Zenghui Zhou, Man Li, Xiaoke Fang +3LLM Evaluation Automated Software Testing
School of Automation Science and Electrical Engineering, Beihang University, Xueyuan Road, Haidian District, Beijing, 100191, China
A Metamorphic Testing Approach to Diagnosing Memorization in LLM-Based Program Repair
Apr 23, 2026Milan De Koning, Ali Asgari, Pouria Derakhshanfar +1LLM Evaluation Metamorphic Testing
JetBrains Research, Amsterdam, The Netherlands · Delft University of Technology, Delft, The Netherlands

Aug 6, 2026, cs.LGJ/K move · Enter open · S save

Consistency Has a Computable Blind Spot: A Commutation Theory of Label-Free Reliability for Vision-Language Figure Reading

Rasul Khanbayov, Hasan Kurban

College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar

VLM Evaluation Vision-Language Models Metamorphic Testing

Open paper AskPDF arXiv

Label-free reliability for vision-language models rests on invariance: perturb the input and a faithful reader's answer should not change. This has a known blind spot, a systematic misreading survives the perturbation and gets certified wrong, which we show is computable, not just real: an error is invisible to an edit exactly when the two commute, so the errors a suite cannot reach form its joint centralizer, a set that shrinks as edits are added and can be written down rather than guessed at. We act on the complementary relation, equivariance: edit a figure's data and the correct answer must change by a computable amount. Two matched edits are provably complete for affine reading errors; no suite of swap edits is complete for label permutations, and cyclic relabeling closes most of that gap. We instantiate the theory as the Equivariance-Consistency Score, a label-free, training-free detector, and release REND-EQUIV, pairing matched invariance and equivariance sets over identical data. The predicted ordering holds across three models and a hand-labeled population immune to the one circularity in how it is selected; a second invariance-family method confirms the blind spot belongs to the relation, not to any implementation; and cyclic relabeling delivers its predicted gain on a matched real sample. The same characterization explains a reported inversion of this ordering in the classifier metamorphic-testing literature: detectability is a joint property of the relation and the fault class, never of the relation alone.

Momentum

Related topics

Latest papers 13