Small Change

Research on "small changes" investigates the disproportionate impact of minor alterations in various systems, focusing on how seemingly insignificant modifications can lead to substantial changes in outcomes. Current work explores this phenomenon across diverse fields, including fairness in machine learning (using post-processing algorithms and minimal adjustments to predictions), natural language processing (analyzing the effects of prompt variations on Large Language Model performance and their susceptibility to trivial alterations in tasks), and reinforcement learning (optimizing policies through sparse, interpretable changes). Understanding the sensitivity of complex systems to small changes is crucial for improving model robustness, enhancing fairness, and building more reliable and predictable AI systems.

Papers

August 27, 2024

Post-processing fairness with minimal changes
Federico Di Gennaro, Thibault Laugel, Vincent Grari, Xavier Renard, Marcin Detyniecki
Black Box Post Processing Fairness Research Unbiased Prediction Small Change

June 20, 2024

Dissecting the Ullman Variations with a SCALPEL: Why do LLMs fail at Trivial Alterations to the False Belief Task?
Zhiqiang Pi, Annapurna Vadaparty, Benjamin K. Bergen, Cameron R. Jones
Large Language Model Human Mind Commonsense Reasoning Category Wise Variation Small Change False Belief

January 8, 2024

The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance
Abel Salinas, Fred Morstatter
Large Language Model Complex Prompt Style PROMPT Butterfly Effect Small Change

February 16, 2023

Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks
Tomer Ullman
Large Language Model Artificial Intelligence Complex Reasoning Commonsense Reasoning New Task Intuitive Explanation Small Change

July 13, 2022

Policy Optimization with Sparse Global Contrastive Explanations
Jiayu Yao, Sonali Parbhoo, Weiwei Pan, Finale Doshi-Velez
Reinforcement Learning Markov Decision Process Policy OpTimization Small Change

March 24, 2022

Analyzing Process-Aware Information System Updates Using Digital Twins of Organizations
Gyunam Park, Marco Comuzzi, Wil M. P. van der Aalst
Digital Twin User Digitization Information System Update Organization Name Small Change Process Aware Information System

January 4, 2022

Towards Transferable Unrestricted Adversarial Examples with Minimum Changes
Fangcheng Liu, Chao Zhang, Hongyang Zhang
Adversarial Example Adversarial Perturbation Imperceptible Adversarial Perturbation Unrestricted Adversarial Small Change

November 30, 2021

Minor changes make a difference: a case study on the consistency of UD-based dependency parsers
Dmytro Kalpakchi, Johan Boye
Machine Learning Data Augmentation Training Data Case Study Strong Consistency Qualitative Difference Dependency Parsing Dependency Tree Small Change

November 19, 2021

Small Changes Make Big Differences: Improving Multi-turn Response Selection in Dialogue Systems via Fine-Grained Contrastive Learning
Yuntao Li, Can Xu, Huang Hu, Lei Sha, Yan Zhang, Daxin Jiang
Language Model Dialogue System Qualitative Difference Response Pair Sequence Representation Fine Grained Contrastive Learning Response Selection Small Change Dialogue Response Selection

Small Change

Papers

Post-processing fairness with minimal changes

Dissecting the Ullman Variations with a SCALPEL: Why do LLMs fail at Trivial Alterations to the False Belief Task?

The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance

Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks

Policy Optimization with Sparse Global Contrastive Explanations

Analyzing Process-Aware Information System Updates Using Digital Twins of Organizations

Towards Transferable Unrestricted Adversarial Examples with Minimum Changes

Minor changes make a difference: a case study on the consistency of UD-based dependency parsers

Small Changes Make Big Differences: Improving Multi-turn Response Selection in Dialogue Systems via Fine-Grained Contrastive Learning