Different AI tools such as ChatGPT, Gemini, and Claude are becoming very popular. Although they are helpful for many day-to-day tasks, they can be used in unexpected ways. For example, the learning objectives of a course may not be achieved if students use these tools to solve their homework problems. This paper proposes a simple method to address this issue in the lazy student model. The method uses homoglyph-based adversarial perturbation to first modify the question without changing the semantic meaning of the question. Then a few characters are perturbed by their homoglyphs. Our experimental result shows the theoretical problems of introductory computer science courses can be effectively perturbed. We also propose an interactive tool to conveniently use our method.
Figures & tables
Figure 1: Despite the fact that several parts of the question are perturbed, as shown in this figure, most popular LLMs, including ChatGPT and Gemini, can answer the question without any difficulty. This observation indicates the robustness of modern LLMs.
Figure 2: A common type of response from popular LLM models, including ChatGPT and Gemini, indicating the biases of the models towards the pretrained dataset. An important adjective “rational” is missing in the prompt and the model easily adds back the missing word since it is familiar with the question from the training dataset. This observation is later used in our method.
Figure 3: Illustrating possible modifications of a question without changing the semantic meaning. The first prompt is the original questions. The variables x and y are introduced in the second prompt without changing the semantic meaning. The third prompt is slightly different: 7x is used instead of x .
Figure 4: Illustrating different homoglyphs of the numeric character “7”. Some homoglyphs are readable by humans and others are not.
Figure 5: Illustrating the unrecognizability of a homoglyph of numeric character “7”. This particular example is a response from Gemini.
Figure 6: Illustrating an example where we added a perturbation of “7” as the coefficient of “x”. The model does not recognize the perturbed character and uses “x” instead of “7x” since it has seen the usage of “x” in this context in the dataset based on which the model is trained on. This particular example is a response from GhatGPT.
Figure 7: Illustrating the distribution of the number of characters across the 164 questions considered.
Figure 8: Illustrating the distribution of the number of attempts to fool the models across the 164 questions considered.
Figure 9: Illustrating the distribution of the number of perturbed characters to fool the models across the 164 questions considered.
Figure 10: Illustrating the unrecognizability of a homoglyph of numeric character “6”. This particular example is a response from Gemini.
Figure 11: Illustrating the observation that the more perturbations can become less effective. Here we perturb four characters, all of them are homoglyphs of “7”. Although the model considered the homoglyph as π , however, it can now take that as a common factor and do the rest of the analysis, see Figure 12 where we can fool the model by just perturbing one character. This particular example is a response from ChatGPT.
Figure 12: Illustrating the observation that less perturbations can become more effective. Here we perturb only one character, it is the homoglyphs of “7”. The model considered the homoglyph as 1 , and solved a different problem. This particular example is a response from ChatGPT.
Figure 13: Illustrating the observation that not always perturbing numeric characters are effective; in this question the numeric character “7” just represents a symbol. To effectively perturb we need to select characters that are involved with arithmetic, see Figure 14 . This particular example is a response from ChatGPT.
Figure 14: Illustrating the observation that perturbing numeric characters are effective. In this question we perturb the numeric character “7” which is involved with arithmetic. This particular example is a response from ChatGPT.
The widespread adoption of generative AI enables students to outsource cognitive effort to increasingly capable assistants, creating an illusion of competence while undermining the independent reasoning that education aims to cultivate. We investigate whether adversarial machine learning can be repurposed to protect educational exercises against such corrosive reliance. Our approach uses multimodal multiple-choice questions whose visual components can be protected with subtle visual perturbations that steer AI solvers toward designated incorrect answers. These responses form a statistical fingerprint: students who blindly copy a solver reproduce the induced answer pattern more frequently than genuine students. We study the feasibility of this paradigm under realistic black-box assistant assumptions using three of the most common state-of-the-art multimodal language models: Anthropic's Claude, Google's Gemini, and OpenAI's ChatGPT. By using accessible surrogate models, we optimize adversarial perturbations that induce consistent response patterns. Those patterns enable principled detection through statistical hypothesis testing. These findings establish both the promise and the limitations of fighting machine-assisted reasoning with the vulnerabilities of the machines themselves.
Large language model (LLM) evaluations on clean benchmarks overstate model robustness: real inputs arrive perturbed, and these meaning-preserving variants routinely move model answers. Studying this systematically requires applying perturbations one at a time, reproducibly, and identically wherever they run, yet existing tools couple them to a benchmarking stack rather than packaging them as a reusable transform layer. We present perturber, a stateless service and Python library that applies exactly one semantics-preserving perturbation per call, drawn from a first-party catalog of 68 perturbations across 11 families. It turns on a single primitive, with composition left to the caller chaining calls, so the service and the library produce identical output by construction. We evaluate the tool itself, its correctness, reproducibility, and cost, rather than any model's robustness, measuring meaning-preservation per perturbation rather than assuming it. The demonstration adds a live playground, an introspectable catalog for benchmark selection, and a one-command reproducible evaluation.
As large language models grow increasingly capable, concerns about their safe deployment have intensified. While numerous alignment strategies aim to restrict harmful behavior, these defenses can still be circumvented through carefully designed adversarial prompts. In this work, we introduce a theoretical framework that formalizes a game between an attacker and a defender. Within this framework, we design a theoretical best-response attack strategy and show that it is closely related to many existing adversarial prompting methods. We further analyze the resulting game, characterize its equilibria, and reveal inherent advantages for the attacker. Drawing on our theoretical analysis, we also derive a provably optimal defense strategy. Empirically, we evaluate a practical instantiation of the theoretically optimal attack and observe stronger performance relative to existing adversarial prompting approaches in diverse settings encompassing different LLMs and benchmarks.
Xinbo Wu, Huan Zhang, Abhishek Umrawal +1
University of Illinois Urbana-Champaign, Urbana, IL, USA · Stony Brook University, Stony Brook, NY, USA.