Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question. Recent advancements in few-shot language models trained on code have demonstrated superior performance in generating these representations compared to traditional unimodal language models, which are trained on downstream tasks. Despite these advancements, existing fine-tuned neural semantic parsers are susceptible to adversarial attacks on natural-language inputs. While it has been established that the robustness of smaller semantic parsers can be enhanced through adversarial training, this approach is not feasible for large language models in real-world scenarios, as it requires both substantial computational resources and expensive human annotation on in-domain semantic parsing data. This paper presents the first empirical study on the adversarial robustness of a large prompt-based language model of code, \codex. Our results demonstrate that the state-of-the-art (SOTA) code-language models are vulnerable to carefully crafted adversarial examples. To address this challenge, we propose methods for improving robustness without the need for significant amounts of labeled data or heavy computational resources.
Figures & tables
Linguistic Phenomenon
Samples ( Strikethrough = Original Text, red = Adversarial Perturbation)
Typo (Word-level)
NL : what can you tell te11 me about the th e population of missouri
Substitution (Word-level)
NL : what can will you tell me about the a population of missouri
Paraphrase (Sent.-level)
NL : what can you tell me about the population of missouri
What information can you provide on Missouri’s population?
Table 1: Examples from Robustness Evaluation Set . We show 3 examples from GeoQuery . These examples are generated with three different perturbations, and they all can successfully change the predictions of Codex .
Category
Pert. Strategy
GeoQuery
Scholar
Pert. Acc.
Std. Acc.
Δ
Pert. Acc.
Std. Acc.
Δ
Word-level
TB
53.85
57.14
-3.29
11.35
12.21
-0.86
RD
50.55
-6.59
10.52
-1.69
RS
37.36
-19.78
8.31
-3.90
CS
42.31
-14.83
8.40
-3.81
CI
38.46
-18.68
8.31
-3.90
Table 2: Results of perturbation accuracy (Pert. Acc.) and standard accuracy (Std. Acc.) of zero-shot performance on GeoQuery and Scholar . The zero-shot prompt only contain the table information and initial semantic parsing instruction. Perturbation accuracy is calculated based on each perturbation method.
Pert. Strategy
5-shot
10-shot
20-shot
30-shot
40-shot
50-shot
TB
63.19
71.98
78.02
81.32
81.30
82.42
RD
59.34
64.29
71.43
71.98
70.33
75.27
RS
52.20
52.75
54.87
59.34
60.99
63.14
CS
54.40
56.04
60.44
63.19
65.93
67.03
CI
51.65
54.95
55.49
57.69
58.79
61.02
RB
44.51
47.80
49.45
50.55
54.23
57.27
Table 3: Few-shot performances on GeoQuery . We conduct {5, 10, 20, 30, 40, 50}-shot learning experiments. Average perturbation accuracy (Avg. Pert. Acc.) is the average score of execution accuracies on different perturbation sets. Average robust accuracy (Avg. Robust Acc.) is the average score of execution accuracies on the test sets perturbed by different perturbation methods.
Pert. Strategy
3-shot
5-shot
10-shot
TB
10.57
20.33
34.29
RD
12.09
25.27
31.43
RS
6.04
17.03
25.08
CS
9.34
18.13
26.03
CI
8.24
14.29
20.63
RB
3.30
8.43
18.10
Table 4: Few-shot performances on Scholar . We conduct {3, 5, 10}-shot learning experiments.
Adv. L. Strategy
GeoQuery
Scholar
Avg. Robust Acc.
Std. Acc.
Avg. Robust Acc.
Std. Acc.
No Adv.
77.28
74.37
53.10
23.08
No Adv. ( × 2)
78.74
79.67
55.97
40.32
TB
77.32
73.62
52.75
34.99
RD
77.40
73.64
53.11
33.65
RS
78.30
74.73
54.88
33.46
Table 5: The results of the average robust accuracy obtained through Adversarial In-context Learning (Adv. L. Strategy) with different types of perturbed few-shot examples. Additionally, we include results of applying the method with only standard examples (No Adv.), as well as with a doubled number of standard examples (No Adv. ( × 2)).
Dataset
Metric
Confidence
Cluster-CWE
Cluster-ED
Cluster-TF-IDF
PPL. Asc
PPL. Desc
Random
GeoQuery
Avg. Robust Acc.
78.77
78.02
82.40
85.10
70.36
50.82
73.22
Avg. Pert. Acc.
69.80
68.93
74.41
77.74
62.11
45.71
66.58
Std. Acc.
74.73
78.41
81.32
85.64
73.14
70.76
74.18
Scholar
Avg. Robust Acc.
55.31
60.24
60.68
62.61
53.97
47.18
55.97
Avg. Pert. Acc.
28.55
29.81
30.28
31.44
22.25
7.47
25.35
Std. Acc.
37.99
41.49
42.19
42.97
35.93
34.89
36.91
Table 6: The performance of standard few-shot in-context learning using various sampling methods on the GeoQuery and Scholar datasets. The average robust accuracy, average perturbation accuracy, and standard accuracy are computed for each sampling method to assess their efficiency in this learning scenario. The Random column comes from a separate random-sampling run and therefore differs from the corresponding settings in Tables 3 and 4 .
Dataset
LC. Metric
Confidence
Cluster-CWE
Cluster-ED
Cluster-TF-IDF
PPL. Asc
PPL. Desc
Random
GeoQuery
TTR ↑
7.68
7.24
8.47
10.26
5.94
3.22
6.44
Yule’s I ↑
68.55
64.37
69.59
71.49
62.94
43.41
58.14
MTLD ↑
12.44
12.19
13.37
15.58
11.32
8.16
10.41
Scholar
TTR ↑
28.18
29.91
31.40
33.11
21.15
14.17
25.67
Yule’s I ↑
198.15
207.11
223.76
262.36
177.37
102.17
193.31
MTLD ↑
15.68
15.63
16.34
19.49
11.94
13.12
14.36
Table 7: Results of the language complexity of standard NLs sampled by different sampling strategies, measured by three lexical diversity (LC.) metrics. For the ease of readability and comparison, we multiply both TTR scores and Yule’s I scores by 100.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Linguistic Phenomenon
Samples ( Strikethrough = Original Text, red = Adversarial Perturbation)
Typo (Word-level)
NL : what can you tell te11 me about the th e population of missouri
Random Deletion (Word-level)
NL : what can you tell me about the population of missouri
Random Swap (Word-level)
NL : what can you tell me about missouri the population of missouri about
Appendix
Table 8: Examples from Robustness Evaluation Set . The adversarial utterances in this Table are generated by applying various perturbation strategies to a single utterance “ what can you tell me about the population of missouri ” sampled from the GeoQuery dataset. All of the generated utterances can successfully alter Codex’s output.
Adversarial robustness of language models studies how to keep their outputs and decisions reliable under manipulative prompts, input changes, and malicious fine-tuning.