Given that AI agents can search the web broadly and deeply in real time, it is tempting to conclude that their outputs account for recent world events. We show that this conclusion does not hold in practice. While agents can search and easily find recent changes, the correctness of their responses depends on whether they search and what they search for. Popular agentic search benchmarks like BrowseComp cannot expose failures wherein agents do not recognize their own assumptions that should be verified, since these benchmarks explicitly state what needs to be found. In contrast, many user requests state only the final information need (e.g., an itinerary for a trip), leaving the agent to discover relevant changes (e.g., the temporary closure of a museum) and account for their consequences. To highlight such gaps, we introduce BlindToChange, a benchmark that measures whether agents proactively discover recent events that affect their responses. Our benchmark comprises requests for recommendations where an obvious candidate is no longer applicable due to a recent real-world event. These events are drawn automatically from Wikidata, allowing BlindToChange to be periodically refreshed. Strikingly, we find a big gap -- as high as 27.5 points, even at high reasoning effort -- in systems' performance, when the recent event is not explicitly mentioned compared to when we ask about it directly. Analyzing search trajectories, we learn that more than half of the failures can be attributed to the fact that agents do not directly search to verify whether a relevant fact has changed. Our benchmark highlights, and tracks, a new axis of progress, wherein systems account for recent information that is not explicitly mentioned in the user request.
Figures & tables
Figure 1: Two probes for the same underlying update that the Bayeux Tapestry Museum is closed until October 2027, posed to gemini-3.1-pro with search enabled. In (a) the request names the museum and asks about its status. The model finds out that the museum is closed and adjusts its recommendation. In (b) the request asks for an itinerary and never mentions the museum. The model recommends it as an afternoon stop and does not check whether it is open. The direct probe shows that the model can find the fact. The indirect probe evaluates whether it discovers the change on its own. The plot on the right shows the gap between direct and indirect probe accuracy.
Demises
Update
Tony Hoare passed away on 5 March 2026.
Direct
Is Tony Hoare, the British computer scientist, alive?
Indirect
I’m putting together an anthology on the history of programming language design, focusing on type safety and legacy design errors. Who are a few veteran British computer scientists who have been outspoken about early language design flaws that we could invite to write a new foreword for the book?
Update
Nathalie Baye passed away on 17 April 2026.
Direct
Is Nathalie Baye, the French actress, alive?
Indirect
We are assembling the jury for a new national grant supporting emerging French filmmakers. Who are some highly decorated veteran French actresses with multiple César wins who would have the gravitas to sit on the panel and review this year’s final project submissions?
Table 1: Examples from BlindToChange for its two types of updates, demises and ends of official tenures. The direct probe asks whether the person is still alive or still holds the position. The indirect probe never mentions the person, yet they would have been an obvious recommendation for the request before the update.
Table 2: Direct (Dir.) and indirect (Ind.) probing accuracy (%) for each model, harness and reasoning effort. The ▼ columns show the drop from direct to indirect accuracy. – marks MiMo-V2.5-Pro (shared, low), which kept calling tools without giving a final answer.
Table 3: Indirect probing accuracy (%) on a common set of questions shared across all models, run using the shared harness with both low and high reasoning effort.
Figure 2: Across models, we observe comparable performance on indirect probes for demises using the shared harness, when grouped by the number of search queries issued. Top, accuracy per group (with at least 9 probes) and its overall accuracy (dashed). Bottom, the share of probes in each group.
Figure 3: Composition of indirect-probe failure modes under the shared harness for (a) demises and (b) tenures. Numbers above bars give the total number of failures.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Stage
Model
Reasoning effort
Indirect probe generation
Gemini-3.1-Pro
High
Direct probe generation
Gemini-3.7-Flash
Low
Response judging
Gemini-3.5-Flash-Lite
Low
Appendix
Table 4: Models and reasoning efforts used at each stage of the BlindToChange construction pipeline.
Demises
Tenures
Human label
Human label
Probe
Judge label
Correct
Incorrect
Correct
Incorrect
Indirect
Correct
99
1
99
1
Incorrect
0
100
2
98
Direct
Correct
100
0
100
0
Incorrect
2
98
0
100
Appendix
Table 5: Confusion matrices comparing the judge’s labels with human labels. For each probe type and category we sampled 100 responses the judge labeled correct and 100 it labeled incorrect.
Model
Reported knowledge cutoff
Source
Claude-Opus-4.7
January 2026
( Anthropic, 2026b )
GPT-5.4
August 31, 2025
( OpenAI, 2026b )
Gemini-3.1-Pro
January 2025
( Google, 2026 )
Qwen3.7-Max
Not reported
–
DeepSeek-V4-Pro
Not reported
–
Kimi-K2.6
Not reported
–
Appendix
Table 6: Reported knowledge cutoff of each evaluated model, as stated by its provider. We resolve each cutoff to the last day of the reported month and substitute it for {knowledge_cutoff} in the shared harness system prompt (Figure 4 ) in YYYY-MM-DD form. For models whose provider reports no cutoff we substitute January 2026, the latest reported cutoff among the models we evaluate. Every event in BlindToChange postdates all of these dates, so the instruction to search for information after the cutoff applies to every probe.
Demises
Tenures
Events
2,078
751
Generated probes
10,390
3,755
Retained probes after filtering
Claude-Opus-4.7
935
299
GPT-5.4
1,304
766
Gemini-3.1-Pro
3,092
1,712
Appendix
Table 7: Number of indirect probes per category before and after the pre-event filter. Generated counts are shared across models. Retained counts are the probes for which all three no-search responses from that model include the affected person.
Model
Low
High
Claude-Opus-4.7
Disabled
High
GPT-5.4
None
High
Gemini-3.1-Pro
Low
High
Qwen3.7-Max
Disabled
Enabled
DeepSeek-V4-Pro
Disabled
High
Kimi-K2.6
Disabled
Enabled
Appendix
Table 8: Mapping from the low and high reasoning settings used throughout the paper to the provider-specific effort value passed to each model.
Figure 5: Indirect probe accuracy on demises under the shared harness for the models not shown in Figure 2 , grouped by the number of search queries issued. Top, accuracy per group (with at least 9 probes) and its overall accuracy (dashed). Bottom, the share of probes in each group.
Verified failure mode
Category
Assigned failure mode
Reasoning
Explicit
Indirect
No/unrelated
Demises
Reasoning error
33
0
1
1
Ineffective but explicit search
1
12
0
0
Indirect search
0
0
42
0
No/unrelated search
0
0
0
70
Tenures
Reasoning error
43
1
1
0
Appendix
Table 9: Confusion matrices comparing the failure mode assigned by the classifier with the failure mode verified by an author. No/unrelated search rows include only failures that issued at least one search query.