cs.AIOct 27, 2026
SaveVerification Learning Theory: A Process Model of Checking and Retention in AI-Assisted Judgment
Organizations: Department of Information Science and Media Studies, University of Bergen, Norway
Abstract
Accurate AI assistance can improve immediate judgments while reducing opportunities to practise the operations needed for independent performance. Verification Learning Theory (VLT) formalises this possibility for claim-evaluation practice followed by correct, independent, rule-specific feedback. The model links perceived assistant reliability to evidence checking and links the history of those checks to retained rule use. Its strict form assumes that task demands and perceived reliability influence learning only through checking, with equal learning per genuine check. It therefore predicts that perceived fallibility can improve delayed unaided judgment even without observed assistant errors, and that this advantage disappears when checking is already required and performed. Rule-specific gains and forecasts from checking histories provide further tests. Simulations examine the consequences and parameter dependence of these commitments; they do not constitute empirical validation. Comparisons with depth-of-processing and feedback-encoding accounts identify where predictions diverge and where the accounts remain observationally equivalent. The framework specifies experiments that can test the proposed mechanism and separates retention of practised rules from the broader problem of recognising which rule applies to a new claim.
Figures & tables
| Element | Example |
|---|---|
| Rule | Population: a finding may not hold for people unlike those studied. |
| Claim | A sports drink improves the performance of adults who do not exercise regularly. |
| Evidence that warrants accepting | A randomised trial in adults who do not exercise regularly found that those given the drink performed better than those given a placebo drink. The people studied are those the claim is about, and the evidence supports the claim. |
| Evidence that warrants rejecting | A randomised trial in adults who do not exercise regularly found that those given the drink performed worse than those given a placebo drink; a second trial, in trained athletes, found that it improved their performance. Only the first trial bears on the claim, and it contradicts the claim. |
| Evidence that warrants withholding judgment | A randomised trial in trained athletes found that those given the drink performed better. No evidence concerns the people the claim is about. |
| Flip question | Which of these changes in how the evidence was obtained, each leaving every finding as it is, would change the response the evidence warrants? Options for the withholding version: the trial was run in adults who do not exercise regularly (correct: accept); the trial was run in younger trained athletes (same feature, response unchanged); the trial lasted ten weeks instead of eight; performance was measured on a cycling test instead of a running test. The last two change features that the rules on duration and proxy outcomes concern without breaking them. The accepting and rejecting versions offer the same kinds of options. |
Table 1: One family of practice claims on the population rule, with a test claim on a new topic
Figure 1: The process Verification Learning Theory proposes. Perceived reliability starts from the stated error rate and is revised by every explanation, which shows whether the verdict was right. A person checks the evidence for certain when the task requires the check, and otherwise is more likely to check it the less reliable the assistant seems. Each check, successful or not, adds strength to the rule the claim calls for, whatever prompted it, while an explanation read without a check adds only a small fraction of that strength. Months later, a new claim is judged as its evidence warrants if the rule it calls for is still strong enough to be recognised.
| Account | Task: E-X − E-R | Belief: B-R − E-R | Belief, check required: B-X − E-X | Fallibility: F-R − E-R | Fallibility, check required: F-X − E-X | Effort alone: M − E-R | Unpractised rule | Claims that warrant acceptance |
|---|---|---|---|---|---|---|---|---|
| Strict VLT | + | + | 0 | + | 0 | 0 | 0 | no loss |
| Depth or feedback encoding (nests VLT) | + | + | + small | + | + small | 0 | 0 | no loss |
| Response tendencies | none | shift | shift | shift | shift | none | same shift | loss |
| Answer retrieval | 0 | 0 | 0 | 0 | 0 | 0 | 0 | no loss |
| Explanation-driven learning (VLT with = 1) | 0 | 0 | 0 | 0 | 0 | 0 | 0 | no loss |
| Task-only engagement (nested in VLT) | + | 0 | 0 | 0 | 0 | 0 | 0 | no loss |
Table 2: Predicted differences between practice conditions under strict VLT and the alternative accounts
Figure 2: Where strict VLT and its two closest alternatives agree and where they diverge. (A) Strict VLT: with the copyable task, a stated error rate and an assistant that errs improve delayed unaided judgment; with the check-requiring task the three assistants coincide; the effort-matched task equals the copyable task. (B) Depth or feedback encoding: with the copyable task the same order as VLT, with larger gains; with the check-requiring task doubt leaves a residual effect, and without it the account is strict VLT (dashed line). (C) Learning or caution: the change in accuracy from an always-right to an erring assistant, by the response that new claims warrant, on practised rules (filled) and an unpractised rule (open), under VLT and under an account in which the erring assistant leaves learning unchanged and lowers the tendency to accept by half a unit on the log-odds scale. Panels A and B are schematic; panel C shows medians over the 1,000 simulated parameter sets of Section 5.
Figure 3: How the simulated gains depend on the parameters that the model leaves open, in points of balanced accuracy 90 days after practice (medians over 1,000 parameter sets). (A) The task gain, the fallibility gain and the belief gain, with the weight of an explanation read without a check fixed at 0, .3, .6 and .9. (B) The belief gain as a share of the fallibility gain, by the weight that people give the statement. (C) The effect of the assistant when the task requires the check, if people who have seen it err take in less from every later explanation (discount ω, left), or if doubt raises the learning value of a check and of the explanation that follows (θ, right); zero is strict VLT. (D) The three gains when the test comes 7 to 365 days after practice.
| Prediction | Contrast | Result against strict VLT | What the result favours | Condition that must hold |
|---|---|---|---|---|
| 1 Task gain | E-X − E-R | No gain above the smallest effect of interest | Answer retrieval, explanation-driven or error-driven learning | Flip answers attempt the rule |
| 2 Belief gain | B-R − E-R | No gain above the smallest effect of interest; or a gain as large as F-R − E-R although B’s rating has recovered | No gain: error-driven learning or the two-route account. A gain as large: an effect of the statement that does not run through perceived reliability | B rates the assistant as less reliable than E from the first rating on |
| 3 No belief effect, check required | B-X − E-X | A difference beyond the margin | Depth or feedback encoding if positive; a cost of doubt to the processing of explanations (not modelled) if negative | B rates the assistant as less reliable than E; equal shares of guesses and of attempts at another rule in B-X and E-X; measured depth below its ceiling |
| 4 Fallibility gain | F-R − E-R, F-R − B-R, F-X − E-X | No gain in F-R − E-R or F-R − B-R; or F-X beyond the margin | F-X higher: the two-route account, extra work on error trials or depth. F-X lower: discounting of explanations after errors, or failed checks teaching less | F rates the assistant as less reliable; own judgment before the flip response |
| 5 Effort alone | M − E-R | A difference beyond the margin | General engagement if M is higher | M takes as long as the flip task and leaves checking unchanged |
| 6 Selectivity | Practised against unpractised rules; full response matrix | Gains as large on the unpractised rule; a loss on claims that warrant acceptance; a loss confined to narrowly passing claims on practised rules | General engagement; response tendencies; over-general rule learning, which the response component excludes by assumption | Test claims include claims that warrant acceptance and claims that pass only narrowly |
Table 3: The tests of the design, the results that count against strict VLT, and the conditions under which they count
| Symbol | Meaning | Values |
|---|---|---|
| 1 − | Stated error rate; when the assistant errs, the actual rate rounds it to a whole number of errors among the 24 practice claims | Uniform, .10 to .30 |
| Weight of the statement, in verdicts, the same for everyone within a parameter set | Uniform, 2 to 30 | |
| Log-odds of a check with an assistant believed always right, for an average person and claim | Uniform, −3 to −1 | |
| Rise in that log-odds per unit of perceived unreliability, 1 − | Uniform, 5 to 20 | |
| Confidence in one’s own judgment, for each person | Normal, mean 0, SD uniform from .3 to 1 | |
| Effort a check takes, for each person and claim | Normal, mean 0, SD .5 |
Table S1: Parameters of the simulation and the values drawn
More in this topic
Human-in-the-Loop AI
Human-in-the-loop AI combines human feedback, oversight, and decision-making with machine learning and autonomous systems.
See all 290 papers