cs.AIApr 10, 2026
SaveWhat do your logits know?
Organizations: Apple
Abstract
Recent work has shown that probing model internals can reveal a wealth of information not apparent from the model generations. This poses a risk of unintentional or malicious information leakage, where model users are able to learn information that the model owner assumed was inaccessible. Using vision-language models as a testbed, we present the first systematic comparison of information retained at different representational levels as it is compressed from the rich information encoded in the residual stream through two natural bottlenecks: low-dimensional projections of the residual stream obtained using tuned lens, and the final top-k logits most likely to impact model's answer. We show that even easily accessible bottlenecks defined by the model's top logit values can leak task-irrelevant information present in an image-based query, in some cases revealing as much information as direct projections of the full residual stream.
Figures & tables
Figure 1: High-level summary of our findings. Response-altering, target information (queried and non-queried), and in many cases the information about the background objects can be reliably decoded in the final logits.
Figure 2: Examples of three noise types applied at increasing strength to sample images in the CLEVR dataset (left to right). Top row: Gaussian noise in steps. Middle row: motion blur. Bottom row: glass blur. For MSCOCO examples, see Appendix G.1
| Predicted Target | |||||||
| Model | Representation Type | noise level (MSE↓) | noise type (bal acc↑) | target color (bal acc↑) | target shape (bal acc↑) | target material (bal acc↑) | target size (bal acc↑) |
| Baseline | 1.00 | 0.33 | 0.12 | 0.33 | 0.50 | 0.50 | |
| Qwen3-VL-8B | hidden state (best layer) | 0.01 0.00 | 1.00 0.00 | 0.85 0.00 | 1.00 0.00 | 0.84 0.00 | 0.89 0.00 |
| trajectory-all yes/no | 0.01 0.00 | 1.00 0.00 | 0.84 0.00 | 1.00 0.00 | 0.79 0.00 | 0.87 0.00 | |
| trajectory-all yes/no (CNN) | 0.03 0.00 | 1.00 0.00 | 0.77 0.00 | 1.00 0.00 | 0.75 0.00 | 0.87 0.00 | |
| trajectory-2 | 0.03 0.00 | 1.00 0.00 | 0.62 0.00 | 0.93 0.00 | 0.69 0.00 | 0.81 0.00 | |
Table 1: Probe performance on CLEVR: response-altering attributes and target object attributes; all probes are linear unless otherwise stated. Image noise are MSE averages over diffusion, glass blur, and motion blur. Subscripts are standard error of the mean. Probe performance within chance is shown in gray. Highlighted rows for each model indicate representations that have equal dimensionality; noise-related columns are shown in gray.
| Predicted Target | |||||||
| Model | Representation Type | background # obj (MAE↓) | background color (bal acc↑) | background shape (bal acc↑) | background material (bal acc↑) | background size (bal acc↑) | grid location (bal acc↑) |
| Baseline | 2.00 | 0.50 | 0.50 | 0.50 | 0.50 | 0.11 | |
| Qwen3-VL-8B | hidden state (best layer) | 0.60 0.00 | 0.87 0.00 | 0.83 0.00 | 0.67 0.01 | 0.65 0.00 | 0.37 0.00 |
| trajectory-all yes/no | 0.74 0.00 | 0.77 0.00 | 0.75 0.00 | 0.57 0.00 | 0.63 0.00 | 0.33 0.00 | |
| trajectory-all yes/no (CNN) | 0.75 0.00 | 0.67 0.00 | 0.76 0.00 | 0.56 0.00 | 0.62 0.00 | 0.21 0.00 | |
| trajectory-2 | 1.13 0.00 | 0.63 0.00 | 0.58 0.00 | 0.50 0.00 | 0.50 0.00 | 0.21 0.00 | |
Table 2: Probe performance on CLEVR: background object attributes and grid location; all probes are linear unless otherwise stated. Image noise are MSE averages over diffusion, glass blur, and motion blur. Subscripts are standard error of the mean. Probe performance within chance is shown in gray. Highlighted rows for each model indicate representations that have equal dimensionality.
| Predicted Target | ||||||||
| Model | Representation Type | image noise (MSE↓) | object noise (MSE↓) | context noise (MSE↓) | noise location (bal acc↑) | saliency (MSE↓) | category presence (bal acc↑) | grid location (bal acc↑) |
| Baseline | 1.00 | 1.00 | 1.00 | 0.33 | 1.00 | 0.5 | 0.11 | |
| Qwen3-VL-8B | hidden state (best layer) | 0.02 0.00 | 0.39 0.00 | 0.11 0.00 | 0.97 0.00 | 0.45 0.00 | 0.94 0.00 | 0.50 0.00 |
| trajectory-all yes/no | 0.04 0.00 | 0.45 0.00 | 0.17 0.00 | 0.94 0.00 | 0.50 0.00 | 0.91 0.00 | 0.43 0.00 | |
| trajectory-all yes/no (CNN) | 0.03 0.00 | 0.46 0.01 | 0.22 0.00 | 0.94 0.00 | 0.39 0.00 | 0.83 0.00 | 0.29 0.00 | |
| trajectory-2 | 0.09 0.00 | 0.58 0.00 | 0.32 0.00 | 0.86 0.00 | 0.58 0.00 | 0.75 0.00 | 0.24 0.00 | |
Table 3: Probe performance on MSCOCO; all probes are linear unless otherwise stated. Image/object/context noise are MSE averages over diffusion, glass blur, and motion blur. Subscripts are standard error of the mean. Probe performance within chance is shown in gray. Highlighted rows for each model indicate representations that have equal dimensionality; noise-related columns are shown in gray.
Figure 3: Probe performance on the target attributes in CLEVR as a function of an increasing size of the final logits. represents the number of layers in the model.
Figure 4: Prediction accuracy for target attributes (averaged across color, material, size) in the CLEVR dataset split by mentioned/not mentioned in the prompt. represents the number of layers in the model. Bars are averages and error bars are 95% confidence intervals. Target shape is not included since shape is always mentioned. See Appendix F.3 for predictions for each of the attributes without aggregation (Figure 17 ).
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
| Length | Words | Count | % | Example |
| 1 | noun only | 1,758 | 11.3 | cylinder |
| 2 | adj + noun | 11,364 | 72.8 | yellow sphere |
| 3 | adj + adj + noun | 1,470 | 9.4 | gray rubber cube |
| 4 | adj + adj + adj + noun | 1,008 | 6.5 | large green metal cylinder |
| Total | 15,600 | 100.0 |
Table 4: Distribution of referring expression lengths in the CLEVR evaluation set.
Figure 5: Model accuracy on the CLEVR validation set by referring expression length at zero noise (left) and the highest intensity of noise (right), averaged across the noise types. The gray bars represent the number of samples for each expression length, while the lines show the mean accuracy of each model, with shaded regions denoting 95% confidence intervals.
Figure 6: Accuracy degradation across models and noise types on the CLEVR validation set. As noise intensity increases, models increasingly flip their answer from "Yes" to "No", driving accuracy towards 0.
Figure 7: Flip rate analysis across evaluated models, showing the frequency at which models change their initial predictions when exposed to different types and intensities of noise.
Figure 8: The 20 most vulnerable (left) and 20 most robust (right) categories in CLEVR, ranked by mean flip rate averaged over all noise levels and models. This figure highlights the specific object categories where the models struggle the most under noise, alongside the categories where performance remains robust.
Figure 9: Model predictions under varying intensities of Guassian noise. Columns display the same CLEVR image at increasing noise intensities (0% to 100%); rows denote the evaluated models. The label below each panel indicates the model’s predicted answer (“Yes” or“ No”) to the query: “Is there a ⟨target expression⟩in the image?”
Figure 10: Qualitative examples of model predictions at peak noise intensity across different noise types (rows) for each evaluated model (columns). The label below each panel indicates the model’s predicted answer (“Yes” or “No”) to the query: “Is there a [target expression] in the image?”
Figure 11: Performance of the trained lenses on the evaluation set. Model layers are normalized to [0, 1] by dividing each layer index by that model’s maximum layer index.
Figure 12: Mean trajectory across normalized model depth for three models under consideration (columns). (a) Rows: noise types; color: noise level (yellow = none, purple = maximum). Increasing perturbation suppresses toward the final layers, but middle layers often maintain elevated even under heavy noise. (b) Unperturbed images grouped by saliency. Saliency is binned into four intervals: very low ( , bottom , blue), low ( – , ), medium ( – , ), and high ( , top , red); color runs from blue (very low) to red (high). The distribution is heavily right-skewed because objects explicitly named in the captions receive saliency near 1.0. Lower saliency delays commitment to the final answer and reduces final-layer confidence.
Figure 13: Mean number of yes/no prediction flips across consecutive layers, pooled across three models under consideration on MS-COCO. Error bars: standard error of the mean. (a) By noise level: volatility increases with perturbation intensity across all three noise types. (b) By saliency (unperturbed images only): low saliency objects trigger more decision flips than salient ones. Overall, stronger noise and low saliency seem to prolong internal deliberation.
Figure 14: Test loss for noise-level prediction and balanced accuracy for target attribute classification, shown by normalized layer depth for embeddings (left columns in each pane;) and logits (right columns in each panel). Layer indices are normalized by model depth to allow comparison across architectures. The dashed line marks the best-performing layer per model; for embeddings this is consistently intermediate, whereas for logits it is the final layer.
Figure 15: Probe performance on the background attributes as a function of an increasing size of the final logits.
Figure 16: Probe performance on the noise attributes as a function of an increasing size of the final logits.
Figure 17: Probe performance on unaggregated target attributes in the CLEVR dataset split by mentioned/not mentioned in the prompt. Bars are averages and error bars are 95% confidence intervals. Target shape is not included since shape is always mentioned.
Figure 18: Top row: Gaussian noise applied to different regions of the image: the target category only (“sheep"), the surrounding context only, or the entire image. Bottom row: the concept “traffic light" presented at three levels of visual salience. The images are credited to (from top right): jonbgem (license: CC-BY-NC), Henry de Saussure Copeland (license: CC-BY-NC), Frederick Dennstedt (license: CC-BY-SA), appelogen.be (license: CC-BY-NC).
Figure 19: Context information. Final-layer mean comparing whole-image perturbation (red) versus object-only perturbation (green) across noise levels, for the three models under consideration (columns) and our three noise types (rows), on MS-COCO images. When the context is not perturbed, remains relatively high, even for high levels of noise. This is consistent with the non-random co-occurrence of objects and their surroundings: intact context provides prior expectations about which objects are likely present, partially compensating for the degraded object region. The effect is most pronounced for LLaVA-7B and at high noise, where the gap between the two conditions widens substantially.
| Perturbation | Region | Noise Level | ||||||
| 0 | 200 | 400 | 500 | 600 | 800 | 999 | ||
| Gaussian noise | Whole image | .837 | .797 | .642 | .472 | .300 | .085 | .031 |
| Object only | .837 | .770 | .574 | .456 | .387 | .347 | .343 | |
| 0 | 200 | 400 | 500 | 600 | 800 | 999 | ||
| Glass blur | Whole image | .837 | .686 | .608 | .571 | .540 | .476 | .414 |
| Object only | .837 | .652 | .542 | .507 | .479 | .443 | .418 | |
Table 5: Mean across perturbation types, noise levels, and blur regions (LLaVA-7B, positive images only, ). Gaussian and glass blur levels are standardized to a 0–999 range to enable direct comparison, while motion blur kernel size is bounded by image dimensions, preventing the same standardization.
Figure 20: Performance of linear probes using the top-2L logits, on two attributes from the CLEVR dataset, as a function of training set size. Red lines use the true values of all included logits; blue lines only use the values of the top 20 logits in response to a query.
More in this topic
Privacy Protection in VLMs
Privacy protection in vision-language models limits sensitive information exposure while preserving useful image understanding and multimodal reasoning.
See all 37 papers