Organizations: College of Computer Science and Software Engineering, Shenzhen University · Independent researcher · Institute for Network Sciences and Cyberspace, Tsinghua University · Department of Computer Science and Technology, Tsinghua University
Scaling laws indicate that the generalization error of a large language model (LLM) decreases monotonically with model size once data is sufficient, and the corollary that larger models outperform smaller ones is now widely accepted. We argue that this corollary fails in security: decisions there hinge on anomalous inputs, which are long-tailed (e.g., zero-day attacks) and therefore poorly served by likelihood fitting. Worse still, the capability brought about by scale (e.g., inferring the intended whole from a corrupted part) leads a LLM silently repair the anomaly before any downstream security head ever sees it. We formalize this as a tradeoff between two quantities: the cognitive capability K(W)=I(W;P) that a model holds about the world distribution, and the security-information retention R(W)=I(A;S)I(Z;S) that survives in its intermediate representation. We derive three information-theoretic bounds to connect them: an upper bound on restoration accuracy in terms of K(W), an upper bound on R(W) in terms of the restoration error, and a Fano lower bound on security-task error in terms of R(W). These theorems approximately imply that usable security information is a product of how much security information survives and how well it can be extracted, indicating that optimal usable security information can be achieved at a certain model size. We state this as the condensing law of security information: usable security information is unimodal in model size, so security generalization error is U-shaped and is minimized at a finite N⋆ rather than decreasing without bound. The experiments on various real-world LLMs validate the condensing law, indicating that there is an optimal model size to solve security issues.
Figures & tables
Figure 1: Pipeline of LLM
Figure 2: Cognitive capability and security-information retention versus model size. Cognitive capability is upper-bounded by H(P) . By Theorem 2 , retention decreases affinely with capability, so against model size it is a reflected (and rescaled) copy of the capability curve.
Figure 3: The Condensing Law: usable security information U(N) is the product of the two curves of Fig. 2 and peaks at an intermediate model size N⋆ . Security generalization error is minimized at the same point.
Figure 4: Centroid distance between attacked and normal representations at blocks 8 and 12, with the eight attack conditions kept separate. Every condition falls with model size.
Figure 6: Safety-assessment score against total parameter count for twelve models (Colors distinguish models).
Large language model safeguards decide whether to answer before seeing how an answer will be used. This creates a basic problem for dual-use tasks: the same answer can help an authorized professional or an attacker, while an attacker can imitate a benign request and interaction history. We separate the capability released by the model from the evidence available about downstream use. When that evidence is copyable, we derive the exact worst-case floor on attacker assistance while preserving useful answers. The result yields a safety trilemma: Useful Capability, Reliable Safety, and Open Access cannot coexist. We then show how a trusted credential can complement existing safeguards by adding hard-to-copy information that predicts actual downstream use, and identify the stronger condition needed to eliminate the floor. Evidence from dual-use evaluations, adaptive attacks, and deployed trusted-access programs supports the practical relevance of these conditions.
Pingyu Wu, Lingyao Zhu, Weiming Zhang +1
University of Science and Technology of China · Hefei AiDA Lab · Zhejiang Wanli University
Upweighting high-quality data in LLM pretraining often improves performance, but in datalimited regimes, especially under overtraining, stronger upweighting increases repetition and can degrade performance. However, standard scaling laws do not reliably extrapolate across mixture recipes or under repetitions, making the selection for optimal data recipes at scaling underdetermined. To solve this, we introduce InfoLaw (Information Scaling Laws), a data-aware scaling framework that predicts loss from consumed tokens, model size, data mixture weights, and repetition. The key idea is to model pretraining as information accumulation, where quality controls information density and repetition induces scaledependent diminishing returns. We first collect the model performance after training on datasets that vary in scale, quality distribution, and repetition level. Then we build up the modeling for information so that information accurately predicts those model performance. InfoLaw predicts performance on unseen data recipes and larger scale runs (up to 7B, 425B tokens) with 0.15% mean and 0.96% max absolute error in loss, and it extrapolates reliably across overtraining levels, enabling efficient data-recipe selection under varying compute budgets.
Deployed language model safeguards (safety fine-tuning, filtering, unlearning) vary by principal only outside the model weights: filters are reconfigured, tiers are multiplied, and artefacts are reissued; inside one set of weights every request meets the same model configuration. This motivates us to define capability-gated deployment: per-principal access control inside one set of weights, whose configurations form a lattice - meets accumulate a principal's restrictions and joins pool a coalition's reach. We instantiate it by sparse rank gating over an existing nested-factorisation mechanism, guide profile search with one-pass attribution, and read every result once from a pre-registered held-out split. Security approximately composes: provably exactly at meets under a monotone-elicitation assumption we falsify pointwise. In two lineages the median held-out meet deepens suppression; the one effect surviving correction strengthens it. Utility does not: individually harmless profiles can compose to retention and fluency damage, and no compositional bound exists.
Patrikas Vanagas, Augustas Mačijauskas, Laurynas Lopata
BPTI Vilnius, Lithuania · Independent Researcher Vilnius, Lithuania · askEarth AG Zürich, Switzerland