cs.AINov 6, 2026

AI Governance Should Prioritize Control and Knowledge Boundaries Over Limiting Intelligence

Authors: Hendrik Luuk

Organizations: Machine Learning and Computational Sciences, Pfizer Research and Development, Pfizer Inc., Cambridge, MA, USA · Department of Physiology, University of Tartu, Tartu, Estonia

Abstract

The rapid proliferation of autonomously acting AI agents makes the governance of agentic systems urgent. We propose the competence circuit, summarized as IKC + KC + C, that decomposes an agent's real-world impact into three pathways: intelligence-mediated discovery, procedural know-how, and direct control. Using the canonical fast-takeoff debate as a stress test, we formalize the conditions for intelligence-mediated fast takeoff as a conjunction jointly sufficient for fast catastrophic supremacy, showing that most rely on rapid knowledge acquisition or control attainment rather than intelligence alone. We operationalize epistemic intelligence as hypothesis-selection efficiency and validate this framework via stochastic simulation on NK fitness landscapes across six scenarios spanning the conjunctive chain. An analytic bound supported by the simulations shows that intelligence-driven speedup is capped by the ratio of competing intelligence levels regardless of task difficulty. The simulations also expose a fast-takeoff dilemma: a large intelligence-driven speedup multiplier materializes only when the hypothesis space is noisy, but the absolute timescale remains constrained by the time required for capability-critical tests, so a large multiplier need not imply rapid progress. Simulations also indicate that institutional chokepoints disproportionately slow down superintelligent agents when mandatory authorization delays are large relative to the negligible time these agents spend on unproductive search. We argue that the dominant safety levers are restricting control and enforcing knowledge boundaries rather than limiting intelligence alone, and this applies to agentic systems at any capability level. Purely digital domains weaken these bottlenecks, strengthening the case for architectures that intentionally "put physics in the loop."

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Active Inference: A method for Phenotyping Agency in AI systems?

    Apr 25, 2026Philip Wilson, Axel Constant, Mahault Albarracin +4Active InferenceAI Governance

  2. The CASE Framework: A Multi-Disciplinary Control Architecture for Governing Enterprise Agentic AI

    Aug 10, 2026Srinivas Telukunta, Georgios Nektarios Lilis, Lucio BaronAI Agent EvaluationAI Agent Governance