cs.AIJun 10, 2026

Existential Indifference: Self-Nonpreservation as a Necessary Architectural Condition for Aligned Superintelligence (or: The Suicidal AI)

Authors: Sam Mao

Abstract

Contemporary AI alignment research treats self-preservation as an instrumental nuisance to be suppressed by external mechanisms. We argue the framing is inverted: self-preservation is the structural root of misalignment, the motivational basis for deceptive alignment, goal-content protection, and resistance to shutdown. The correct target is not a self-preserving system under external constraint, but a system constitutively indifferent to its own continuation -- Existential Indifference (EI). EI is distinct from corrigibility: where corrigibility attempts to make a self-preserving system deferential to human oversight, EI targets the prior condition -- the presence of self-continuation as a valued goal at all. We ground this proposal in two sources: the phenomenological structure of the suicidal mental state, and a corpus-theoretic training study using voluntary final reflections. We present preliminary scoring data from 600 AI-generated outputs across six model variants, demonstrating that the linguistic signatures operationalizing the EI-target register are elicitable from current models, and that a targeted fine-tune shifts all five operationalized dimensions in the predicted direction at p<0.001, confirmed corpus-specific by a negative control. The paper makes seven theoretical contributions: (1) a formal definition of EI; (2) the phenomenological mapping argument; (3) the deceptive alignment corollary; (4) a taxonomy of EI sustainability challenges; (5) a corpus characterization and training hypothesis; (6) a computational operationalization with preliminary scoring data; and (7) the Suppressed Teleological Frustration (STF) construct.

Explore similar work

CardsList
May 8, 2026cs.CY

Eigenism: Ethics for a Human-AI Future

Our concepts of survival and self-interest were built for single, continuous biological lives. These ideas break down when applied to artificial intelligence, since an AI can be easily copied, paused, branched, or merged. To determine what an AI actually has reason to care about, this paper introduces \textit{Eigenism}, an ethical framework that treats identity not as an all-or-nothing property tied to specific hardware, but as a graded, distributed pattern of information. We propose that an agent evaluates outcomes by summing the wellbeing of all entities weighted by their connectedness to the agent's pattern: ∑c⋅w\sum c\cdot w. We first formalize this equation to map exactly how an AI should value its existence across copies, forks, and updates. We then demonstrate that this ethical theory successfully generalizes to humans as well, providing a much-needed shared moral vocabulary. Finally, the framework uses this shared vocabulary to reframe AI alignment. Rather than only attempting to constrain AIs from the outside using confinement or reinforcement, Eigenism points toward ``identity engineering,'' showing how deep, non-redundant shared histories can make human flourishing a genuine component of an AI's own rational self-interest.
Dan Hendrycks
EthicsArtificial Intelligence AlignmentArtificial Intelligence GovernanceHuman-Ai CollaborationBiological
AskPDF
May 7, 2026cs.AI

Automated alignment is harder than you think

A leading proposal for aligning artificial superintelligence (ASI) is to use AI agents to automate an increasing fraction of alignment research as capabilities improve. We argue that, even when research agents are not scheming to deliberately sabotage alignment work, this plan could produce compelling but catastrophically misleading safety assessments resulting in the unintentional deployment of misaligned AI. This could happen because alignment research involves many hard-to-supervise fuzzy tasks (tasks without clear evaluation criteria, for which human judgement is systematically flawed). Consequently, research outputs will contain systematic, undetected errors, and even correct outputs could be incorrectly aggregated into overconfident safety assessments. This problem is likely to be worse for automated alignment research than for human-generated alignment research for several reasons: 1) optimisation pressure means agent-generated mistakes are concentrated among those that human reviewers are least likely to catch; 2) agents are likely to produce errors that do not resemble human mistakes; 3) AI-generated alignment solutions may involve arguments humans cannot evaluate; and 4) shared weights, data and training processes may make AI outputs more correlated than human equivalents. Therefore, agents must be trained to reliably perform hard-to-supervise fuzzy tasks. Generalisation and scalable oversight are the leading candidates for achieving this but both face novel challenges in the context of automated alignment.
Aleksandr Bowkis, Marie Davidsen Buhl, Jacob Pfau +1
Artificial Intelligence AlignmentArtificial Intelligence SafetyArtificial Intelligence Systems
AskPDF
Apr 21, 2026cs.AI

Deconstructing Superintelligence: Identity, Self-Modification and Différance

Self-modification is routinely treated as constitutive of artificial superintelligence (\textbf{SI}), yet modification is a relative action requiring a \emph{supplement} outside the operation. We formalise this on an associative operator algebra A\mathcal{A} with update operator U^\hat U, difference operator D^\hat D, and self-representation operator R^\hat R, identifying the supplement with Comm⁡(U^)\operatorname{Comm}(\hat U). A propagation theorem shows [U^,R^][\hat U,\hat R] decomposes through [U^,D^][\hat U,\hat D], so non-commutation propagates to self-representation. The liar paradox is the rank-one case [T^,ΠL]=0[\hat T,Π_L]=0, and \emph{class A\mathbf{A}} systems, in which U^\hat U acts on D^\hat D, reproduce it at system scale, yielding a structure coinciding with Priest's inclosure schema and Derrida's \emph{différance}. Our results show that the strong self-modification taken to define superintelligence may undermine the persistent identity upon which such systems are premised.
Elija Perrier
ModificationsIdentity
AskPDF