The Rise and Fall of $G$ in AGI

David C. Krakauer

q-bio.NCApr 10, 2026

The Rise and Fall of $G$ in AGI

Authors: David C. Krakauer

Organizations: Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM. 87501. USA

Abstract

In the psychological literature the term general intelligence' describes correlations between abilities and not simply the number of abilities. This paper connects Spearman's $g$-factor from psychometrics, measuring a positive manifold, to the implicit ``$G$-factor'' in claims about artificial general intelligence (AGI) performance on temporally structured benchmarks. By treating LLM benchmark batteries as cognitive test batteries and model releases as subjects, principal component analysis is applied to a models $\times$ benchmarks $\times$ time matrix spanning 39 models (2019--2025) and 14 benchmarks. Preliminary results confirm a strong positive manifold in which all 28 pairwise correlations positive across 8 benchmarks. By analyzing the spectrum of the benchmark correlation through time, PC1 explains 90\% of variance on a 5-benchmark core battery ($n=19$)) reducing to 77\% by 2024. On a four benchmark battery, PC1 is found to peak at 92\% of the variance between 2023--2024 and reduce to 64\% with the arrival of reasoning-specialized models in 2024. This is coincident with a rotation in the G-factor as models outsource reasoning' to tools. The analysis of partial correlation matrices through time provides evidence for the evolution of specialization beneath the positive manifold of general intelligence (AI-hedgehog) encompassing diverse high dimensional problem solving systems (AI-foxes). In strictly psychometric terms, AI models exhibit general intelligence suppressing specialized intelligences. LLMs invert the ideal of substituting complicated models with parsimonious mechanisms, a `Ptolemaic Succession' of theories, with architectures of increasing hierarchical complication and capability.

Figures & tables

Figure 1: The score matrix $\mathbf{X}$ . A representative subset of 21 models (rows, ordered by release date) and 14 benchmarks (columns, grouped by cognitive domain). Cell color encodes score intensity (0–100%); dashes mark missing evaluations. The matrix exhibits two key structural features: (i) a gradient from low scores (upper-left) to high scores (lower-right), reflecting the correlated improvement that the PCA must dissect; and (ii) a shifting pattern of sparsity, as early benchmarks (HellaSwag, ARC, WinoGrande) are not evaluated on late models, while late benchmarks (GPQA, MMLU-Pro, SWE-bench) do not exist for early models. Horizontal lines mark algorithmic-epoch boundaries; epoch labels are shown on the right.

Benchmark	$n$	AIC ${}_{\text{L}}$	AIC ${}_{\text{G}}$	AIC ${}_{\text{Lin}}$	Best	$L$
MMLU	7	25.9	26.1	27.0	Logistic	94%
GSM8K	10	44.3	45.0	43.5	Linear	—
MATH	9	43.5	44.8	36.0	Linear	—
HumanEval	10	35.8	34.7	37.0	Gompertz	104%
GPQA Diamond	8	16.7	16.8	29.6	Logistic	92%

Table 1: Growth model fits to the frontier envelope. AIC values for logistic (L), Gompertz (G), and linear (Lin) models fitted to the running-maximum score trajectory. Bold indicates the selected model.

L

denotes the estimated asymptote for sigmoidal fits;

n

is the number of frontier-envelope points.

Figure 2: The phenomenon to be dissected: benchmark performance rising across all tasks simultaneously. Bold markers show the running-maximum score (frontier envelope) for each benchmark; faded markers show all individual model scores. Curves are the best-fit growth model selected by AIC (Table 1 ). MMLU is best fit by a logistic with asymptote $L=94\%$ ; GPQA Diamond by a logistic with $L=92\%$ ; HumanEval by a Gompertz with $L=104\%$ (no saturation yet). The shaded region marks $>90\%$ scores where discriminability is lost. The correlated rise across all five benchmarks, despite their testing different cognitive demands, is the positive manifold in its raw form.

Figure 3: The positive manifold in LLM benchmarks. Pairwise Pearson correlations across six major benchmarks, computed using pairwise-complete observations (minimum $n=5$ pairs per cell). All 15 off-diagonal entries are positive, satisfying the Spearman criterion for a general factor. Note the near-zero correlation between HumanEval and MMLU-Pro ( $r=0.04$ ), suggesting that code generation and hard general knowledge are nearly independent.

Component	$\lambda$	Variance Explained	Cumulative
PC1 ( $G$ )	4.50	90.0%	90.0%
PC2	0.35	6.6%	96.6%
PC3	0.10	1.9%	98.5%
PC4	0.05	0.9%	99.4%
PC5	0.03	0.6%	100.0%

Table 2: Eigenvalue decomposition of the 5-benchmark core battery. PCA on standardized scores for 19 models with complete data across MMLU, GSM8K, MATH, HumanEval, and GPQA Diamond. Only PC1 exceeds the Kaiser criterion (

\lambda>1

Figure 4: Factor loading plot for the 5-benchmark core battery. Arrows show each benchmark’s loading on PC1 ( $G$ -factor, 90% variance) and PC2 (7% variance). All benchmarks load positively on PC1, confirming a general factor. PC2 separates an execution/fluency pole (GSM8K, HumanEval—positive PC2) from a reasoning pole (MATH, GPQA—negative PC2). MMLU is near the origin on PC2, contributing primarily to $G$ rather than to the residual structure.

Figure 5: Normalized $G$ : LLM general intelligence factor over time. Each point represents a model’s projection onto PC1 of the 5-benchmark battery (MMLU, GSM8K, MATH, HumanEval, GPQA Diamond), rescaled to a 0–100 range where 0 is the lowest-scoring model (Llama 2 70B Chat) and 100 is the highest (o1-preview). PC1 captures 90% of total variance across the battery. The 19 models with complete data on all five benchmarks are shown, colored by organization. The normalization makes $G$ directly interpretable as a percentile-like score: a model at $G=50$ is halfway between the least and most capable systems in the sample.

Epoch	$n$	Mean $r$	$\rho_{1}$	$\lambda_{2}$
I (2019–2023)	3	0.72	79%	1.24
II (2023–2024.03)	8	0.90	92%	0.28
III (2024.04–2024.09)	7	0.73	80%	0.49
IV (2024.09–2025)	4	0.43	64%	1.88

Table 3: Factor structure across algorithmic epochs. PCA on a 4-benchmark battery (MMLU, GSM8K, MATH, HumanEval) within each epoch.

\rho_{1}

is the variance explained by PC1;

\lambda_{2}

is the second eigenvalue. The Kaiser criterion (

\lambda_{2}>1

) is exceeded only in Epochs I and IV, suggestive of a two-factor structure but with insufficient data to statistically validate.

Figure 6: Scree plots across algorithmic epochs. Eigenvalue decomposition of the 4-benchmark battery (MMLU, GSM8K, MATH, HumanEval) computed within each epoch. The red dashed line marks the Kaiser criterion ( $\lambda=1$ ). During Epoch II (2023–2024.03), a single dominant factor captures 92% of variance and no second eigenvalue approaches 1.0. In Epoch IV (2024.09+), a second eigenvalue (1.88) increases alongside the creation of inference-time reasoning models (o1, DeepSeek R1).

Figure 7: Expanding-window tracking changing $G$ . Models are added in chronological order of release date; at each step, PCA is computed on the 4-benchmark battery. Top: $\rho_{1}$ , the fraction of total variance captured by PC1 ( $G$ ). The 90% threshold (red dotted line) is exceeded throughout, with a peak at 95.5% around the Claude 3 Opus release (early 2024). Bottom: All four normalized variance fractions $\rho_{1},\rho_{2},\rho_{3},\rho_{4}$ plotted on the same scale. $G$ (blue) dominates throughout; $\rho_{2}$ (red) is the only component that shows any temporal dynamics, growing modestly as post-2024.09 models enter the window. The dashed gray line at 25% marks the uniform distribution ( $\rho_{k}=1/K$ , no factor structure).

Figure 8: Effective dimensionality of the LLM benchmark space through time. Expanding-window analysis on two benchmark batteries. Top: Number of principal components required for 99% cumulative variance. On the 4-benchmark battery (blue circles), dimensionality stabilizes at 3 of 4—near-maximal compression. On the 5-benchmark battery (purple squares, adding GPQA Diamond), dimensionality rises from 3 to 5 as post-2024.09 models enter the window. Bottom: Participation ratio $d_{\text{eff}}$ , a continuous measure ranging from 1 (perfect single-factor structure) to $K$ (no structure). The 4-benchmark battery remains near $d_{\text{eff}}\approx 1.1$ throughout—a near-perfect $G$ . The 5-benchmark battery rises from 1.3 to 1.9, suggestive of a second factor associated with inference-time reasoning. Vertical dotted lines mark algorithmic-epoch boundaries.

Figure 9: Eigenvalue spectrum through time: cumulative variance partitioned by principal component. Each colored band shows one component’s marginal contribution to cumulative variance in the expanding-window PCA; the top of each band is the cumulative variance through that component. (a) 4-benchmark battery: PC1 ( $G$ , blue) accounts for 92–95% throughout; PC1 $+$ PC2 exceed 97% at every time point. The spectrum is effectively one-dimensional—a near-perfect general factor. (b) 5-benchmark battery (adding GPQA Diamond): the spectrum visibly unfolds as post-2024.09 models enter the window. With the expanded sample ( $n=19$ ), for the 5-benchmark spectrum PC1 remains near 90% throughout.

Figure 10: Test (i): CUSUM change-point analysis on eigenvalue diagnostics. (a) Variance explained by PC1 ( $\rho_{1}$ ) in the expanding-window 4-benchmark PCA, showing a peak around the Claude 3 Opus/Gemini Ultra releases (early 2024) followed by decline. (b) CUSUM statistic on $\rho_{1}$ , with maximum absolute deviation marked; $p=0.004$ by permutation test, confirming a significant structural break. (c) Dominance ratio $\delta=\lambda_{1}/\lambda_{2}$ , showing a sharp jump from $\sim$ 12 to $\sim$ 28 when the GPT-4-era models enter, followed by gradual decline. (d) CUSUM on $\delta$ ; $p<0.001$ , the strongest signal. Vertical dotted lines mark epoch boundaries. Both CUSUM statistics identify the early-2024 transition as a significant change point in the factor structure.

Figure 11: Test (ii): Eigenvector alignment. (a) Angular displacement $\theta$ between consecutive first eigenvectors as each model enters the expanding window. Blue bars: 4-benchmark battery (max $\theta=0.57\textdegree$ , near-perfect stability). Purple bars: 5-benchmark battery (max $\theta=6.4\textdegree$ at DeepSeek V3 entry). The 5-benchmark battery reveals rotations invisible to the 4-benchmark analysis because GPQA Diamond introduces a dimension along which post-2024.09 models diverge from pre-2024.09 models. (b) Individual benchmark loadings $|v_{1j}|$ on $G$ over time (4-benchmark battery), confirming structural invariance: all four loadings remain within a 0.04-unit band across the entire timeline.

Figure 12: Partial correlation structure after removing $G$ . (a) Raw correlation matrix (5-benchmark battery, $n=19$ models with complete data): all 10 off-diagonal correlations positive. (b) Partial correlation matrix after projecting out PC1: 7 of 10 correlations are now negative, with mean $r_{\text{resid}}=-0.24$ . The positive manifold is entirely attributable to $G$ . (c) Revealed group factor structure derived from the residual correlations. Group I (Reasoning: MATH, GPQA) and Group II (Execution/Fluency: GSM8K, HumanEval) show positive within-group and negative between-group residual correlations; MMLU is isolated, with near-zero or negative residual correlations with all other benchmarks. Green values indicate positive residual $r$ ; red values indicate negative.

Figure 13: Epoch-specific partial correlations. Partial correlation matrices after removing each epoch’s own PC1 from the 4-benchmark battery. (a) Epoch II (2023.03–2024.03, $n=8$ , $\rho_{1}=92\%$ ), (b) Epoch III (2024.04–2024.09, $n=7$ , $\rho_{1}=80\%$ ). Additional panels show the 5-benchmark and global analyses. Of 6 benchmark pairs in the 4-benchmark analysis, 4 maintain the same sign across Epochs II and III; only the two weakest pairs flip sign.

Figure 14: Trajectory through $G$ -space . Four points are plotted in a space defined by mean benchmark performance, $\rho_{1}$ (variance explained by $G$ ), and effective dimensionality $d_{\text{eff}}$ . Point 1: Epoch II (within-epoch, 5-benchmark battery). Point 2: Epoch III (within-epoch). Point 3: all models, raw PCA. Point 4: all models, detrended PCA. Solid arrow: temporal trajectory (1 $\to$ 2). Dashed purple arrow: detrending correction (3 $\to$ 4). Dashed green line: Spearman prediction (performance increases, $\rho_{1}$ and $d_{\text{eff}}$ fixed). Sphere colour encodes the number of negative partial correlations out of 10 benchmark pairs. Projection lines show coordinate pairs on the floor (performance, $\rho_{1}$ ) and back wall (performance, $d_{\text{eff}}$ ).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Analysis	$n$	$K$	$\lambda_{1}^{\text{obs}}$	$\lambda_{1}^{95\text{th}}$	$p(\lambda_{1})$	PC1 retained?
4-bench, all models	22	4	3.73	1.81	$<.0001$	Yes ( $p<.01$ )
5-bench, all models	19	5	4.50	2.06	$<.0001$	Yes ( $p<.01$ )
6-bench, all models	9	6	3.73	2.91	$.0002$	Yes ( $p<.01$ )
Epoch II, 4-bench	8	4	3.66	2.44	$<.0001$	Yes ( $p<.01$ )
Epoch II, 5-bench	7	5	4.57	2.89	$<.0001$	Yes ( $p<.01$ )
Epoch III, 4-bench	7	4	3.21	2.56	$.0005$	Yes ( $p<.01$ )

Appendix

Table 4: Permutation Analysis For each analysis, the observed first and second eigenvalues are compared against the 95th and 99th percentiles of the permutation null distribution (

B=10{,}000

). A factor is retained if

\lambda_{\text{obs}}>\lambda_{95}

. Exact

p

-values are the fraction of permuted eigenvalues exceeding the observed value.

Artificial General Intelligence

Artificial General Intelligence studies AI systems that can learn, reason, and act across diverse tasks and environments.

See all 23 papers

Large-scale factor analysis shows machine intelligence is only partially interpretable
Sep 29, 2026Faiz Ghifari Haznitrama, Afrizal Hasbi Azizy, Faeyza Rishad ArdiLLM Evaluation
School of Computing, KAIST · Independent Researcher
The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning
Mar 9, 2026Sahar Vahdati, Andrei Aioanei, Haridhra Suresh +1Abstract Reasoning ARC-AGI
TIB - Leibniz Information Centre for Science and Technology · L3S Research Center, Leibniz University of Hannover · Dresden University of Technology +1
AGC-Bench: Measuring Artificial General Creativity
Jul 1, 2026Roger Beaty, Vijeta Deshpande, Clin K. Y. Lai +9Creativity Assessment LLM Evaluation
1Pennsylvania State University · University of Massachusetts Lowell · University of Amsterdam +2

Sep 29, 2026J/K move · Enter open · S save

Large-scale factor analysis shows machine intelligence is only partially interpretable

Faiz Ghifari Haznitrama, Afrizal Hasbi Azizy, Faeyza Rishad Ardi

School of Computing, KAIST · Independent Researcher

LLM Evaluation

Open paper AskPDF arXiv

A common assumption in language model development is that cognitive abilities are organized around a general, domain-free intelligence factor, like fluid intelligence in humans. This assumption is rarely tested directly, and prior attempts have done so only at a much smaller scale. We take a latent variable approach to intelligence in language models, similar to how psychometricians study psychological constructs. Performance in every specific problem set is influenced by a domain-specific and a domain-agnostic latent factor. Using factor analysis as a dimension-reduction technique, we analyzed 13,251 published evaluation scores covering 1,618 language models across 456 different text-only benchmarks. Due to the super-sparse nature of the dataset, we triangulate our analysis across different data densifiers and imputation methods. A robust pattern across different modes of bias is that 1. A general intelligence factor accounts for 70.8% of variance in model performance at our most generous estimate, and far less than that in most of our solutions, 2. Content-similar benchmarks do not necessarily cluster together, and 3. The

g

factor is not dominated by any common theme, and there is a lack of evidence that it is well-proxied by standard "intelligence" benchmarks. Our findings go against current endeavors of defining, identifying, and targeting general intelligence as a tangible construct in language model development. This leaves the strategy of targeting a single conceptual ability without support, since the first-order abilities it would have to reach are often partially idiosyncratic and not identifiable in practice.

Abstract

Artificial General Intelligence

Explore similar work

Large-scale factor analysis shows machine intelligence is only partially interpretable

The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning

AGC-Bench: Measuring Artificial General Creativity