physics.hist-phNov 18, 2025

DNNs, Dataset Statistics, and Correlation Functions

Authors: Robert W. Batterman, James F. Woodward

Abstract

This paper argues that dataset structure is important in image recognition tasks (among other tasks). Specifically, we focus on the nature and genesis of correlational structure in the actual datasets upon which DNNs are trained. We argue that DNNs are implementing a widespread methodology in condensed matter physics and materials science that focuses on mesoscale correlation structures that live between fundamental atomic/molecular scales and continuum scales. Specifically, we argue that DNNs that are successful in image classification must be discovering high order correlation functions. It is well-known that DNNs successfully generalize in apparent contravention of standard statistical learning theory. We consider the implications of our discussion for this puzzle.

Figures & tables

Explore similar work

CardsList
  1. Sublinearly Structured Deep Neural Networks Achieve Feature Learning Consistency for Compositional Functions

    Jun 22, 2026Sehwan Kim, Yan Sun, Faming LiangNeural Network GeneralizationHierarchical Representation Learning

  2. Memorization in image classification: test set behavior and scaling laws

    Oct 12, 2026Jeevesh Attri, Florentin GuthNeural Network MemorizationNeural Network Training Dynamics

  3. A unified theory of feature learning in RNNs and DNNs

    Feb 17, 2026Jan P. Bauer, Kirsten Fischer, Moritz Helias +1Recurrent Neural NetworksRepresentation Learning