CircuitTree: Composing Circuits into A Syllable Transcriber
Organizations: 123Univerity of Toronto
Abstract
The goal of circuit discovery is to decompose a monolithic neural network into tiny modular subnetworks which are interpretable in the sense that each does an identifiable subtask. We develop a practical method for composing arbitrarily many circuits into a single model. We propose a greedy, density-scaled composition algorithm that overcomes limitations of previous attempts to compose circuits by naive union, then generalize it to circuits via CircuitTree, a new data structure built on a tree-compose schedule. We compose circuits pruned from HuBERT for classifying individual phonemes into a syllable classifier, a capability none of the 39 phoneme circuits has individually. Chaining CircuitTrees into a hierarchy-of-circuits pipeline, we transcribe open syllables at 64.8% accuracy, within 1.2 points of a 66.0% ceiling that shares no circuitry at all, with 99.5% of circuit components shared.
Figures & tables
| Schedule | Accuracy(%) | Density(%) |
| Unmerged | 82.3 | 1.13 |
| Linear ( ) | 77.9 | 1.11 |
| Balanced ( ) | 81.8 | 1.10 |
| Kruskal merge | 82.2 | 1.10 |
| HuBERT Frozen [ 16 ] | 86.4 | 100 |
| System | Accuracy (%) | Merge Ratio (%) | |
| All | Partial | ||
| Unmerged | 66.0 | 0.0 | 0.0 |
| CircuitTree | 64.8 | 30.3 | 99.5 |
Circuit Discovery
Circuit discovery identifies compact groups of neural-network components that causally implement specific computations or behaviors.
See all 39 papers