cs.LGAug 19, 2026
SaveBeyond receptive fields: sequence-pooled normalization can supply most of a sequence labeler's context
Organizations: Department of Computer Science, University of Alabama at Birmingham, Birmingham, AL 35294, USA
Abstract
A convolutional sequence labeler's receptive field is routinely treated as the extent of the model's usable context: it sets dilation schedules, bounds streaming horizons, and underwrites locality claims. However, we show that this can be false: when a normalization layer computes statistics from the current input along the sequence at inference, those statistics open a sequence-spanning path that bypasses the convolutional receptive field to provide global context. We derive this from the layer's Jacobian (the criterion needs no experiment), and what the path carries has a closed form. On a synthetic labeling process with computable optima, the global summary that a sequence-spanning normalization encodes already supplies almost all of what a larger receptive field would buy where labels come in long runs: a network reaching 9 positions comes within 0.009 of the whole-sequence optimum, against a near-chance bound for its reach. Closing the path, by taking the same statistics per position, multiplies what enlarging the receptive field is worth by up to an order of magnitude on simulated genomes at every difficulty level tested and on real 1000 Genomes haplotypes. The same path also confounds attribution: ablating a trained network's receptive-field-enlarging blocks severs part of the path, overstating their contribution 8.3-16.1-fold relative to retraining from scratch. The substitution of normalization for receptive field fades as labels switch more often. Where labels run long, neither the receptive-field justification nor the ablation is wrong about its numbers, but both credit the wrong component.
Figures & tables
Figure 1: The two paths from input to output. Blue: the receptive field, the bounded set of positions the convolutions reach, which is the quantity architectures report and justify. Orange: the path a normalization layer opens when its statistics are pooled along the sequence: every position contributes to the statistic, and the statistic reaches every position. Each single term is (with the number of positions pooled), small enough to dismiss, but the terms do not cancel one another: they sum to exactly minus the position’s own term , so the normalization path is as large as the convolutional one it shadows (Section 3 ). No receptive-field calculation accounts for it, because it passes through no convolution.
Figure 2: The information landscape. The horizontal axis is , the expected number of label switches in a sequence, so below one means the most likely number of switches in a sequence is zero: one label from end to end. Three of the four lines are references, properties of the generating process computed before any model was trained, with nothing fitted: the whole-sequence optimum (gray, solid), the class-proportion oracle (green, dash-dotted), and a conservative bound for 9 positions of context (blue, dashed). The green class-proportion oracle is the summary a sequence-spanning normalization encodes, to first order. The fourth, the dotted curve, is the closed form derived in Section 3 , also with nothing fitted, which tracks the proportion oracle for and necessarily departs below it. Orange points are trained networks (each for one switch rate): the dilated residual stack of A cut to its first block (receptive field 9 positions), with normalization statistics pooled along the whole sequence rather than taken at each position. The networks track the proportion curve rather than the bound for their own receptive field, falling short of the proportion oracle by 0.009 where labels rarely switch and by at most 0.037 anywhere.
Figure 3: Two routes to a sequence labeler’s context. The two stacks are identical in architecture, parameter count, data and seed; they differ only in the axes their normalization statistics are pooled over. Each convolution reads a fixed window of its input: the first block’s kernel-5 filter is drawn, and reach accumulates over the layers not shown as the dilation doubles. Each normalization layer standardizes its own input activations, and the panels differ in what it standardizes them against: in (a) statistics taken at that position alone, so the layer adds no path between positions; in (b) statistics pooled along the whole activation sequence (faint red lines), so the layer’s output at every position depends on every other. The normalization layer in (b) receives two things, the activation being standardized and a summary of all of them, and the second is what no receptive-field calculation accounts for, supplied by a single layer, without depth. The panel below states the exposure criterion and sorts common layers by it. The drawing is schematic: one block is shown, each block being a normalization-and-convolution unit with a residual connection (not drawn), so the normalization layers sit inside the units that ablation removes (Section 6 ). The pooled contributions are drawn thin because each term is , not because their aggregate is small: by Equation 1 , they sum to minus the diagonal exactly.
Figure 4: (P1) Accuracy against receptive field, with (orange) and without (blue) the sequence-spanning path. Every comparison varies only the axes the normalization statistics are pooled over, holding architecture, parameter count, data and seeds fixed. Reach worth is the accuracy the 227-fold enlargement of the receptive field buys. (a) the dilated stack on a simulated genomic process at 5 population divergence levels, from 0.24 (dark, the easiest) to 0.010 (light, the hardest); the sequence-pooled family is nearly flat while the per-position one climbs to meet it, so their reach worth stands in a ratio of 6.3–8.3 at all 5 divergence (difficulty) levels, with no systematic trend across them. (b) the same stack on real 1000 Genomes haplotypes, European and Gujarati Indian panels ( B ), where that ratio is 12.1. (c) the Conv-TasNet separator, with its recommended gLN (pooled along the sequence), a per-position control, and its causal cLN (pooled over the past only); the gray dotted line is the exact optimum from the whole sequence. cLN and that optimum appear only in (c): cLN is Conv-TasNet’s own layer, and only the synthetic process it runs on makes the optimum computable. Points are means over three seeds, with bands of one standard deviation; (a) omits them for clarity, and in (c) they are narrower than the plotted lines.
| normalization | statistics pooled over | path at inference | reach worth |
|---|---|---|---|
| none | — | no | +0.3740 0.0052 |
| per position | channels | no | +0.3741 0.0052 |
| instance | length | yes | +0.0424 0.0080 |
| group | channels and length | yes | +0.0452 0.0107 |
| batch | batch and length; running at eval | no | +0.3687 0.0046 |
Table 1: (P2) Reach worth (the accuracy gained by enlarging the receptive field) by the axes the normalization statistics span. The third column is the exposure criterion of Section 3 (a path exists exactly when the statistics are computed from the current input, along the sequence, at inference) applied to each layer’s definition, before any of these numbers were measured. BatchNorm is the row where training and inference differ, pooling along the sequence while training but substituting running constants at evaluation. Thus, the criterion predicts no path, and it behaves like the rows without one (none and per position). All rows are at one divergence level, . Means over three seeds.
Figure 5: (P3) Normalization substitutes for reach (the receptive field) less and less as labels switch more often. Reach worth is the accuracy gained by enlarging the receptive field, and the ratio is reach worth without the sequence-spanning path divided by reach worth with it, so a large ratio means the normalization had already supplied what reach would have bought. (a) the two quantities the ratio is built from, reach worth with the path (orange) and without it (blue). Both depend on the switch rate, and over most of the panel in opposite directions: as the summary degrades (more switches) the path supplies less of the context, so reach is worth more to a sequence-pooled model; a per-position model has no path at all, so its context is its receptive field and more switches leave less within it to integrate. (b) their ratio minus one, both axes logarithmic; the dotted reference is , which is a slope of on these axes. Solid lines are the synthetic process, dashed the genomic one, and the bands in (a) are one standard deviation over three seeds.
Figure 6: Two ways of estimating the contribution of a dilated convolutional stack’s long-range blocks (blocks that enlarge the receptive field; Section 4 ): ablating them from a trained network and re-evaluating it with no fine-tuning (orange) against training the same architecture without them from the start (blue). (a) Accuracy lost by removing the blocks, so further right is a larger reported loss, and ablation sits to the right of retraining in every row. The gap between them depends only on which axes the normalization statistics are pooled over: the five rows above the rule share a task, an architecture, seeds and data, and differ only in the normalization layer, while the two below repeat the contrast on real haplotypes. Where the statistics do not span the sequence at inference (including BatchNorm, which spans it only while training), the two measurements nearly agree (there, the blocks’ reach was genuinely needed, and retraining without sequence-spanning statistics does not recover much). Where they do span it, the two differ by 9.1–9.8 times: the difference is context that sequence-spanning statistics can recover and reach alone cannot. (b) The sequence-pooled (GroupNorm) configuration, at a shallower cut that removes only the blocks with dilation : the two estimates disagree twice over, about how much the blocks matter and about whether that changes with task difficulty. Retention, the share of above-chance accuracy that survives removal, is far lower by ablation than by retraining at every difficulty, and it falls steeply as the task hardens where retraining finds it relatively flat. Points in (a) are means over three seeds; the bands in (b) are one standard deviation.
| normalization | statistics pooled over | by ablation | by retraining | ratio |
|---|---|---|---|---|
| none | — | |||
| per position | channels | |||
| batch | batch and length, running at eval | |||
| instance | length | |||
| group | channels and length |
Table 2: Removing the long-range blocks of a dilated convolutional stack (every block that enlarges the receptive field; Section 4 ) from the trained network costs accuracy: column 3, the cost with no retraining. Column 4 retrains the same reduced architecture from scratch, and how much of the drop it recovers depends on the one thing that varies across the rows, whether the normalization statistics give an inference-time path along the sequence. Without one (top three rows), retraining recovers little and the two costs nearly agree, differing by 1.1–1.3 times (the gap adaptation and compensation alone produce): there, the blocks’ reach was genuinely needed. With one (bottom two rows), the costs differ by 9.1–9.8 times: retraining recovers nearly all of the drop, because the network re-obtains the sequence summary through the normalization layers it keeps. The inference-time path along the sequence, then, can supply most of the context the blocks were credited with, without any of their reach; the excess of the ablated cost over the retrained one, beyond the adaptation-and-compensation gap the top rows calibrate, measures the sequence-spanning path severed with the removed blocks (each block carries its own normalization layers), not the blocks themselves. The “by retraining” cost is the full network’s accuracy minus that of a network trained from scratch without the long-range blocks, which is the same difference Table 1 reports as reach worth (the accuracy gained by enlarging the receptive field). All rows are at one divergence level, . The ratios pooled across the difficulty sweep are in Table F.11 . Means over three seeds.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| normalization | ||||||
|---|---|---|---|---|---|---|
| pooled over length | 0.2405 | 0.9631 | 0.9817 | 0.9962 | 0.9982 | 0.9983 |
| 0.0719 | 0.9353 | 0.9478 | 0.9686 | 0.9836 | 0.9877 | |
| 0.0380 | 0.9158 | 0.9250 | 0.9381 | 0.9532 | 0.9610 | |
| 0.0199 | 0.8502 | 0.8649 | 0.8745 | 0.8907 | 0.8969 | |
| 0.0096 | 0.7272 | 0.7339 | 0.7399 | 0.7507 | 0.7592 | |
| per position | 0.2405 | 0.7228 | 0.8786 | 0.9849 | 0.9980 | 0.9983 |
Table F.1: Held-out accuracy of networks retrained with the first blocks, by divergence level and normalization. The columns are receptive fields in positions. Means over three seeds; this is the grid behind Figure 4 a and Table F.2 .
| pooled over length | per position | ratio | |
|---|---|---|---|
| 0.2405 | 7.8 | ||
| 0.0719 | 7.0 | ||
| 0.0380 | 8.3 | ||
| 0.0199 | 6.8 | ||
| 0.0096 | 6.3 |
Table F.2: Reach worth at each divergence level of the simulated genomes, with statistics pooled along the sequence and per position. Their ratio, per position over pooled, does not vary systematically with difficulty.
| process | pooled over length | per position | ratio | ||
|---|---|---|---|---|---|
| genomic | 0.29 | 8.3 | 2.08 | ||
| 0.99 | 3.1 | 2.13 | |||
| 3.00 | 1.7 | 2.08 | |||
| 9.89 | 1.3 | 2.68 | |||
| synthetic | 0.09 | 62.8 | 5.80 | ||
| 0.29 | 15.2 | 4.05 |
Table F.3: Switch-rate dose–response on both simulated processes and on real haplotypes. Both middle columns are the accuracy gained by the 227-fold enlargement of the receptive field; the last column is the constant of the regularity at each density. It is flat within each process and differs between them.
| layer | statistics pooled over | |||||
|---|---|---|---|---|---|---|
| gLN | channels and time | 0.9246 | 0.9244 | 0.9245 | 0.9289 | 0.9407 |
| cLN | channels and time up to | 0.8676 | 0.8705 | 0.8740 | 0.8860 | 0.9186 |
| per position | channels | 0.5423 | 0.5681 | 0.6262 | 0.7314 | 0.8745 |
Table F.4: The Conv-TasNet separator on the synthetic process at 0.27 switches per sequence, against an exact whole-sequence optimum of 0.9522. Its depthwise kernel is 3, so a block adds rather than and the receptive fields differ from the other tables. Means over three seeds.
| switches per sequence | pooled over length | per position | difference | exact optimum | shortfall |
|---|---|---|---|---|---|
| 0.27 | 0.9497 | 0.9472 | +0.0025 | 0.9522 | 0.0025 |
| 0.98 | 0.8956 | 0.8932 | +0.0024 | 0.8993 | 0.0037 |
| 2.98 | 0.8146 | 0.8125 | +0.0021 | 0.8194 | 0.0047 |
| 9.79 | 0.7023 | 0.7000 | +0.0022 | 0.7058 | 0.0035 |
Table F.5: (P4) A transformer, whose attention spans the input sequence at every depth, with sequence-pooled and with per-position normalization. The two are within a few thousandths of each other and of the exact optimum at every switch density: with the context already available there is nothing for the path to supply. The last column is the better of the two normalizations against the optimum, and the range the main text quotes is the widest of these. Means over three seeds.
| switches/sequence | |||||
|---|---|---|---|---|---|
| 0.09 | -0.0006 | -0.0017 | -0.0050 | -0.0124 | -0.0172 |
| 0.29 | -0.0003 | -0.0016 | -0.0049 | -0.0115 | -0.0119 |
| 0.99 | -0.0008 | -0.0016 | -0.0030 | -0.0049 | -0.0013 |
| 2.98 | -0.0010 | -0.0007 | +0.0012 | +0.0019 | -0.0001 |
| 5.99 | -0.0008 | +0.0005 | +0.0031 | +0.0084 | -0.0009 |
| 9.89 | -0.0003 | +0.0010 | +0.0051 | +0.0091 | -0.0002 |
Table F.6: Per-position networks against the local bound for their own receptive field, on the synthetic process: each entry is held-out accuracy minus that bound, so zero is the bound and positive is above it. This is the grid behind every number quoted in Section 5 for this comparison: 35 cells, mean absolute value 0.004, 11 of them positive, the largest 0.0091. Exceedances are expected, since is a blocked bound and therefore conservative, and the entries are finite-sample means (Section 4 ). Adding the corresponding entry of Table F.7 recovers the accuracy itself. Means over three seeds.
| switches | full | proportion oracle | |||||
|---|---|---|---|---|---|---|---|
| 0.09 | 0.5479 | 0.5908 | 0.6741 | 0.8165 | 0.9537 | 0.9759 | 0.9760 |
| 0.29 | 0.5479 | 0.5905 | 0.6738 | 0.8110 | 0.9335 | 0.9504 | 0.9348 |
| 0.99 | 0.5482 | 0.5903 | 0.6708 | 0.7951 | 0.8835 | 0.8986 | 0.8390 |
| 2.98 | 0.5493 | 0.5913 | 0.6660 | 0.7666 | 0.8115 | 0.8194 | 0.7239 |
| 5.99 | 0.5474 | 0.5875 | 0.6543 | 0.7211 | 0.7466 | 0.7508 | 0.6567 |
| 9.89 | 0.5466 | 0.5846 | 0.6424 | 0.6869 | 0.7008 | 0.7031 | 0.6290 |
Table F.7: The reference values on the synthetic process: the local bound at every receptive field it was computed for, and the exactly computed optima with full context and with an oracle given only the sequence’s class proportion. The intermediate columns are here because the main text reads values off them: brackets the U-Net’s receptive field (Section 5 ) and is where the per-position control matches a sequence-pooled model at (Section 5 ). At 0.09 switches the proportion oracle exceeds full context by 0.0001. The two references are not nested and neither bounds the other: the oracle is handed the sequence’s class proportion, a function of the labels, while full context is the best any predictor of the observations can do. Where nearly every sequence carries one label, the oracle is right at every position of those sequences and a predictor reading evidence of strength is not, so the two cross.
| split time | , ours | , benchmark | difference | accuracy |
|---|---|---|---|---|
| 3200 | 0.2405 | 0.2434 | -0.0029 | 0.9983 |
| 1600 | 0.1366 | 0.1376 | -0.0010 | 0.9955 |
| 800 | 0.0719 | 0.0736 | -0.0017 | 0.9877 |
| 400 | 0.0380 | 0.0384 | -0.0004 | 0.9610 |
| 200 | 0.0199 | 0.0200 | -0.0001 | 0.8969 |
| 100 | 0.0096 | 0.0097 | -0.0001 | 0.7592 |
Table F.8: The simulated ancestry process against the benchmark it was ported from ( Tian, 2026 ) , at all eight of its split times. is measured from the simulated panels rather than set, so agreement is a check on the demography, the coalescent draw and the estimator together. Accuracy is the reference architecture’s, over three seeds.
| switches/sequence | depth | receptive field | pooled over length | per position |
|---|---|---|---|---|
| 0.27 | 1 | 22 | 0.9247 | 0.5730 |
| 2 | 48 | 0.9252 | 0.6060 | |
| 3 | 96 | 0.9255 | 0.6493 | |
| 4 | 192 | 0.9276 | 0.7039 | |
| 5 | 384 | 0.9326 | 0.7756 | |
| 6 | 768 | 0.9417 | 0.8482 |
Table F.9: The U-Net depth sweep on the synthetic process, at 6 depths and two switch densities, means over three seeds. “Receptive field” is the convolutional span, measured by autograd rather than derived; with statistics pooled along the sequence the same measurement returns 4096 at every depth, which is the whole sequence. Depth is what sets the receptive field here, and it also sets how many values each normalization statistic is taken over, so this is the one architecture in which the two cannot be varied separately.
| normalization | ablated | retrained | ratio | |
|---|---|---|---|---|
| pooled over length | 0.2405 | 0.3117 | 0.0351 | 8.9 |
| 0.0719 | 0.4273 | 0.0524 | 8.2 | |
| 0.0380 | 0.4099 | 0.0452 | 9.1 | |
| 0.0199 | 0.3774 | 0.0467 | 8.1 | |
| 0.0096 | 0.2289 | 0.0320 | 7.2 | |
| per position | 0.2405 | 0.4085 | 0.2754 | 1.5 |
Table F.11: Accuracy lost by removing the long-range blocks (those that enlarge the receptive field; Section 4 ), per divergence level and normalization: ablating them from the trained network against retraining without them. The pooled figures of Section 6 are means over these rows: 0.3511 against 0.0423 (ratio 8.3) with statistics pooled along the sequence, and a ratio of 1.2 per position. Means over three seeds.
| switches per sequence | computed | error | |
|---|---|---|---|
| 0.09 | 0.4760 | 1.3029 | -63% |
| 0.29 | 0.4348 | 0.7471 | -42% |
| 0.99 | 0.3390 | 0.4001 | -15% |
| 2.98 | 0.2239 | 0.2311 | -3% |
| 5.99 | 0.1567 | 0.1631 | -4% |
| 9.89 | 0.1290 | 0.1269 | +2% |
Table F.12: The derived advantage of an oracle given the sequence’s class proportion, , against the value computed exactly. No quantity is fitted. The approximation requires and (Section 3 ) and fails below because is bounded in .