NoLBERT-Calls: Reading Earnings Calls Without Hindsight
Organizations: University of California, Berkeley Berkeley, California, USA
Abstract
Language models are now a standard tool for measuring what managers say on earnings calls, but the encoders behind these measures were usually pretrained on text from the same years a backtest covers. Holding out a test period does not remove that hindsight, because it sits in the features. We adapt NoLBERT, an encoder pretrained only on text published between 1976 and 1995, to earnings calls without letting it read anything dated after 31 December 2023. The calls supply the training signal themselves: every analyst question is followed by management's answer, and the model learns to pick that answer out from the other answers of the same call. We test on every US call from January 2025 to October 2026 (20,323 calls, 254,575 questions) and make every modeling choice on calls from 2024. The adapted model, NoLBERT-Calls-2023, ranks the true answer first for 78.0% of questions (95% interval 77.8-78.2; chance 8.0%), against 26.3% before adaptation. It is 1.0 point ahead of a call-adapted ChronoBERT, 3.4 points ahead of a call-adapted FinBERT, and 18.7 and 19.8 points ahead of the best off-the-shelf retriever and BM25. A year-by-year test over 2013-2026 finds no look-ahead premium: relative to a model adapted only through 2012, the released model does no better on the years it was trained on than on years it never saw. The price of the cutoff is compute: under one shared 150-minute recipe NoLBERT adapts least well of four encoders, and its lead comes from a longer, staged adaptation of about 13 GPU-hours.
Figures & tables
| Protocol | Training calls | Validation | Test |
| Main (cutoff 2023) | US calls up to 31 Dec 2023: 196,569 calls, 3,052,636 exchanges, 785,924 passages | 300 US calls from 2024 (3,763 questions) | Every US call from Jan 2025 to Oct 2026: 20,323 calls, 254,575 questions |
| Time (cutoff 2012) | US calls up to 31 Dec 2012: 59,523 calls, 1,067,124 exchanges | 300 US calls from 2013–2015 (5,398 questions) | 500 US calls a year, 2013–2024 (6,268 to 8,864 questions a year), then the main test |
| Group | Readers | Training text and its dates |
| Word overlap | TF-IDF, BM25 | No training. Word weights are fitted on calls up to the cutoff. |
| Pretrained encoders, zero-shot | NoLBERT, ChronoBERT-2012, ChronoBERT-2023, FinBERT, DeBERTa-v3-base | NoLBERT: text of 1976–1995. ChronoBERT: text up to each year end. FinBERT: pretraining includes calls of 2004–2019. DeBERTa-v3: no date control. |
| Off-the-shelf retrievers | all-mpnet-base-v2, BGE-base-en-v1.5, E5-base-v2 | Web-scale training data with no date control. |
| Same recipe, calls up to 2023 | NoLBERT-, ChronoBERT-, FinBERT- and DeBERTa-Calls-2023 | US calls up to 2023; one 150-minute run on one T4 each. |
| Same recipe, calls up to 2012 | NoLBERT-Calls-2012, ChronoBERT-Calls-2012 | US calls up to 2012; one 150-minute run on one T4 each. |
| Our staged model | NoLBERT-Calls-2012, stage 1 | Calls from all countries up to 2012; 480 minutes on one RTX 3060. |
| Reader | Top-1 [95% interval] | + word overlap | Top-3 | MRR | to released [95% interval] |
| Adapted on calls up to 2023 | |||||
| NoLBERT-Calls-2023 (released) | 78.0 [77.8, 78.2] | 79.0 | 94.0 | 86.4 | — |
| ChronoBERT-Calls-2023 at 256 tokens | 77.0 [76.8, 77.2] | 77.9 | 93.5 | 85.7 | 1.0 [0.9, 1.2] |
| FinBERT-Calls-2023 | 74.7 [74.5, 74.9] | 77.1 | 92.8 | 84.2 | 3.4 [3.2, 3.5] |
| ChronoBERT-Calls-2023 | 72.3 [72.1, 72.6] | 74.6 | 91.2 | 82.4 | 5.7 [5.5, 5.9] |
| DeBERTa-Calls-2023 | 68.2 [68.0, 68.4] | 74.4 | 89.6 | 79.7 | 9.8 [9.6, 10.0] |
| Lead over NoLBERT-Calls-2012 | Premium [95% interval] | ||||
| Reader | 2013–23 | 2021–23 | 2025–26 | 2013–23 vs 2025–26 | 2021–23 vs 2025–26 |
| Adapted on calls up to 2023 | |||||
| NoLBERT-Calls-2023 (released) | +12.0 | +13.3 | +12.9 | 0.9 [ 1.2, 0.5] | +0.3 [ 0.3, +1.0] |
| FinBERT-Calls-2023 | +10.0 | +10.7 | +9.6 | +0.4 [+0.1, +0.8] | +1.1 [+0.5, +1.8] |
| ChronoBERT-Calls-2023 | +7.1 | +7.8 | +7.2 | 0.1 [ 0.4, +0.3] | +0.5 [ 0.1, +1.1] |
| DeBERTa-Calls-2023 | +3.1 | +4.1 | +3.1 | +0.0 [ 0.4, +0.5] | +1.0 [+0.2, +1.8] |
| Passage matching | Masked terms | |||||
| Reader | Top-1 | Top-10 | Role | Top-1 | Top-5 | SubjECTive-QA probe |
| NoLBERT-Calls-2023 (released) | 7.6 | 21.9 | 96.2 | 25.6 | 47.1 | 64.8 |
| NoLBERT-Calls-2012, stage 1 | 5.8 | 17.9 | 95.9 | 23.9 | 45.1 | 65.4 |
| NoLBERT-Calls-2023, same recipe | 6.3 | 15.8 | 95.6 | 21.3 | 40.2 | 65.1 |
| FinBERT-Calls-2023 | 8.2 | 27.9 | 97.8 | 39.6 | 65.0 | 66.5 |
| ChronoBERT-Calls-2023 | 6.6 | 19.2 | 95.4 | 25.2 | 47.1 | 65.4 |
| SubjECTive-QA, weighted F1 | GLUE, development sets | ||||
| Encoder | Six properties (sd) | Four properties | CoLA | QNLI | SST-2 |
| NoLBERT, zero-shot base | 64.4 (0.9) | 54.8 | 14.8 | 88.0 | 85.9 |
| NoLBERT-Calls-2023, same recipe | 65.7 (1.0) | 56.8 | 26.8 | 88.2 | 86.4 |
| NoLBERT-Calls-2023 (released) | 65.8 (1.5) | 56.8 | 31.9 | 88.2 | 86.8 |
| FinBERT-Calls-2023 | 66.2 (0.9) | 57.4 | — | — | — |
| ChronoBERT-Calls-2023 | 65.6 (1.0) | 56.5 | — | — | — |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Reader | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 | 2022 | 2023 | 2024 | 2025 | 2026 |
| TF-IDF (word weights from calls up to 2012) | 49.7 | 49.9 | 50.5 | 51.1 | 52.1 | 53.1 | 53.3 | 53.7 | 55.9 | 55.6 | 56.3 | 56.1 | 56.8 | 57.3 |
| NoLBERT, zero-shot | 22.5 | 22.4 | 22.9 | 23.1 | 24.8 | 24.2 | 25.0 | 24.8 | 25.1 | 25.9 | 25.7 | 25.9 | 26.3 | 26.4 |
| ChronoBERT-2012, zero-shot | 20.7 | 20.5 | 21.6 | 21.6 | 22.9 | 21.8 | 22.1 | 22.4 | 22.3 | 22.5 | 23.2 | 23.7 | 23.8 | 23.6 |
| FinBERT, zero-shot | 29.5 | 28.8 | 28.8 | 30.2 | 30.1 | 29.8 | 30.3 | 30.8 | 31.0 | 31.3 | 31.6 | 31.6 | 31.9 | 31.8 |
| NoLBERT-Calls-2012, stage 1 | 70.1 | 70.2 | 70.6 | 70.2 | 70.8 | 71.2 | 71.9 | 69.8 | 71.3 | 71.4 | 71.2 | 71.1 | 71.3 | 71.5 |
| NoLBERT-Calls-2012, 150-minute stage 1 | 57.6 | 56.9 | 58.5 | 58.5 | 58.5 | 59.7 | 59.2 | 58.1 | 60.3 | 60.5 | 59.9 | 60.0 | 60.5 | 60.7 |
| Reader | Top-1 | 95% interval | Top-1, call-weighted | Top-3 | MRR | to released [95% interval] |
| Adapted on calls up to 2023 | ||||||
| NoLBERT-Calls-2023 (released) | 78.0 | [77.8, 78.2] | 80.2 | 94.0 | 86.4 | — |
| + word overlap | 79.0 | [78.8, 79.2] | 81.0 | 94.4 | 87.1 | 1.0 [ 1.0, 0.9] |
| ChronoBERT-Calls-2023 at 256 tokens | 77.0 | [76.8, 77.2] | 79.2 | 93.5 | 85.7 | 1.0 [0.9, 1.2] |
| + word overlap | 77.9 | [77.7, 78.1] | 80.0 | 93.9 | 86.3 | 0.1 [ 0.0, 0.3] |
| FinBERT-Calls-2023 | 74.7 | [74.5, 74.9] | 77.0 | 92.8 | 84.2 | 3.4 [3.2, 3.5] |
| Run | Change | Exchanges per call | QA batches | Learning rate | Answer tokens | Steps (best) | Validation top-1 | Test top-1 | Test, + word overlap |
| v1 | Warm start from stage 1 | 8 | 75% | 128 | 6,963 (6,270) | 75.8 | 73.8 | 75.8 | |
| v2 | + same-call packing | 32 | 75% | 128 | 7,239 (7,239) | 77.1 | 74.4 | 76.6 | |
| v3 | v2 + QA-only batches, higher LR | 32 | 100% | 128 | 6,882 (4,821) | 78.5 | 75.8 | 77.6 | |
| v4 | v2 + 256-token answers | 32 | 75% | 256 | 3,397 (3,397) | 80.3 | 77.7 | 78.6 | |
| v4c (released) | v4 + a second 150-minute run | 32 | 75% | 256 | 3,722 (3,722) | 80.7 | 78.0 | 79.0 | |
| Ensemble | Average of v4 and v3 (not released) | 80.3 | 78.4 | 79.3 |
| Passage matching | Masked terms | |||||
| Reader | Top-1 | Top-10 | Role | Top-1 | Top-5 | SubjECTive-QA probe |
| NoLBERT-Calls-2023 (released) | 7.6 | 21.9 | 96.2 | 25.6 | 47.1 | 64.8 |
| NoLBERT-Calls-2012, stage 1 | 5.8 | 17.9 | 95.9 | 23.9 | 45.1 | 65.4 |
| NoLBERT-Calls-2023, same recipe | 6.3 | 15.8 | 95.6 | 21.3 | 40.2 | 65.1 |
| v4 (stage-2 run 1) | 8.3 | 22.1 | 96.6 | 24.9 | 46.4 | 65.2 |
| v3 | 3.9 | 11.5 | 95.6 | 25.9 | 48.1 | 65.0 |
| Encoder | Assertive | Cautious | Clear | Optimistic | Relevant | Specific | Mean |
| NoLBERT, zero-shot base | 49.0 (1.4) | 59.9 (2.2) | 82.5 (0.8) | 55.4 (0.9) | 84.9 (1.1) | 54.8 (1.6) | 64.4 (0.9) |
| NoLBERT-Calls-2023, same recipe | 49.9 (3.6) | 61.0 (2.3) | 82.5 (0.8) | 59.5 (1.7) | 84.8 (1.1) | 56.6 (1.3) | 65.7 (1.0) |
| v3 | 49.9 (4.0) | 61.2 (2.7) | 82.5 (0.8) | 60.0 (1.2) | 84.9 (1.1) | 56.4 (2.5) | 65.8 (1.4) |
| NoLBERT-Calls-2023 (released) | 50.3 (4.3) | 61.3 (3.0) | 82.5 (0.8) | 59.4 (2.8) | 84.9 (1.1) | 56.3 (2.8) | 65.8 (1.5) |
| FinBERT-Calls-2023 | 50.0 (4.3) | 62.0 (1.9) | 82.4 (0.7) | 58.2 (3.6) | 85.2 (1.1) | 59.4 (1.9) | 66.2 (0.9) |
| ChronoBERT-Calls-2023 | 48.8 (4.1) | 60.7 (1.1) | 82.8 (1.2) | 60.0 (1.4) | 84.8 (1.0) | 56.7 (0.3) | 65.6 (1.0) |
| Setting | Stage 1 | Stage 2 (released) |
| Starting weights | NoLBERT | Selected stage 1 checkpoint |
| Training calls | All countries, up to 31 Dec 2012 | US, up to 31 Dec 2023 |
| Batches | 75% question–answer, 25% passage pairs | 75% question–answer, 25% passage pairs |
| Exchanges of one call per batch | Up to 8 | Up to 32 |
| Tokens, question and answer | 128 and 128 | 128 and 256 |
| Batch size | 32 | 32 |
| Item | Hardware | Budget |
| Stage 1 | One RTX 3060 (12 GB) | 480 minutes |
| Stage 2, released model | One T4 per run | 2 150 minutes |
| Released model, total | About 13 GPU-hours | |
| Stage 2 development runs (v1–v3) | One T4 each | 3 150 minutes |
| Same-recipe adaptations, six models | One T4 each | 6 150 minutes |
| Stage 1, 150-minute run (reference) | One RTX 3060 | 150 minutes |