LLM Training

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

31 papers in the last four weeks, up 72% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 226

All topics

MemGuard-Alpha: Limits of Membership Inference for Detecting and Filtering Memorization-Contaminated Signals in LLM-Based Financial Forecasting
Mar 26, 2026Anisha Roy, Dip RoyMembership Inference Attacks LLM Auditing
Department of Electronics and Communication Engineering, Jaypee Institute of Information Technology, Noida, India · Department of Computer Science and Engineering, Indian Institute of Technology Patna, India
Language Generation with Replay: A Learning-Theoretic View of Model Collapse
Mar 12, 2026Giorgio Racca, Michal Valko, Amartya SanyalModel Collapse Data Contamination in Language Models
University of Copenhagen · Isara Labs
Maximizing Mutual Information Between Prompt and Response Improves LLM Performance With No Additional Data
Mar 10, 2026Hyunji Nam, Haoran Li, Natasha JaquesLLM Personalization Mutual Information Maximization
Stanford University · Character AI · University of Washington
.tmu: A Low-Entropy Tree-Structured Representation for LLM-Assisted Scientific Writing
Mar 3, 2026Tianyou Liu, Ziqiang Li, Xurui Liu +2AI-Assisted Scientific Research LLM Training
Southern University of Science and Technology · Alibaba · Tsinghua University +2
You Can Learn Tokenization End-to-End with Reinforcement Learning
Feb 15, 2026Sam Dauncey, Roger WattenhoferSubword Tokenization LLM Training
Department of Electrical Engineering, ETH Zurich, Zurich, Switzerland.
MSign: An Optimizer Preventing Training Instability in Large Language Models via Stable Rank Restoration
Feb 2, 2026Lianhai Ren, Yucheng Ding, Xiao Liu +2Language Model Pretraining LLM Training
Microsoft SIGMA Team, Microsoft Research.
Quantifying the Effect of Test Set Contamination on Generative Evaluations
Jan 7, 2026Rylan Schaeffer, Joshua Kazdan, Baber Abbasi +8Language Model Generation Evaluation Benchmark Contamination
Stanford Computer Science · Stanford Statistics · EleutherAI +3
Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation
Dec 24, 2025Wei-Rui Chen, Vignesh Kothapalli, Ata Fatahibaarzi +5CoT Distillation Selective Knowledge Distillation
1The University of British Columbia · 2LinkedIn · 3Canada Research Chair in NLP and ML
EDGC: Entropy-driven Dynamic Gradient Compression for Efficient LLM Training
Nov 13, 2025Qingao Yi, Jiaang Duan, Jun Zhang +5Communication-Efficient Distributed Training Gradient Compression
University of Shanghai for Science and Technology, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China · Zhejiang University, Hangzhou, China
The Environmental Impacts of Language Model Training Keep Rising Now is the Time to Catch Impacts on the Rebound
Oct 10, 2025Clément Morand, Anne-Laure Ligozat, Aurélie NévéolEnergy-Efficient ML LLM Training
STL · Université Paris-Saclay, CNRS, LISN · ENSIIE, LISN, STL +2
Geometrically Principled Randomized Optimization for Efficient LLM Training
Oct 2, 2025Sahar Rajabi, Nayeema Nonta, Sirisha RambhatlaLanguage Model Pretraining Deep Learning Optimization
CriticalML Lab, University of Waterloo
CHRONOBERG: Capturing Language Evolution and Temporal Awareness in Foundation Models
Sep 26, 2025Niharika Hegde, Subarnaduti Paul, Lars Joel-Frey +4Language Modeling Semantic Change Detection
DFKI · TU Darmstadt · University of Bremen +2
NoLoCo: No-all-reduce Low Communication Training Method for Large Models
Jun 12, 2025Jari Kolehmainen, Nikolay Blagoev, Semih Kara +3Communication-Efficient Distributed Training Decentralized Learning
1Gensyn
Learning Composable Chains-of-Thought
May 28, 2025Fangcong Yin, Zeyu Leo Liu, Liu Leqi +2LLM Fine-Tuning CoT Reasoning
The University of Texas at Austin · Princeton University
Learning by Surprise: Adaptive Mitigation of Model Collapse in Large Language Models
Oct 16, 2024Daniele Gambetta, Gizem Gezici, Fosca Giannotti +3Model Collapse Training Data Selection
University of Pisa, Pisa, Italy · ISTI-CNR, Pisa, Italy · Scuola Normale Superiore, Pisa, Italy +1
Musec: MomentUm SpEctral Clipping for Stable Muon-type Training
Date pendingZhuanghua Liu, Menglian Wang, Luo LuoMuon Optimizer LLM Training
National University of Singapore · Wonders Information Co., Ltd. · Fudan University

Mar 26, 2026, cs.LGJ/K move · Enter open · S save

MemGuard-Alpha: Limits of Membership Inference for Detecting and Filtering Memorization-Contaminated Signals in LLM-Based Financial Forecasting

Anisha Roy, Dip Roy

Department of Electronics and Communication Engineering, Jaypee Institute of Information Technology, Noida, India · Department of Computer Science and Engineering, Indian Institute of Technology Patna, India

Membership Inference Attacks LLM Auditing LLM-Based Time Series Forecasting LLM Training Finance

Open paper AskPDF arXiv

Large language models are increasingly used to generate financial alpha signals, but many have memorized the historical data in their training corpora, producing apparent accuracy that collapses out of sample. Membership inference attacks (MIA) have been proposed as a diagnostic. What has not been established is whether MIA scores are informative about memorization in this setting, or whether signal-level filtering built on them helps once realistic costs are applied.We introduce MemGuard-Alpha, comprising a composite contamination score combining five MIA methods with a temporal proximity feature, and Cross-Model Memorization Disagreement, which exploits variation in training cutoffs across models. We then audit both. Across seven LLMs (124M-7B), 50 S&P 100 constituents, 42,800 prompts and 299,600 prompt-model MIA scores spanning 2019-2024, three findings emerge.First, where in-sample status is defined by a training cutoff, a temporal proximity feature recovers that label perfectly (ROC-AUC 1.000) because it is a monotone transform of the defining variable; any composite score containing such a feature reports separation that is arithmetic rather than detection. Second, the discriminative power of the MIA scores is largely attributable to scale differences between models: asked at a fixed date which models had that date in training, raw scores appear highly informative (AUC up to 0.99), but under three independent within-model normalizations discrimination falls to 0.487-0.537. Third, with transaction costs applied symmetrically, no filtering variant improves risk-adjusted performance over the unfiltered ensemble, none attains significant Fama-French five-factor alpha, and excluding the single weakest model outperforms every contamination-based filter.We report these as negative results with the failure modes that produced them, and release all artifacts needed to reproduce them.

Momentum

Related topics

Latest papers 226