Search research

Topics & fields

Synthetic Data357 papers Synthetic Data233 papers Synthetic Data Augmentation155 papers Synthetic Dataset54 papers Synthetic Tabular Data42 papers Synthetic Training Data33 papers

1,771–1,800 of 2,587

Papers

To See the Unseen: on the Generalization Ability of Transformers in Symbolic Reasoning
Apr 23, 2026Nevena Lazić, Liam Fowl, András György +1Transformer Encoder Transformer Architectures
UCM: Unified Modeling of Camera Control and Memory with Time-aware Positional Encoding Warping for World Models
Feb 26, 2026Tianxing Xu, Zixuan Wang, Guangyuan Wang +5Controllable Video Generation Camera Control
LLM-Extracted Covariates for Clinical Causal Inference: Rethinking Integration Strategies
Apr 18, 2026Lei Liu, Jialin Chen, Kathy MacropolCausal Inferences Mimic-Iv Datasets
A Factor Graph Approach to Scalable Multi-Output Gaussian Process Regression
Aug 12, 2026Wouter W. L. Nuijten, Esther G. van Pelt, Albert Podusenko +2Gaussian Process Factor Graphs
The Hidden Cost of Pairwise Verification in Synthetic Speech Source Tracing
Jun 10, 2026Anton Firc, Zbyněk Lička, Vojtěch Staněk +1Speaker Verification Performance Ground Truth
Estimation, Prediction, and Assortment Optimization for Markov Chain Choice Models with Panel Data
Jul 10, 2026Yalcin Akcay, Gerardo Berbeglia, Young-San LinDiscrete Choice Models Semiparametric Framework
Statistical Matching via Schrödinger Bridge beyond Conditional Independence
Jun 22, 2026Eunho Koo, Tongseok Lim, Jinwon SohnImputation Schrödinger Bridges
TrajDLM: Topology-Aware Block Diffusion Language Model for Trajectory Generation
May 11, 2026Wilson Wongso, Lihuan Li, Arian Prabowo +4Diffusion Language Models
GEODE: Angle-Adaptive OOD Detection with Universal Scorer Compatibility
May 1, 2026Bruno AbrahaoOut-Of-Distribution Detection Out-Of-Distribution
PrefMoE: Robust Preference Modeling with Mixture-of-Experts Reward Learning
May 1, 2026Ziqin Yuan, Ruiqi Wang, Dezhong Zhao +2Process Reward Modeling
COMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Training
Apr 26, 2026Zhuoqi Lyu, Qing KeVast Chemical Space Molecular Graphs
Time-Varying Bayesian Optimization Without a Metronome
Jan 31, 2025Anthony Bardou, Patrick ThiranBayesian Optimization Thompson Sampling
Robust Streaming PCA
Feb 8, 2019Daniel Bienstock, Minchan Jeong, Apurv Shukla +1
REMEDI: A Benchmark for Retention and Unlearning Evaluation in Multi-label Clinical Disease Inference
Jun 5, 2026Anurag Sharma, Sai Teja Chunchu, Prasenjit Mitra +2Unlabeled Data Disease Detection
MonoDuo: Using One Robot Arm to Learn Bimanual Policies
May 28, 2026Sandeep Bajamahal, Lawrence Yunliang Chen, Toru Lin +3Bimanual Manipulation Robotic Manipulation Policies
Classical and quantum kernel fusion for two-sample testing
Date pendingYu Terada, Yugo Ogio, Ken Arai +2Quantum Kernels Two-Sample Testing
Seeing Below the Limit of Detection: A Censored-Poisson Bayesian Latent-Growth Change-Point Detector (the Span Detector) for Serial ctDNA in HR+/HER2- Metastatic Breast Cancer
Jun 10, 2026Aarchi Singh Thakur, Abhijoy SarkarNon-Small Cell Lung Cancer Change-Point Detection
Demand-Side Measurement for Generative Engine Optimization: Constructing and Validating a Million-Persona, Intent-Annotated Buyer Corpus
Aug 30, 2026Dmitrij Żatuchin, Daniil DzemesjukPersona Online Shoppers Purchasing Intention
Synthetic Sociality: How Generative Models Privatize the Social Fabric
May 13, 2026Ana Dodik, Moira WeigelSocial Dynamics Generative Models
PANOPTICON: A PII-Based Assemblage of Naturalistic Output Tokens for Investigating Privacy Leakage Within LLM Context Window
Jul 17, 2026Ryan Thornton, Mir Mehedi Ahsan Pritom, Maanak GuptaLanguage Model Evasion Attacks Profile
Repairing Shape-Prior Shortcuts in Long-Range Single-Shot Fringe Projection Profilometry
Jul 10, 2026Adam Haroon, Cody Fleming, Beiwen LiFringe Projection Profilometry Capture-Aware Prototype Calibration
PCA-Enhanced Adaptive NVAR Framework for High-Resolution Sea Surface Temperature Forecasting in the East Sea
Jun 10, 2026Sherkhon Azimov, Susana López-Moreno, Eric Dolores-Cuenca +2Sea-Surface Temperature Data Reduced-Order Models
Learning What Not to Impute: An Uncertainty-Aware Diffusion Framework for Meaningful Missingness
Jun 3, 2026Lixing Zhang, Yidong Ouyang, Weifu Li +3Multivariate Time Series Imputation Imputation
Do Real-World Datasets Contain Natural Experiments? An Empirical Study Using Causal Feature Selection
Jun 2, 2026Gautam Gare, John Galeotti, Michael Mozer +2Causal Graph Covid-19
Dexora: Open-source VLA for High-DoF Bimanual Dexterity
May 18, 2026Zongzheng Zhang, Jingrui Pang, Zhuo Yang +22Dexterous Manipulation Pretrained Vision-Language-Action
Federated Reinforcement Learning for Efficient Mobile Crowdsensing under Incomplete Information
May 4, 2026Sumedh J. Dongare, Patrick Weber, Andrea Ortiz +3Federated Reinforcement Learning
1D-Bench: A Benchmark for Iterative UI Code Generation with Visual Feedback in Real-World
Feb 20, 2026Qiao Xu, Yipeng Yu, Chengxiao Feng +1Code Generation Rapid Prototyping
A Recipe for Long-Context Reasoning in Large Language Models via On-Policy Optimization and Distillation
Date pendingMiguel Moura Ramos, Duarte M. Alves, André F. T. MartinsEfficient Long-Context Inference Token-Level Supervision
Data-driven Prediction of Satellite-observed Avalanche Activity from Snowpack Simulations
Sep 14, 2026Jakob Grahn, Filippo Maria Bianchi, Bert Kruyt +1Orogenic Au Sentinel-2
MedFailBench: A Clinician-Built Open-Source Benchmark for Medical AI Safety Boundary Inspection
Jul 16, 2026Goktug OzkanArtificial Intelligence Safety Annotations

Apr 23, 2026 · cs.AIJ/K move · Enter open · S save

To See the Unseen: on the Generalization Ability of Transformers in Symbolic Reasoning

Nevena Lazić, Liam Fowl, András György, Csaba Szepesvári

We investigate the ability of decoder-only transformer models to perform abstract symbolic reasoning; specifically solving propositional logic reasoning problems given in-context. Previous work demonstrated that models fail to generalize to problems involving variable names that were not observed during training, and it was shown that one reason behind this is the difficulty of copying (or generating) unseen tokens. We show both theoretically and empirically that a particular representational collapse also has a crucial role: the unembeddings (last-layer weights) of unseen tokens collapse to nearly the same vector during training. The collapse makes distinguishing multiple unseen variables difficult for the model (especially when the embedding and unembedding parameters are shared), and provides a mechanistic explanation for the effectiveness of existing heuristic interventions like "active forgetting", which periodically reset the token (un)embeddings. Based on these observations, we devise a combination of techniques, involving a small architecture change facilitating copying, data diversity, and freezing or resetting (un)embeddings, that achieves generalization to unseen tokens. We support our claims with extensive controlled experiments on propositional logic reasoning problems. Beyond synthetic experiments, we also observe evidence of (un)embedding collapse in the open-weight models in the Gemma 3 family, which includes 99 unused tokens reserved for downstream use. Empirically we find that the correlated embeddings of these tokens are a poor initialization for finetuning applications.

Transformer Encoder Transformer Architectures Discrete Tokens Unseen Generalisation Catastrophic Forgetting

Open paper Ask about it PDF