Non Causal

Non-causal models, which process data without strict temporal ordering constraints, are a growing area of research across various machine learning domains. Current efforts focus on improving their performance in tasks like speech recognition and image generation, often employing architectures like cascaded encoders and diffusion probabilistic models, and exploring techniques such as knowledge distillation for model compression. This research aims to enhance model efficiency, robustness, and generalization capabilities, leading to more effective and resource-efficient applications in areas such as speech processing, computer vision, and causal inference. The development of more sophisticated non-causal models promises significant advancements in these fields.

Papers

September 13, 2024

NEST-RQ: Next Token Prediction for Speech Self-Supervised Pre-Training
Minglun Han, Ye Bai, Chen Shen, Youjia Huang, Mingkun Huang, Zehua Lin, Linhao Dong, Lu Lu, Yuxuan Wang
Next Token Prediction Speech Supervised Learning Model Non Streaming Automatic Speech Recognition Non Causal Nest Agent Random Projection Quantizer

August 20, 2024

A Full DAG Score-Based Algorithm for Learning Causal Bayesian Networks with Latent Confounders
Christophe Gonzales, Amir-Hosein Valizadeh
Practical Algorithm Probabilistic Graphical Model Latent Confounders Causal Bayesian Network Non Causal Dag DLT

April 27, 2024

Causal Diffusion Autoencoders: Toward Counterfactual Generation via Diffusion Probabilistic Models
Aneesh Komanduri, Chen Zhao, Feng Chen, Xintao Wu
Diffusion Probabilistic Model Counterfactual Generation Counterfactual Image Diffusion Autoencoder Non Causal

March 6, 2024

Causality-based Cross-Modal Representation Learning for Vision-and-Language Navigation
Liuyi Wang, Zongtao He, Ronghao Dang, Huiyi Chen, Chengju Liu, Qijun Chen
Causal Model Vision and Language Navigation Robust Navigation Cross Modal Representation Learning Non Causal

February 22, 2024

How Transformers Learn Causal Structure with Gradient Descent
Eshaan Nichani, Alex Damian, Jason D. Lee
Transformer Megatron Decepticons Gradient Descent Causal Structure Latent Causal Non Causal

December 15, 2023

On the compression of shallow non-causal ASR models using knowledge distillation and tied-and-reduced decoder for low-latency on-device speech recognition
Nagaraj Adiga, Jinhwan Park, Chintigari Shiva Kumar, Shatrughan Singh, Kyungmin Lee, Chanwoo Kim, Dhananjaya Gowda
Knowledge Distillation Linear Compression Cascade Model High Quality Decoder Recurrent Neural Network Transducer Speech Recognition Accuracy Non Causal

August 24, 2023

Causal Parrots: Large Language Models May Talk Causality But Are Not Causal
Matej Zečević, Moritz Willig, Devendra Singh Dhami, Kristian Kersting
Causal Inference Causal Model Causal Pattern Structural Causal Model Causal Event Causal Knowledge Non Causal

July 17, 2023

Lifted Sequential Planning with Lazy Constraint Generation Solvers
Anubhav Singh, Miquel Ramirez, Nir Lipovetzky, Peter J. Stuckey
Constraint Programming Sequential Planning Non Causal Clause Elimination

December 1, 2022

Navigating causal deep learning
Jeroen Berrevoets, Krzysztof Kacprzyk, Zhaozhi Qian, Mihaela van der Schaar
Deep Learning Model Causal Event Causal Knowledge Non Causal

April 13, 2022

A Unified Cascaded Encoder ASR Model for Dynamic Model Sizes
Shaojin Ding, Weiran Wang, Ding Zhao, Tara N. Sainath, Yanzhang He, Robert David, Rami Botros, Xin Wang, Rina Panigrahy, Qiao Liang, Dongseong Hwang, Ian McGraw, Rohit Prabhavalkar, Trevor Strohman
Automatic Speech Recognition Model Size State of the Art Encoders Encoder Model Non Causal