Fast Inference

Ask about this topic

Momentum

8 papers in the last four weeks, up 60% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 101

All topics
CardsList

  1. Dual-Stream Simultaneous Translation via 2D Grid Attention

    Sep 28, 2026Yu Pu, Wei-Qiang ZhangBidirectionalFast Inference

    Department of Electronic Engineering, Tsinghua University, Beijing 100084, China · Institute for Embodied Intelligence and Robotics, Tsinghua University, Beijing 100084, China

  2. RAVEL: Asynchronous Rolling Inference for Flow-Based Vision-Language-Action Models

    Sep 28, 2026Yuhan Chen, Ke Yu, Pengfei Liu +3Flow-Matching Vision-Language-ActionInference Latency

    Zhejiang University · Nanyang Technological University

  3. ReSync: Re-Aligning the Two Clocks of Asynchronous World-Action Models

    Sep 27, 2026Xi Lin, Feihong Zhang, Yulong Shi +6Recent World-Action ModelsWorld Models

    Johns Hopkins University · Tsinghua University · Yinwang Intelligent Technology Co., Ltd. +1

  4. FoldQuantVLA: Native Low-Bit Quantization of Vision-Language-Action Models via Consistent Folding

    Sep 21, 2026Hung T. Ho, Khanh D. Nguyen, Quang D. Nguyen +5Post-Training QuantizationFast Inference

    VinRobotics, Vietnam · AICV Lab, EECS Department, University of Arkansas · School of Advanced Manufacturing and Robotics, Peking University, China +2

  5. FASA: Feedback-Aware Sampling Adaptation for Efficient Diffusion-Based VLA Models

    Sep 16, 2026Yuchen Han, Jianhan Wu, Xiaoyang Qu +3Diffusion-Based Vision-Language-ActionsAdaptive Sampling

    South China University of Technology Guangzhou, China · Ping An Technology (Shenzhen) Co., Ltd. Shenzhen, China · Harbin Institute of Technology Shenzhen, China

  6. rMuscle: Robotic Muscle Memory for Efficient Vision-Language-Action Model Inference

    Sep 16, 2026Kaijun Zhou, Zhiyang Li, Le Chen +1Embodied Artificial IntelligenceFast Inference

    Institute of Parallel and Distributed Systems, Shanghai Jiao Tong University

  7. CSWAM: Better Causal Semantic Representations for Out-of-Distribution Generalization in World Action Models

    Sep 16, 2026Tianbin Liu, Jian Zhu, Taiyi Su +5Efficient World-Action ModelWorld Models

    AIRC, Midea Group · Tongji University

  8. Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Models

    Sep 14, 2026Shijian Xu, Andrea Miele, Metod Jazbec +3Diffusion Language ModelsFast Inference

    University of Basel · University of Amsterdam · Johns Hopkins University

  9. OCR-MetaReasoning Benchmark: Evaluating the Meta-Reasoning Ability of MLLMs in Text-Rich Image Understanding

    Aug 31, 2026Gengxu Li, Yuan Wu, Yi ChangMultimodal Large Language ModelsReasoning Skills

    School of Artificial Intelligence, Jilin University · 2Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China · 3International Center of Future Science, Jilin University

  10. SmoothRL: Online Reinforcement Learning During Asynchronous Execution

    Aug 30, 2026Guang Gao, Yuxuan Nong, Baifu Huang +1Asynchronous ExecutionOffline Reinforcement Learning

    Astribot Team

  11. Accelerating Unified Multimodal Models with Core-Expansion Routing and Unified Computation Scheduling

    Aug 29, 2026Wengyi Zhan, Chenqian Yan, Songwei Liu +2Multimodal ModelFast Inference

    1Xiamen University · 2ByteDance

  12. LipCache: A Local Inference Proxy with Certified Caching for Edge Image Classification Service

    Aug 13, 2026Zhengzhe Xiang, Yinlin Chen, Fuli Ying +3Fast InferenceCache

  13. APEX: Adaptive Expert Prefetching for Memory-Efficient Edge MoE Inference

    Aug 12, 2026Alish Kanani, Layan Badawi, Umit Y. OgrasMixture-Of-Expert InferenceMixture-Of-Experts

    University of Wisconsin–Madison

  14. How Much Does It Cost to Answer My Question? Benchmarking Cloud VLM-based VQA Systems

    Aug 8, 2026Henri Vanhuynegem, Weitao Xu, Yiran Shen +1Recent Vision-Language ModelsQwen3

    Department of Software Technology, Delft University of Technology, Delft, The Netherlands · Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China · School of Software, Shandong University, Jinan, China +1

  15. Robust and Efficient Motion Reasoning for Privacy-Aware Classroom Incident Recognition

    Aug 5, 2026Paritosh Parmar, Landy Lan, Hong Yang +2Open-Vocabulary Action RecognitionFast Inference

    Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore

  16. CascadeLUT: Information-Ordered Streaming Inference for Bandwidth-Constrained FPGAs

    Aug 1, 2026Oliver Cassidy, Marta Andronic, George A. ConstantinidesField-Programmable Gate ArraysFast Inference

    Department of Electrical and Electronic Engineering, Imperial College London, UK

  17. Towards Autonomous Aircraft Surveillance from Nanosatellites through On-Board Inference and Generative Data Augmentation

    Jul 30, 2026Antonio Delgado-Rosa, David Muñoz-Valero, Enrique Adrian Villarrubia-Martin +1Multiple Satellite SystemsUnmanned Aerial Vehicle Detection

    Escuela de Ingeniería Industrial y Aeroespacial de Toledo, Department of Technologies and Information Systems, Universidad de Castilla–La Mancha, Avenida Carlos III, s/n, Toledo, 45071, Spain · Escuela Superior de Informática, Department of Technologies and Information Systems, Universidad de Castilla-La Mancha, Paseo de la Universidad 4, Ciudad Real, 13071, Spain

  18. Driving up Inference Energy on SNNs: Per-Sample and Universal Sponge Attacks

    Jul 30, 2026Spyridon Raptis, Haralampos-G. StratigopoulosSpiking Neural NetworksTime-To-First-Spike

    Sorbonne Universit´e, CNRS, LIP6, Paris, France

  19. Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memory

    Jul 29, 2026Yanbo Ding, Zhizhi Guo, Quanyue Song +4Audio-Video GenerationReal-Time Systems

    Shenzhen Key Lab of Computer Vision and Pattern Recognition, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd., China · School of Artificial Intelligence, University of Chinese Academy of Sciences +2

  20. AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding

    Jul 28, 2026Hong Liu, Rui Cen, Junhan Shi +10Speculative DecodingBlock-Diffusion Drafter

    Tencent Inc.

  21. WHTMix: Efficient Stereo Depth Estimation via Walsh-Hadamard Token Mixing

    Jul 28, 2026Prathyush Sajith, Emadeldeen Hamdan, Ahmet Enis CetinStereo MatchingStereo

    Dept. of Electrical and Computer Engineering, University of Illinois at Chicago Chicago, IL, USA

  22. A Motion-Aware Vector Quantization Framework with Centroid Reuse for Efficient VLA Inference

    Jul 27, 2026Zhuoran Song, Haozhe Jiang, Chunyu Qi +4Fast InferenceResidual Vector Quantization

    School of Computer Science, Shanghai Jiao Tong University, 2 Institute of Automation, Chinese Academy of Sciences China

  23. LoRA for Gender-Inclusive Rewriting and Activation Steering for Counter-Narrative Generation

    Jul 25, 2026Akhil Rajeev P, Manoj Balaji JLinear Activation SteeringGender

    C-DAC, Bangalore · Indian Institute of Technology, Kharagpur

  24. Sparse by Command: Task-Conditional Compute Skipping for Multi-Task Inference Accelerators

    Jul 24, 2026Afzal Ahmad, Gaoyu Mao, Shoubo Hu +4Fast InferenceSparsity

    The Hong Kong University of Science and Technology · Huawei Noah’s Ark Lab · The Hong Kong University of Science and Technology (Guangzhou)

  25. How Fast Can Reward Models Score? A Systems Study of C++ and PyTorch Inference Runtimes for RLHF

    Jul 22, 2026Venkata Naga Sai Vishnu Rohit Pulipaka, Anish Katta, Deva Rohit Reddy PeddireddyDestination-Only ScoringFast Inference

  26. Wavefront Parallelization for Efficient Learned Image Compression

    Jul 21, 2026Shimon Murai, Fangzheng Lin, Kasidis Arunruangsirilert +1Learned Image CompressionAutoregressive Image Generation

    School of Fundamental Science and Engineering Waseda University Tokyo, Japan · School of Engineering Institute of Science Tokyo Tokyo, Japan

  27. FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications

    Jul 20, 2026Krish Agarwal, Zhuoming Chen, Yanyuan Qin +3FlashMultimodal Agents

    Carnegie Mellon University · AMD · University at Buffalo

  28. NIFA: Nonlinear IMC enhanced FPGA for efficient ML inference

    Jul 16, 2026Jiajun Hu, Ruthwik Reddy Sunketa, Lei Zhao +3In-Memory ComputingField-Programmable Gate Arrays

    Arizona State University, Tempe, AZ, USA · Hewlett Packard Enterprise Labs, Fort Collins, CO, USA

  29. Boogu-Image-0.1: Boosting Open-Source Unified Multimodal Understanding and Generation

    Jul 14, 2026Guoxuan Chen, Chufeng Xiao, Haoran Yang +30Modern Text-To-Image ModelsText-To-Image

    Boogu Team

  30. Jetson-PI: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference

    Jul 14, 2026Zebin Yang, Qi Wang, Yunhe Wang +6Fast InferenceNvidia Jetson Orin Nano

    Peking University · PrimeBot Research Institute · AIRS

  31. Full-Pipeline Inference Optimization for MiMo-V2.5 Series: Pushing Hybrid SWA Efficiency to the Limit

    Jul 14, 2026Xiaomi MiMo Team, Anqi Liu, Aoxin Ma +28Fast InferenceHybrid Mamba-Transformer Model

    MiMo Team, Xiaomi

  32. MobileSAM2: Lightweight Segment Anything for Spatial Intelligence

    Jul 14, 2026Kai Jiang, Jiaxing Huang, Jingyi Zhang +5Knowledge DistillationMobilenetv2

    Hong Kong Polytechnic University, HK, China · Nanyang Technological University, Singapore · Xidian University, China +1

  33. Amplitude-Only FFN Intervention for Tool-Structured LLM Inference Method: Gated Evaluation Protocol, and Cross-Model Empirical Results

    Jul 13, 2026Sheng Xu, Junhua Wang, Boyuan Huang +3LLM Inference OptimizationFast Inference

    Alibaba Cloud

  34. Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots

    Jul 2, 2026Ling Xu, Chuyu Han, Borui Li +6Embodied Artificial IntelligenceFast Inference

    Southeast University · Nanjing University · Microsoft Research +1

  35. Scaling Up Thermodynamic AI Models

    Jun 30, 2026Andrew G. MooreNeural Network InferenceFast Inference

  36. HARD-KV: Head-Adaptive Regularization for Decoding-time KV Compression

    Jun 27, 2026Yuxuan Yang, Feiyang Ren, Bowen Zeng +4Key-Value Cache CompressionLLM Inference Optimization

    The State Key Laboratory of Blockchain and Data Security, Zhejiang University

  37. SpikeVLA: Vision-Language-Action Models with Spiking Neural Networks

    Jun 26, 2026Ruiqi Song, Dujun Nie, Siyu Teng +7Diffusion-Based Vision-Language-ActionsFast Inference

    College of Surveying and Geo-informatics, Tongji University, Shanghai, China · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · OpenSpace Lab, China +6

  38. Dual Distribution Estimation for Zero-shot Noisy Test-Time Adaptation with VLMs

    Jun 24, 2026Wenjie Zhu, Yabin Zhang, Liang Xu +3Out-Of-Distribution DetectionVision-Language Model Adaptation

    The Hong Kong Polytechnic University · Harbin Institute of Technology (Shenzhen) · Shanghai Jiao Tong University +1

  39. Start Right, Arrive Right: Asynchronous Execution via Initial Noise Selection

    Jun 18, 2026Trong-Bao Ho, Quang-Tan Nguyen, Thien-Loc Ha +7Action ChunksRobot Policies

    VinRobotics · VinUniversity · DFKI +2

  40. Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance

    Jun 17, 2026Kangsheng Duan, Ziyang Xu, Wenyu Liu +3InpaintingFast Inference

    Huazhong University of Science and Technology · *Equal contribution. Completed this work as interns at VIVO AI Lab. · † Project leader. +1

  41. STREAM: Multi-Tier LLM Inference Middleware with Dual-Channel HPC Token Streaming

    Jun 11, 2026Anas Nassar, Steve Mohr, Leonard Apanasevich +1LLM Inference OptimizationFast Inference

    Advanced Cyberinfrastructure for Education and Research (ACER) University of Illinois Chicago Chicago, Illinois, USA

  42. Real-Time Execution with Autoregressive Policies

    Jun 11, 2026Sangkyu Lee, Seohyeon Park, Tackgeun You +4Fast InferenceReal-Time Systems

    Korea Institute of Science and Technology · Seoul National University · Google Research

  43. ViT-FREE: Efficient Face Recognition via Early Exiting and Synthetic Adaptation

    Jun 10, 2026Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira +2Face Recognition ModelVision Transformer

    Fraunhofer Institute for Computer Graphics Research IGD, Germany · Department of Computer Science, TU Darmstadt, Germany

  44. Battery detection of XRay images using transfer learning

    Jun 10, 2026Nermeen Abou Baker, David Rohrschneider, Uwe HandmannLithium-Ion BatteriesYolo26

    1- Ruhr West University of Applied Sciences - Dept of Computer Science Lutzowstrassse 5, 46236 Bottrop - Germany

  45. Fast Speech Foundation Model Distillation Using Interleaved Stacking

    Jun 10, 2026Eungbeom Kim, Kyogu LeeFast InferenceSpeech Language Models

    1IPAI, 2AIIS, 3Dept. of Intelligence and Information Seoul National University, Seoul, Republic of Korea

  46. SpAArSIST: Sparsified AASIST for Efficient and Reliable Anti-Spoofing

    Jun 10, 2026Anton Firc, Vojtěch Staněk, Zbyněk Lička +2SpoofingSelf-Supervised Learning

  47. AVIS: Adaptive Test-Time Scaling for Vision-Language Models

    Jun 10, 2026Ahmadreza Jeddi, Minh Ngoc Le, Amirhossein Kazerouni +8Fast InferenceEfficient Inference

    AI Center-Toronto, Samsung Electronics · University of Toronto · Vector Institute +1

  48. Bridged SBI: Correcting Biased Low-Fidelity Posteriors for Cost-Efficient High-Fidelity Inference

    Jun 8, 2026Gahee Kim, Yuki Kadokawa, Sandro M. Alcantara Tacora +5High-FidelityFast Inference

    NARA INSTITUTE OF SCIENCE AND TECHNOLOGY, Nara, Japan · PUBLIC WORKS RESEARCH INSTITUTE, Ibaraki, Japan

  49. vla.cpp: A Unified Inference Runtime for Vision-Language-Action Models

    Jun 6, 2026Khanh D. Nguyen, Hung T. Ho, Chinh T. Nguyen +5Vision-Language-Action FrameworkFast Inference

    VinRobotics, Vietnam · Max Planck Research School for Intelligent Systems, Germany · University of Stuttgart, Germany +3

  50. Multilingual Detection of Alzheimer's Disease from Speech: A Cross-Linguistic Transfer Learning Approach

    Jun 4, 2026Nadine Yasser Abdelhalim, Emmanuel Akinrintoyo, Nicole SalomonsMultilingual Language ModelsCross-Lingual Transfer

    Department of Computing, Imperial College London, United Kingdom

  51. Audio Interaction Model

    Jun 3, 2026Zhifei Xie, Zihang Liu, Ze An +8Audio UnderstandingAutomatic Speech Recognition

    1NTU · 2NUS · 3CUHK

  52. Spike-Aware INT8 Execution for Spiking Language Models on Commodity CPUs

    Jun 2, 2026Ting LiuTime-To-First-SpikeFast Inference

    SymbolicLight Research Foshan, Guangdong, China

  53. Permissive Safety Through Trusted Inference: Verifiable Belief-Space Neural Safety Filters for Assured Interactive Robotics

    Jun 1, 2026Haimin HuSafety FiltersAI Trustworthiness

    Department of Computer Science, Johns Hopkins University, USA

  54. X-Stream: Exploring MLLMs as Multiplexers for Multi-Stream Understanding

    Jun 1, 2026Peiwen Sun, Xudong Lu, Huadai Liu +10StreamingMultimodal Large Language Models

    MMLab, Chinese University of Hong Kong · Hong Kong University of Science and Technology · Huawei Research

  55. VisualThink-VLA: Visual Intermediate Reasoning for Effective and Low-Latency Vision-Language-Action Policies

    May 28, 2026Mingjian Gao, Wenqiao Zhang, Yuqian Yuan +9Latent Visual ReasoningVisual Reasoning

    1Zhejiang University · 2Cornell University · 3National University of Singapore +1

  56. ESAM++: Efficient Online 3D Perception on the Edge

    May 28, 2026Qin Liu, Lavisha Aggarwal, Saptarashmi Bandyopadhyay +43D PerceptionFast Inference

    Stanford University · Google · UC San Diego

  57. PrunePath: Towards Highly Structured Sparse Language Models

    May 27, 2026Zhexuan Gu, Zixun Fu, Yancheng YuanUnstructured PruningSparsity

    Department of Applied Mathematics, The Hong Kong Polytechnic University

  58. JetViT: Efficient High-Resolution Vision Transformer with Post-Training Attention Search

    May 26, 2026Dongyun Zou, Zhuoyang Zhang, Junyu Chen +8Self-Supervised Vision TransformersRecent Vision Foundation Models

    MIT · NVIDIA · University of Pennsylvania +1

  59. RT-Lynx: Putting the GEMM Sparsity In a Right Way for Diffusion Models

    May 26, 2026Xing Cong, Hanlin Tang, Kan Liu +3Diffusion TransformersDiffusion Models

    1Alibaba Group · 2Independent Researcher

  60. Do Language Models Need Sleep? Offline Recurrence for Improved Online Inference

    May 25, 2026Sangyun Lee, Sean McLeish, Tom Goldstein +1LLM Inference OptimizationFast Inference

    Carnegie Mellon University · University of Maryland