Modality-Specific Encoders

Ask about this topic

Momentum

0 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21
View on map

Latest papers 32

All topics
CardsList

  1. OmniRoute: Mapping Temporal Semantic Evidence to Audio-Visual Token Budgets for Efficient Omnimodal Large Language Models

    Sep 29, 2026Yuchen Deng, Zidang Cai, Feidiao Yang +4Omni-ModalAudio-Visual Reasoning

    Shenzhen International Graduate School, Tsinghua University, China · Pengcheng Laboratory, China

  2. Towards Unified Dynamic Face Landmark Detection

    Aug 11, 2026Sebastian Regalado, Varshanth R. Rao, Ruowei Jiang +2One-Shot Medical Landmark DetectionModality-Specific Encoders

    University of Toronto · 2ModiFace · †Work completed while employed at ModiFace +1

  3. MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities

    Jul 28, 2026Mingqiao Ye, Zhaochong An, Zhitong Gao +11Modality-Specific EncodersComplementary Modalities

    1EPFL · University of Copenhagen · 4CUHK +3

  4. OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models

    Jul 25, 2026Jinsen Su, Yongdong Luo, Yuexiao Ma +3Token CompressionOmni-Modal

    Media Analytics and Computing Lab, Xiamen University, Xiamen, China · Institute of Artificial Intelligence, Xiamen University, Xiamen, China · School of Informatics, Xiamen University, Xiamen, China +1

  5. Advancing All-Weather Building Damage Mapping to the Instance Level: Outcomes and Insights from the 2026 Bright Challenge

    Jul 23, 2026Hongruixuan Chen, He Huang, Haifeng Wang +19Post-Disaster Damage AssessmentSynthetic Aperture Radar

  6. Mixture of Probes: Learning from Privileged Modalities in Multimodal LLMs Through Probing

    Jul 9, 2026Dominick Reilly, Qiyu Wu, Hiromi Wakaki +2Multimodal Large Language ModelsModality-Specific Encoders

    Sony Group Corporation · University of North Carolina at Charlotte · Sony AI

  7. Integrating Neural Encoders in Bayesian Generalized Linear Mixed Models for Multimodal Data

    Jul 6, 2026Yuankang Zhao, Youngsoo Baek, Felipe A. Medeiros +2Generalized Linear ModelMultimodal Learning

    Department of Biostatistics & Bioinformatics, Duke University · Department of Ophthalmology, University of Miami

  8. OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models

    Jul 3, 2026Shijie Cao, Qingyu Zhang, Boxi Yu +6Token CompressionAudio-Visual Reasoning

    School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Limerick +1

  9. TactX: Learning Shared Tactile Representations Across Diverse Sensors

    Jun 30, 2026Junsung Park, Sachin Bhadang, Carmelo Sferrazza +2TactileTac

    UC San Diego · Seoul National University · Amazon FAR

  10. LoRA-Tuned Large Language Models for Dementia Detection via Multi-View Speech-Derived Features

    Jun 26, 2026Jonghyeon Park, Olivier Jiyoun Jung, Myungwoo OhAlzheimerSpeaker

    NAVER Cloud, South Korea · Division of Communication and Media, Ewha Womans University, South Korea

  11. MJEPA: A Simple and Scalable Joint-Embedding Predictive Architecture for Audio-Visual Learning

    Jun 23, 2026Revant Teotia, Adrien Bardes, Michael Rabbat +3Video Joint Embedding Predictive ArchitectureAudio-Visual Reasoning

    1FAIR at Meta · 2New York University

  12. AVOC: Enhancing Hour-Level Audio-Video Understanding in Omni-Modal LLMs via Retrieval-Inspired Token Compression

    Jun 23, 2026Yijing Chen, Wenhui Tan, Xiaoyi Yu +7Audio-Visual ReasoningLarge Audio Language Models

    1Gaoling School of Artificial Intelligence, Renmin University of China · 2Huawei Inc.

  13. TactSpace: Learning a Physics-enriched Shared Latent Space for Tactile Sim-to-Real Transfer

    Jun 17, 2026Arunim Joarder, Arjun Bhardwaj, René Zurbrügg +6TactileSim-To-Real Reinforcement Learning

    Robotic Systems Lab, ETH Zürich · ETH AI Center · NVIDIA +1

  14. VinQA: Visual Elements Interleaved Long-form Answer Generation for Real-World Multimodal Document QA

    Jun 15, 2026Young Rok Jang, Hyesoo Kong, Kyunghwan An +3Modality-Specific EncodersLong-Form Generation

    LG AI Research

  15. Modality Forcing for Scalable Spatial Generation

    Jun 11, 2026Bardienus Pieter Duisterhof, Deva Ramanan, Jeffrey Ichnowski +2Multi-Reference Image GenerationDense Prediction

    Carnegie Mellon University · World Labs

  16. Explicit Representation Alignment for Multimodal Sentiment Analysis

    Jun 8, 2026Baode Wang, Ziming Wang, Huacan Wang +2Multimodal Sentiment AnalysisModality-Specific Encoders

    AgentAlpha

  17. V-LynX: Token Interface Alignment for Video+X LLMs

    May 30, 2026Jungin Park, Jiyoung Lee, Kwanghoon SohnModality-Specific EncodersRobotwin

    Yonsei University, Seoul, South Korea · Ewha Womans University, Seoul, South Korea

  18. Two-Stage Multimodal Framework for Emotion Mimicry Intensity Prediction

    May 21, 2026Dinithi Dissanayake, Shaveen Silva, Ovindu Atukorala +2Emotion RecognitionModality-Specific Encoders

    Augmented Human Lab, National University of Singapore, Singapore · University of Moratuwa, Sri Lanka

  19. Speech-Guided Multimodal Learning for Vocal Tract Segmentation in Real-Time MRI

    May 18, 2026Daiqi Liu, Lukas Mulzer, Md Hasan +11Vocal Tract ShapeMagnetic Resonance Imaging

    Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany · Smart Imaging Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany · Harvard Medical School / Massachusetts General Hospital, Boston, USA +3

  20. OmniSelect: Dynamic Modality-Aware Token Compression for Efficient Omni-modal Large Language Models

    May 18, 2026Morunliu Yang, Ruotao Xu, Le Li +6Visual Token PruningModality-Specific Encoders

    1Soochow University · Peking University

  21. jina-embeddings-v5-omni: Geometry-preserving Embeddings via Locked Aligned Towers

    May 8, 2026Florian Hönicke, Michael Günther, Andreas Koukounas +4Multimodal EmbeddingsModality-Specific Encoders

    Jina by Elastic

  22. PRIMED: Adaptive Modality Suppression for Referring Audio-Visual Segmentation via Biased Competition

    May 8, 2026Yuchen He, Jing ZhangVideo Object SegmentationModality-Specific Encoders

    School of Information Science and Engineering, East China University of Science and Technology, Shanghai, China

  23. OmniEncoder: See, Hear, and Feel Continuous Motion Like Humans With One Encoder

    May 2, 2026Detao Bai, Shimin Yao, Weixuan Chen +4Audio-Visual ReasoningOmni-Modal

    1Tongyi Lab Alibaba Group · Shenzhen University of Advanced Technology

  24. Group Cognition Learning: Making Everything Better Through Governed Two-Stage Agents Collaboration

    May 1, 2026Chunlei Meng, Pengbin Feng, Rong Fu +7Multimodal LearningCross-Modal

    The College of Intelligent Robotics and Advanced Manufacturing, Fudan University · University of Southern California · University of Macau +5

  25. Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling

    Apr 26, 2026Zhen Ye, Xu Tan, Aoxiong Yin +8Audio-Video GenerationAutoregressive Diffusion

    Hong Kong University of Science and Technology · Independent Researcher · Zhejiang University +3

  26. Uni-Encoder Meets Multi-Encoders: Representation Before Fusion for Brain Tumor Segmentation with Missing Modalities

    Apr 24, 2026Peibo Song, Xiaotian Xue, Jinshuo Zhang +5Brain Tumor SegmentationMagnetic Resonance Imaging

    School of Mathematics, Shandong University · 2GSFS, The University of Tokyo · 3MedVisAI Lab +2

  27. Breaking the Illusion: When Positive Meets Negative in Multimodal Decoding

    Apr 22, 2026Yubo Jiang, Yitong An, Xin Yang +7Modality-Specific EncodersObject Hallucination

    School of Astronautics, Beihang University, Beijing 102206, China · Longcat Interaction Team, Meituan, Beijing 100102, China · Tianmushan Laboratory, Beihang University, Hangzhou 311115, China

  28. OmniGCD: Abstracting Generalized Category Discovery for Modality Agnosticism

    Apr 16, 2026Jordan Shipard, Arnold Wiliem, Kien Nguyen Thanh +2Generalized Category DiscoveryModality-Specific Encoders

    SAIVT, QUT, Australia · Shield AI, Australia · La Trobe University, Australia

  29. DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression

    Mar 15, 2026Bingzhou Li, Tao HuangToken CompressionAudio-Visual Reasoning

    Shanghai Jiao Tong University, Shanghai, China · Tongji University, Shanghai, China

  30. Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey

    Jul 21, 2025Jindong Li, Yali Fu, Jiahong Liu +5Large Language Model QuantizationMultimodal Large Language Models

    Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Jilin University, Changchun, China · The Chinese University of Hong Kong, Hong Kong, China +2

  31. Structured-Noise Masked Modeling for Video, Audio and Beyond

    Mar 20, 2025Aritra Bhowmik, Carlos Hinojosa, Fida Mohammad Thoker +2Noise-AwareNeural Audio

    University of Amsterdam · King Abdullah University of Science and Technology

  32. MAD: Modality-Adaptive Decoding for Mitigating Cross-Modal Hallucinations in Multimodal Large Language Models

    Date pendingSangyun Chung, Se Yeon Kim, Youngchae Chee +1Multimodal Large Language ModelsModality-Specific Encoders

    Integrated Vision Language Lab, KAIST, South Korea