Label Attention

Label attention mechanisms enhance machine learning models by focusing on the most relevant parts of input data for each predicted label, improving both accuracy and interpretability. Current research emphasizes developing label attention models for multi-label classification tasks, particularly in complex domains like medical coding and natural language processing, often integrating these mechanisms within transformer architectures or other deep learning frameworks. This focus on interpretability is crucial for building trust in AI systems, especially in high-stakes applications like healthcare, where understanding model decisions is paramount for clinical decision-making. Improved accuracy and explainability through label attention are driving advancements across various fields.

Papers

October 31, 2024

Beyond Label Attention: Transparency in Language Models for Automated Medical Coding via Dictionary Learning
John Wu, David Wu, Jimeng Sun
Language Model Transparency Index Dictionary Learning Medical Coding Label Attention Medical Code Prediction

October 30, 2024

Don't Just Pay Attention, PLANT It: Transfer L2R Models to Fine-tune Attention in Extreme Multi-Label Text Classification
Debjyoti Saharoy, Javed A. Aslam, Virgil Pavlu
Human Attention Attention Layer Extreme Multi Label Label Attention

September 16, 2024

DILA: Dictionary Label Attention for Mechanistic Interpretability in High-dimensional Multi-label Medical Coding Prediction
John Wu, David Wu, Jimeng Sun
Mechanistic Interpretability Label Prediction Label Attention Dense Embeddings Medical Code Prediction

July 8, 2024

SLIM: Spuriousness Mitigation with Minimal Human Annotations
Xiwei Xuan, Ziquan Deng, Hsuan-Tien Lin, Kwan-Liu Ma
Spurious Correlation Selective Annotation Spurious Feature Label Attention

December 10, 2023

MISCA: A Joint Model for Multiple Intent Detection and Slot Filling with Intent-Slot Co-Attention
Thinh Pham, Chi Tran, Dat Quoc Nguyen
Intent Detection Joint Modeling Slot Filling Slot Attention Label Attention Intent Taxonomy

September 16, 2023

MHLAT: Multi-hop Label-wise Attention Model for Automatic ICD Coding
Junwen Duan, Han Jiang, Ying Yu
Language Model Multi Hop ICD Code International Classification of Disease Informative Representation Label Attention

July 24, 2023

Integration of Frame- and Label-synchronous Beam Search for Streaming Encoder-decoder Speech Recognition
Emiru Tsunoo, Hayato Futami, Yosuke Kashiwagi, Siddhant Arora, Shinji Watanabe
Automatic Speech Recognition Speech Recognition Indoor Wall Frame Installation Beam Search Accurate Decoding Label Attention

April 24, 2023

MixPro: Data Augmentation with MaskMix and Progressive Attention Labeling for Vision Transformer
Qihao Zhao, Yangyu Huang, Wei Hu, Fan Zhang, Jun Liu
Vision Transformer Data Augmentation Mixed Attention Label Attention

April 8, 2023

Co-attention Propagation Network for Zero-Shot Video Object Segmentation
Gensheng Pei, Yazhou Yao, Fumin Shen, Dan Huang, Xingguo Huang, Heng-Tao Shen
Optical Flow Cross Modal Feature Label Attention Zero Shot Video Object Segmentation

March 1, 2023

Label Attention Network for Temporal Sets Prediction: You Were Looking at a Wrong Self-Attention
Elizaveta Kovtun, Galina Boeva, Andrey Shulga, Alexey Zaytsev
Self Attention Multi Label Classification Sequential Data Multi Label Recognition Label Attention

November 15, 2022

An Automatic ICD Coding Network Using Partition-Based Label Attention
Daeseong Kim, Haanju Yoo, Sewon Kim
International Classification of Disease Label Attention

October 16, 2022

This Patient Looks Like That Patient: Prototypical Networks for Interpretable Diagnosis Prediction from Clinical Text
Betty van Aken, Jens-Michalis Papaioannou, Marcel G. Naik, Georgios Eleftheriadis, Wolfgang Nejdl, Felix A. Gers, Alexander Löser
Clinical Text Patient Friendly Language Prototypical Network Patient Specific Diagnosis Prediction Label Attention

January 18, 2022

Label Dependent Attention Model for Disease Risk Prediction Using Multimodal Electronic Health Records
Shuai Niu, Qing Yin, Yunya Song, Yike Guo, Xian Yang
Language Model Disease Prediction Label Prediction Clinical Corpus Multimodal Electronic Health Record Label Attention

Label Attention

Papers

Beyond Label Attention: Transparency in Language Models for Automated Medical Coding via Dictionary Learning

Don't Just Pay Attention, PLANT It: Transfer L2R Models to Fine-tune Attention in Extreme Multi-Label Text Classification

DILA: Dictionary Label Attention for Mechanistic Interpretability in High-dimensional Multi-label Medical Coding Prediction

SLIM: Spuriousness Mitigation with Minimal Human Annotations

MISCA: A Joint Model for Multiple Intent Detection and Slot Filling with Intent-Slot Co-Attention

MHLAT: Multi-hop Label-wise Attention Model for Automatic ICD Coding

Integration of Frame- and Label-synchronous Beam Search for Streaming Encoder-decoder Speech Recognition

MixPro: Data Augmentation with MaskMix and Progressive Attention Labeling for Vision Transformer

Co-attention Propagation Network for Zero-Shot Video Object Segmentation

Label Attention Network for Temporal Sets Prediction: You Were Looking at a Wrong Self-Attention

An Automatic ICD Coding Network Using Partition-Based Label Attention

This Patient Looks Like That Patient: Prototypical Networks for Interpretable Diagnosis Prediction from Clinical Text

Label Dependent Attention Model for Disease Risk Prediction Using Multimodal Electronic Health Records