Scientific Figure
Scientific figures are crucial for communicating complex research findings, but creating and understanding them remains challenging. Current research focuses on automating figure caption generation and synthesis using multimodal models, often incorporating transformer architectures like GPT variants and CLIP, along with techniques like cross-modal learning and knowledge augmentation from textual metadata within the associated scientific papers. These advancements aim to improve the accessibility and searchability of scientific information, reducing the time and effort required for researchers to create and interpret figures, ultimately accelerating scientific progress.
Papers
SciCapenter: Supporting Caption Composition for Scientific Figures with Machine-Generated Captions and Ratings
Ting-Yao Hsu, Chieh-Yang Huang, Shih-Hong Huang, Ryan Rossi, Sungchul Kim, Tong Yu, C. Lee Giles, Ting-Hao K. Huang
The Solution for the ICCV 2023 1st Scientific Figure Captioning Challenge
Dian Chao, Xin Song, Shupeng Zhong, Boyuan Wang, Xiangyu Wu, Chen Zhu, Yang Yang
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day
Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, Jianfeng Gao
FigGen: Text to Scientific Figure Generation
Juan A Rodriguez, David Vazquez, Issam Laradji, Marco Pedersoli, Pau Rodriguez