Unified Textual Representation
Unified textual representation aims to create a single, consistent format for representing information from diverse sources, such as text, images, tables, and graphs, enabling seamless cross-modal understanding and processing. Current research focuses on developing methods to achieve this unification, often leveraging large language models and contrastive learning techniques, and exploring architectures like masked autoencoders and transformer-based models to learn robust and transferable representations. This work is significant because it facilitates the development of more powerful and versatile AI systems capable of handling complex, multi-modal data, with applications ranging from improved question answering and information retrieval to enhanced knowledge extraction and summarization.
Papers
A Pure Transformer Pretraining Framework on Text-attributed Graphs
Yu Song, Haitao Mao, Jiachen Xiao, Jingzhe Liu, Zhikai Chen, Wei Jin, Carl Yang, Jiliang Tang, Hui Liu
Converging Dimensions: Information Extraction and Summarization through Multisource, Multimodal, and Multilingual Fusion
Pranav Janjani, Mayank Palan, Sarvesh Shirude, Ninad Shegokar, Sunny Kumar, Faruk Kazi