Graphical User Interface

Graphical User Interfaces (GUIs) are the visual interfaces through which humans interact with computers, and research focuses on improving their design, usability, and automation. Current research emphasizes using multimodal large language models (LLMs) and graph neural networks to understand and generate GUIs, often incorporating computer vision techniques for image analysis and action prediction. This work aims to improve GUI design, automate tasks through GUI interaction, and enhance accessibility for users with disabilities, impacting software development, user experience, and assistive technologies.