Sketch Based Visual Understanding

Sketch-based visual understanding aims to enable computers to interpret hand-drawn sketches, bridging the gap between human creativity and machine perception. Current research focuses on improving the generalization capabilities of models, often leveraging large pre-trained models like CLIP or employing diffusion models guided by U-Net architectures for sketch-to-image synthesis. These advancements address challenges in handling sketch variability, including different abstraction levels and limited training data, and contribute to improved accuracy in sketch recognition and generation, with applications in image retrieval, editing, and 3D modeling.