Design is a non-linear, reflective process in which practitioners engage with visual, semantic, and other expressive materials to explore, iterate, and refine ideas. As Generative AI (GenAI) becomes integrated into professional design practice, traditional interaction approaches focusing on prompts or whole-image manipulation can misalign AI output with designers' intent, forcing visual thinkers into verbal reasoning or post-hoc adjustments. We present three interaction approaches from DesignPrompt, FusAIn, and DesignTrace that distribute control across intent, input, and process, enabling designers to guide AI alignment at different stages of interaction. We further argue that alignment is a dynamic negotiation, with AI adopting proactive or reactive roles according to designers' instrumental and inspirational needs and the creative stage.
Figures & tables
Figure 1. DesignPrompt supports intent decomposition through multiple input modalities: (A) image inpainting, (B) color, and (C) semantic tags. Multimodal inputs are mapped to editable text prompts, enabling interactive prompt editing and reordering to refine textual input to GenAI.
Figure 2. FusAIn reifies image-based visual prompts as personalized “pens” that encode specific visual properties. With “texture pens” users extract a texture from a source image, load it into a pen, and draw to apply that texture onto target visuals. The system then generates an image that preserves other properties while consistently applying the texture.
Figure 3. DesignTrace: a GenAI design tool for exploring alternatives and tracking design progress. Designers can extract semantic attributes from inspirational images and system suggestions, apply them to selected regions or entire images, branch multiple alternatives under shared semantic categories, customize prompts for image generation, make localized edits while navigating editing history, and collapse unused versions.
Facultad de Inform´atica Universidad Complutense de Madrid Madrid, Spain · Universitat Polit`ecnica de Catalunya Villanova i la Geltru, Spain · Instituto de Tecnolog´ıa Del Conocimiento Universidad Complutense de Madrid Madrid, Spain