We propose CoLyricist, an AI-assisted lyric writing tool designed to support the typical workflows of experienced lyricists and enhance their creative efficiency. While lyricists have unique processes, many follow common stages. Tools that fail to accommodate these stages challenge integration into creative practices. Existing research and tools lack sufficient understanding of these songwriting stages and their associated challenges, resulting in ineffective designs. Through a formative study involving semi-structured interviews with 10 experienced lyricists, we identified four key stages: Theme Setting, Ideation, Drafting Lyrics, and Melody Fitting. CoLyricist addresses these needs by incorporating tailored AI-driven support for each stage, optimizing the lyric writing process to be more seamless and efficient. To examine whether this workflow-aligned design also benefits those without prior experience, we conducted a user study with 16 participants, including both experienced and novice lyricists. Results showed that CoLyricist enhances the songwriting experience across skill levels. Novice users especially appreciated the Melody-Fitting feature, while experienced users valued the Ideation support.
Figures & tables
Figure 1. CoLyricist is an AI-assisted lyric writing tool designed to align with lyricists’ typical workflows. The process involves the following steps: Users upload a melody and specify the song’s theme (1).They then brainstorm ideas based on the theme, using the AI-Assisted Brainstorm function to generate relevant phrases and the Rhyming Recommendation function to get relevant rhyme suggestions. They can store these idea in a keyword/key-phrase memo box (2). Next, users draft lyrics, either manually or by using draft lyrics generation functions (3), which the output is influenced by both the theme and stored keywords. Users can use the Auto Melody-Fitting feature to adjust the draft lyrics to fit the melody line by line (4), while preserving their original meaning. Users can also use rhyming recommendations to further polish their lyrics (5). The tool supports an iterative process, enabling continuous refinement with AI assistance. Screenshot of the CoLyricist user interface illustrating a workflow-aligned lyric writing process. The interface is divided into three main panels. On the left is the Settings Panel, where users upload a melody and specify the song theme and keywords, corresponding to the theme-setting stage (1). Below it is the Ideation Panel, which displays AI-assisted brainstorming phrases and thematically relevant rhyming suggestions that users can save as key phrases, corresponding to the ideation stage (2). On the right is the Editing Panel, where users draft lyrics and view musical notation aligned with the uploaded melody. Users can generate draft lyrics either for the full song or line by line, influenced by the selected theme and saved keywords, representing the drafting stage (3). An auto melody-fitting function allows users to adjust draft lyrics to fit the melody line by line, producing multiple candidate lyrics aligned with musical notes (4). Rhyming recommendations are also shown alongside the drafts to support further refinement during revision (5). Numbered markers, arrows, and dashed lines indicate an iterative workflow in which users repeatedly move between ideation, drafting, and melody fitting with AI assistance.
Figure 2. High-level typical lyric writing workflow, comprising four primary stages: Theme Setting, Ideation, Lyrics Drafting, and Melody Fitting. Gray boxes represent stage outputs, while blue boxes denote actions. Diagram illustrating a high-level lyric writing workflow composed of four stages arranged vertically. At the top is “Theme Setting,” which receives input from the melody and produces a theme. To the right is “Ideation,” which takes the theme as input and generates keywords. Below is “Lyrics Drafting,” where draft lyrics are created based on the theme and keywords. At the bottom is “Melody Fitting,” where draft lyrics are adjusted to fit the melody, producing final lyrics. Blue boxes represent user actions or workflow stages, while gray boxes represent intermediate or final outputs. Solid arrows indicate the primary progression from theme setting to ideation, drafting, and melody fitting. Dotted arrows indicate iterative feedback loops, showing that users can revisit earlier stages such as ideation or drafting after melody fitting.
Figure 3. The first-half workflow: Step 0 (uploading melody information), Step 1 (setting the theme), and Step 2 (ideation). Annotated screenshot of the CoLyricist interface illustrating the first half of the lyric writing workflow. On the left is the Settings panel, where Step 0 highlights the melody upload field used to input melody information. Below it, Step 1 highlights the theme input field, where users enter a short sentence describing the main theme of the song. An optional keyword or key-phrase input field is also shown, indicating that users may specify words they wish to incorporate into the lyrics. On the right is the ideation area corresponding to Step 2, which includes two AI-assisted features: the AI Brainstorm panel that generates several thematically relevant phrases from a given keyword or phrase, and the Rhyming Recommendation panel that allows users to input a target word and syllable count to obtain rhyming word suggestions. Arrows, colored outlines, and textual annotations visually connect each interface element to its corresponding workflow step.
Figure 4. The rest half of the workflow: Step 3 (creating the draft lyric) and Step 4 (melody fitting) Annotated screenshot of the CoLyricist interface illustrating the second half of the lyric writing workflow. Step 3 shows the draft lyrics creation stage, where users can generate a full-length draft based on the provided melody, theme, and keywords, or generate draft lyrics line by line using a generate button for each line. Each draft line is displayed alongside the corresponding melody staff, and users can listen to the melody playback. Step 4 shows the melody fitting stage, where users can manually edit draft lyrics to better match the melody or use the Fit to Melody function to receive three to five AI-generated lyric revisions aligned with the melody. The revised lyric candidates are presented in a list, each with audio playback controls, allowing users to listen to and select the desired version. Colored outlines, arrows, and annotations indicate the relationship between user actions, AI-assisted features, and the corresponding workflow steps.
Figure 5. Questionnaire overall evaluation results comparing experienced and novice users’ responses to CoLyricist. The evaluation contains 8 criteria: helpfulness, reuse intent, writing speed impact, satisfaction with output quality, creativity stimulation, ease of use, enjoyment, and sense of ownership. Responses are categorized on a 5-point Likert scale from "Strongly Disagree" to "Strongly Agree". Both user groups generally responded positively, with high agreement on the tool’s helpfulness, potential for reuse, and ability to speed up lyric writing. Grouped horizontal stacked bar charts showing overall questionnaire responses from experienced and novice users evaluating CoLyricist across eight criteria: helpfulness, reuse intent, writing speed impact, satisfaction with output quality, creativity stimulation, ease of use, enjoyment, and sense of ownership. Each criterion is represented by a pair of bars corresponding to experienced users and novice users. Bar segments are color-coded to indicate response categories on a five-point Likert scale, ranging from strongly disagree to strongly agree, as shown in the legend. The length of each colored segment represents the proportion of responses in that category for the corresponding criterion and user group, allowing visual comparison of response distributions between experienced and novice participants.
Figure 6. Questionnaire result for each feature. The chart displays ratings for six key functions of an AI-assisted lyric writing tool: Brainstorming, Rhyming Dictionary, Generate All Draft, Generate Single Draft, Auto Melody Fitting, and Singing Voice Synthesis. Responses from experienced and novice users are categorized on a 5-point Likert scale from "Strongly Disagree" to "Strongly Agree," with an additional option for "I did not use this function." The data suggests that most features were perceived as helpful, with variations between experienced and novice users in their utilization and appreciation of specific functions. A set of horizontal stacked bar charts showing questionnaire responses evaluating the perceived helpfulness of six individual features of CoLyricist: Brainstorming, Rhyming Dictionary, Generate All Draft, Generate Single Draft, Auto Melody Fitting, and Singing Voice Synthesis. For each feature, two bars represent responses from experienced users and novice users. Bar segments are color-coded to indicate response categories, including an option for not using the function, followed by a five-point Likert scale ranging from strongly disagree to strongly agree, as shown in the legend. The length of each colored segment corresponds to the proportion of participants selecting that response, enabling visual comparison of feature usage and perceived helpfulness between experienced and novice users across the different functions.
Figure 7. User Created Lyrics Assessment results. The box plot displays normalized scores for four criteria: Melody Alignment, Creativity, Coherence, and Overall quality. The result suggests variations in performance between novice and experienced users across the different assessment criteria is not significant. Box plots showing normalized assessment scores of user-created lyrics for four evaluation criteria: Melody Alignment, Creativity, Coherence, and Overall quality. For each criterion, two box plots represent scores from novice users and experienced users, as indicated by the legend. The vertical axis shows normalized scores per participant, while each box depicts the distribution of scores, including the median, interquartile range, whiskers, and outliers. This visualization allows comparison of score distributions between novice and experienced participants across the four assessment criteria.
Figure 8. Examples of lyrics showing differences in melody alignment. (a) Lyrics by an experienced participant with high scores (E1: Good, E2: Good), demonstrating strong prosody and effective rhyming. (b) Lyrics by a novice participant with lower scores (E1: Poor, E2: Very Poor), featuring misaligned key words and mismatched syllable counts. Side-by-side musical score excerpts with lyrics illustrating examples of melody-aligned lyric writing by different participants. The left panel shows lyrics created by an experienced participant, where syllables are closely aligned with note durations and melodic structure across multiple phrases. The right panel shows lyrics created by a novice participant, where syllable placement and word segmentation differ across notes, resulting in less consistent alignment with the melody. Each excerpt presents staff notation with tempo markings and lyrics placed beneath the notes, allowing visual comparison of how lyrical content is mapped onto the same melodic framework by experienced and novice users.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9. The user interface of CoLyricist includes: (A) Settings panel for configuring the song’s melody, theme, and other parameters; (B) Ideation panel for assisting in brainstorming and providing rhyming recommendations; (C) Editing panel for drafting and refining lyrics, where AI provides automated revised lyrics that fit with the melody. Annotated overview of the CoLyricist user interface divided into three main panels. Panel A on the left is the Settings panel, where users configure core inputs such as melody upload, song title, theme, and optional keywords, and can generate a full-length draft based on these inputs. Panel B in the center is the Ideation panel, which includes an AI-assisted Brainstorm feature for generating thematically related phrases from a given keyword or phrase, and a Rhyming Recommendation feature that provides rhyming word suggestions based on a target word and specified syllable count. Panel C on the right is the Editing panel, where users draft lyrics line by line, view the corresponding musical staff, listen to melody playback, and request AI-assisted revisions that fit the melody. Highlighted sublabels and callouts indicate specific interactive elements, such as generation buttons, melody fitting actions, and lists of revised lyric candidates, illustrating how users move between configuration, ideation, and iterative lyric refinement.
Figure 10. Selected Lyrics 1: Created by novice participants Musical score excerpt with lyrics showing a song created by a novice participant. The figure presents staff notation with a tempo marking at the top, followed by multiple measures of melody with lyrics aligned beneath the notes. Each line displays syllables segmented across notes to match the melodic rhythm, illustrating how the generated lyrics are mapped onto the melody throughout the song. This example serves as a representative output of lyrics created by novice users using the CoLyricist system.
Figure 11. Selected Lyrics 2: Created by experienced participants Musical score excerpt with lyrics showing a song created by a experienced participant. The figure presents staff notation with a tempo marking at the top, followed by multiple measures of melody with lyrics aligned beneath the notes. Each line displays syllables segmented across notes to match the melodic rhythm, illustrating how the generated lyrics are mapped onto the melody throughout the song. This example serves as a representative output of lyrics created by experienced users using the CoLyricist system.
Generative AI may enable new forms of context-aware creative expression by dynamically tailoring media content to its consumption context. For instance, AI systems could adapt song lyrics to the listener and their current activity. However, existing media adaptation systems primarily optimize for audience experience, often neglecting artists' intent, style, and preference. We address this challenge by introducing a novel creator-centered approach to adaptive media authoring and present MultiVerse, a system that instantiates this approach for steering adaptive lyrics. Our approach allows creators to explicitly author controls based on their intent, lyric structure, and audience context, and uses rule-based validations to ensure controls are followed. We conducted a study with 10 songwriters, comparing MultiVerse with a prompting-based workflow for composing adaptive lyrics. The comparison revealed that creators preferred to author how lyrics adapt by explicitly specifying relevant context and adaptation constraints, while recognizing trade-offs in flexibility and iteration speed. Interviews further revealed that creators viewed adaptive media as enabling new forms of audience connection, introducing a distinct creative process, favoring new compositional strategies, and reshaping notions of authorship.
Alexander Wang, Chris Donahue, David Lindlbauer
Human-Computer Interaction Institute Carnegie Mellon University Pittsburgh, Pennsylvania, USA · Computer Science Department Carnegie Mellon University Pittsburgh, Pennsylvania, USA
Artificial intelligence is increasingly being integrated into professional audio production workflows, yet a gap persists between the tools developers produce and the requirements of practising sound designers. This paper investigates this gap through a mixed-methods study comprising a survey of 76 practitioners and follow-up semi-structured interviews with 20 industry professionals. Results were analysed using descriptive statistical analysis and thematic analysis to identify patterns across both datasets. Five themes emerged from our analysis: Context, Workflow, Potential, Risks, and Right Use. Our work indicates that current AI tools perform adequately in fast-consumption media contexts but lack the narrative sophistication required for high-end sound design (films, immersive experiences etc). Practitioners demonstrate a preference for assistive, task-specific applications, particularly in audio restoration and library management, over end-to-end generative systems. This work contributes to the on-going discussion on the use of AI and AI-enhanced tools in the creative industries. We report on the current status of the field from the point of view of sound designers and creative audio practitioners, and offer a set of recommendation for sound technologist and developers based on our findings to guide the development of more informed AI tools for sound design.
This paper examines a use of AI in creative practice as an interpretive sounding board for human-generated material, rather than the more familiar pattern of AI generation followed by human curation. Through the lens of resonance as theorized by Hartmut Rosa, I present a first-person case study of songwriting from July 2025 to March 2026, drawing on 16 original pieces in English, French, and other languages along with piano solos. I describe a configuration in which resonance is not located between user and model, but in the author's deepening contact with their own material, mediated through the model. This kind of resonance was supported rather than inhibited by AI when sounding-board behavior was cultivated through sustained calibration by the user. Two failure modes appeared when calibration was absent: sycophantic drift and magical overinterpretation. This account suggests both the potential and the risks of AI as an interpretive partner in creative practice.
Xiao Xiao
Institute for Future Technologies De Vinci Research Center, De Vinci Higher Education Paris, France · MIT Media Lab Cambridge, MA, USA