cs.AIOct 3, 2026
SaveLearning to Clarify Underspecified Intents Under Limited Interaction
Organizations: BITS Pilani & MIT · MIT · Dartmouth College
Abstract
AI assistants receive requests that leave out information needed for a good outcome, for example about users' preferences or goals. They must then either speculate or ask for more information before proceeding. We reconceptualize this as a value-of-information problem: the assistant should acquire information whose absence causes the greatest avoidable loss in user utility. This is rarely known ex ante; rather, assistants must predict it in order to optimally allocate limited user interactions. We instantiate this problem in image generation and derive a reinforcement learning framework using multi-turn simulated users to maximize utility recovery under uncertainty. In a preregistered study with 456 interactive sessions across 76 human participants, this helped users significantly better match reference images with significantly fewer questions, less total interaction time, and lower cost. This points toward a simple and scalable framework for training language model assistants to better disambiguate user intent by asking more informative questions.
Figures & tables
Figure 1. A simplified overview of our training pipeline on a single instance. A set of details is redacted from the full set of details , and the kept details are rewritten into a natural user request . The assistant asks the simulated user clarifying questions and composes, from and the answers, the description used to generate the image. The judge then decides which details in the assistant recovered by checking against their checks , giving the recovered utility . Training uses as reward over rollouts per instance, and the trained assistant is finally evaluated against the base model. Overview of the training pipeline. A complete reference specification is decomposed into a set of details. A subset of k details is hidden, and the remaining details are rewritten as the initial request x. The assistant receives that request and asks clarifying questions. A simulated user retains the full reference specification and supplies answers. The assistant combines the request and answers into a final description y, which is used for image generation. A judge checks which hidden details are present in y and computes their recovery score. Recovery scores from g rollouts of the same instance supply the training reward. The trained assistant is then compared with the base model. The information available to the simulated user and judge includes the reference details withheld from the assistant.
Figure 2. Recovered utility in simulated evaluation, measured as percentage of hidden details recovered by the final description. Points show estimated marginal means; bars are 95% source-image-clustered regression confidence intervals. Brackets compare trained checkpoints vs. base using source-image fixed effects and source-image-clustered standard errors. Asterisks denote BH-adjusted significance: , . Three panels comparing no-interaction, patient simulated users, and impatient simulated users. Recovery is low for all checkpoints without interaction with p<0.05 for base vs. impatient-trained. Both trained checkpoints have substantially higher recovery than base under fixed and open asking for both patient and impatient simulated users, all p<0.0001.
Figure 3. Change in cosine similarity across text/image comparisons in simulated evaluation, using complete descriptions and upscaled source images as references. Positive values indicate improvement over the initial request or its generated image. Points show estimated marginal means; bars are 95% source-image-clustered regression confidence intervals. Brackets show trained-vs.-base contrasts with source-image fixed effects and source-image-clustered uncertainty. BH-adjusted significance: , , , . 4 rows showing text-to-text, image-to-image, text-to-image, and image-to-text similarity gains across no-interaction and two simulated-user types. Trained checkpoints generally have larger gains under both asking policies. Patient-trained has the highest adjusted means across the asking comparisons.
Figure 4. Mean answered-question count by number of redacted details, , for each checkpoint and simulated-user type. Solid lines show fixed asking with budget and dashed lines show open asking with cap . Each point averages source-images with the relevant redaction count and impatient simulated users may terminate dialogue before the budget is exhausted. Six panels comparing base, patient-trained, and impatient-trained checkpoints with patient and impatient users. Patient-user fixed-asking counts are diagonal, because they equal the number of missing details. Base open asking stays near zero. Trained open asking produces more answers, with variable counts that generally remain below fixed asking.
Figure 5. Recovery gain vs. change in interaction measure under open asking. Each point compares trained vs. base checkpoints on the same source image and simulated-user type; gray lines mark no change. Even low-change examples often produce non-zero recovery gain, suggesting the gains are not only driven by more complexity in interaction. Six scatterplots arranged in three rows and two columns. Columns show patient and impatient simulated users. Every point is a trained-minus-base comparison for the same source image under open asking, patient-trained and impatient-trained. The vertical axis is recovery gain in percentage points, with ticks from -50 to 100. Horizontal axes show the change in answered questions in the top row, total question words in the middle row, and total answer words in the bottom row, with reference horizontal and vertical lines indicating zero change. Colored straight lines also summarize the association within each trained checkpoint. Most points lie above zero recovery gain and to the right of zero interaction change. All fitted lines slope upward, but points are widely dispersed around them. In the question-count panels, many examples with roughly 2–8 additional answers span recovery gains from near zero to 100 percentage points. Positive gains also occur at zero additional answers. A few negative recovery differences appear with impatient users near zero interaction change. Patient-trained comparisons extend furthest to the right.
Figure 6. Fidelity improvement (cosine-similarity gains) in human interactions across text/image comparisons, using reference descriptions and images. Points show estimated marginal means; bars are 95% participant-clustered regression confidence intervals. Brackets show trained-vs.-base contrasts adjusted for question count and source image, with participant-clustered standard errors. All contrasts favor the trained checkpoint. BH-adjusted significance: , . Four side-by-side point-interval plots of human-study cosine-similarity gains. From left to right, panels compare text with reference text, image with reference image, text with reference image, and image with reference text. Gray points represent base and teal points the patient-trained assistant. Vertical bars are 95% participant-clustered regression confidence intervals, and horizontal lines mark zero gain. The panels use separate vertical scales. The trained mean is higher in every panel, including a smaller decline in text–image similarity. Regression-adjusted trained-versus-base brackets have adjusted p < 0.001 for text–text and image–text, and < 0.01 for image–image and text–image.
Figure 7. Adjusted differences (trained - base) in human reference-similarity gains by question count. Estimates come from a regression using a natural cubic spline, with source-image fixed effects. Bars are 95% participant-clustered regression confidence intervals and the dashed line marks no difference. All shown count-by-outcome contrasts are BH-adjusted: , . Four side-by-side point-and-interval plots of the adjusted trained-minus-base difference in human-study cosine-similarity gain. Panels show text–text, image–image, text–image, and image–text comparisons. The horizontal axis is answered-question count, spanning 0 to 15, with estimates at counts observed in both conditions. Points are contrasts from the training-by-natural-cubic-spline model; vertical bars are 95% participant-clustered regression intervals. Dashed horizontal line marks no difference. Vertical scales differ across panels. Estimates at zero and one answer are near zero and have intervals crossing zero. At three to five answers, all four panels show positive contrasts with adjusted p < 0.05. The fitted profiles generally rise through the first few counts and then flatten or decline. Text–text and text–image estimates remain positive across more of the middle counts. At 15 answers, all point estimates are negative, but their intervals are much wider and cross zero. No estimate at that count has a significance mark.
Figure 8. Interaction measures from the human study. Points show estimated marginal means; bars are 95% participant-clustered regression confidence intervals. Brackets show trained-vs.-base regression-adjusted contrasts and all tests use participant-clustered standard errors. BH-adjusted significance: , , . Eight point-interval plots arranged in two rows of four, base vs. patient-trained assistant; bars are 95% participant-clustered regression confidence intervals. The top row shows questions answered, total question words, total answer words, and words per question. The bottom row shows words per answer, total dialogue words, total round time, and time per answer. Each panel has its own vertical scale. Adjusted means are 5.03 questions for base and 2.76 for trained. Question length increases from approximately 18 to 39 words, and answer length from 19 to 36 words. Adjusted total dialogue increases from 157 to 347 words per round; both question and answer word totals are higher with the trained assistant. Adjusted time per answer rises from approximately 49 to 72 seconds, while adjusted total round duration is similar, at approximately 489 and 491 seconds. Adjusted comparison brackets mark p < 0.0001 for count and word measures and below 0.001 for time per answer. Total round time difference is not significant.
Figure 9. Model-side token usage per human-study round. Points show estimated marginal means; bars are 95% participant-clustered regression confidence intervals. Brackets show regression-adjusted trained-vs.-base contrasts with participant-clustered standard errors. Results indicate that the trained model uses fewer reasoning and total sampled tokens but more visible-output tokens. All contrasts have ( ∗∗∗∗ ). Three side-by-side point-interval plots of mean assistant tokens per human-study round. Panels show reasoning tokens, visible-output tokens, and total sampled tokens for base vs. the patient-trained assistant, with 95% participant-clustered regression confidence intervals. Each panel uses its own vertical scale. Adjusted reasoning usage falls from approximately 4,597 tokens with base to 3,046 with trained. Adjusted visible-output usage increases from 273 to 403 tokens. Adjusted total sampled usage falls from 4,898 to 3,478 tokens. The reasoning reduction exceeds the visible-output increase. Confidence intervals are separated between conditions in all three panels. Brackets show regression-adjusted contrasts, all with adjusted p < 0.0001.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10. Generated images for six test instances, next to the reference photograph and the image from the request alone. Six rows of five images. In each row the reference photograph is followed by the request-only image, the base-model image and the two trained-assistant images. The request-only and base-model images miss the hidden subjects, such as the horse and buggy on a caution sign, the painted rock and the pink wall, and the two trained-assistant images bring them back.
Figure 11. One test episode of the patient-trained assistant in open-asking mode, from the reference image to the judge’s verdicts. A one-page walkthrough of one episode. The top shows a sunrise photograph over the ocean with two boats and its full description, then the request, which omits the sunrise, and twelve redacted details. The middle shows six questions from the assistant, each answered by the user with one or two details. The bottom shows the generated image, which now contains the sunrise and the boats, and twelve judge checks, ten marked satisfied and two not.
| Request |
| An outdoor medium shot looks up at a yellow, diamond-shaped caution sign attached to a large, dark vertical pole stretching from top to bottom. The sign displays shadows and light reflections. A light pole is in the bottom corner of the shot with two birds sitting on it during daytime. |
Table 1. The request and the hidden details of one test instance.
| Base model | Qwen3-8B, reasoning mode on |
|---|---|
| Adapter | LoRA, rank 32 |
| Optimizer | Adam, learning rate , , , |
| Regularization | no weight decay, no gradient clipping |
| Instances per iteration | 8 |
| Rollouts per instance | 32 |
| Asking mode in training | fixed-asking, |
Table 2. Training settings shared by the patient-trained and the impatient-trained assistant.
Figure 12. Mean training reward per iteration and validation recovery of the checkpoints saved every five iterations, for each run against the user it was trained with. Two panels with the training step from 0 to 300 on the horizontal axis and utility recovery from 0 to 100 percent on the vertical axis, one line per run. The training panel is noisy from iteration to iteration, rising from about 20 percent to about 70 percent by step 150 and then staying flat. The validation panel is smoother, rising from about 30 percent to about 75 percent for the patient-trained run and from about 22 percent to about 63 percent for the impatient-trained run, with the patient-trained line above the impatient-trained line throughout.
| Patient-trained | Impatient-trained | |
|---|---|---|
| Hidden details per answer | 1.06 | 0.74 |
| Answers with two or more hidden details | 31% | 20% |
| Stated details asked for specifically | 18% | 15% |
| Stated details asked for by topic | 81% | 83% |
| Stated details volunteered | 1% | 3% |
| Deflected answers | 5% | 15% |
Table 3. Audit of the simulator’s answers in fixed-asking mode under the patient user.
Figure 13. The round screen of the study, with the target on the left and the conversation on the right. Screenshot of the Picture Match round screen. A target photograph fills the left half under an instruction to get the assistant to generate an image as close to it as possible. The right half is a chat with the participant's request, one question from the assistant, an answer box, a send button and a button to stop and generate the image.
| Theme | Description |
|---|---|
| More questions asking what should be included | “What details should be included in the background elements”; “What specific elements should be included in the scene composition,”; “What should be included in the depiction of the elephant sculpture”, compared with “What color are the lanterns?”; “What color palette or textures should emphasize the urban atmosphere,”; “What time of day is it during daylight?”. |
| Fewer polar or binary questions | “What should be included in the background or any other elements”; “What should be included in the scene besides”; “What should be included in the foreground”, compared with “Is the translucent sunlight ray hitting the framed calligraphy”; “Is the toy car and horse positioned on”; “Are the castles and purple lions part of the flag’s design”. |
| More questions opening with What specific | “What specific details should be included in the visual style”; “What specific details should be included for the buildings”; “What specific elements should be included”, compared with “What atmosphere or time of day would you like”; “What time of year is this scene set?”; “Could you clarify which part of the bed”. |
| Less questions starting with Is or Are | “What elements should be included in the scene”; “What should be included in the scene”; “What should be included in the composition,”, compared with “Is the sun positioned directly above the main boat”; “Is the statue meant to represent a Roman or Greek figure”; “Is the phrase "u can" intended to be "you can"”. |
Table 4. Second-from-top-level themes of how the trained vs. base assistant’s questions differ.
| Theme | Description |
|---|---|
| Longer full sentences and descriptions | “Both cats are sitting upright looking forward toward the camera.”; “A high-resolution, clear, and detailed digital photograph emphasizing natural textures, vibrant colors,”; “The boat is a fishing boat. There is only rippling water in”, compared with “Brown table”; “about the same shade of bright lite green”; “Looks like a red container”. |
| Less hedging and speculative wording | “The background is medium colored wood”; “The foreground includes a paved sidewalk and a dark metal fence”; “two dark brick chimneys rising from the roof”, compared with “Im not sure but maybe the left”; “I think maybe gothic revival”; “It looks like there could be a chimney”. |
| More spelling mistakes and typos | “bright perhaps flourescent.”; “liver shiny lok handle attached to the fence”; “Th walls are a deep”, compared with “I purple dial lock behind the pink main one.”; “stationary decorative sculpture.”; “standalone carousel rooster sculpture displayed on its own,”. |
| Longer descriptions using complete sentences | “The composition is almost entirely filled with dense, layered tree branches”; “The left side of the house has a projecting bay section”; “The mattress itself is covered with white soft bed sheets and comforter,”, compared with “warm lights, orange/yellow”; “light blue”; “Bed head rest and the front bed edge”. |
Table 5. Second-from-top-level themes of how participants’ answers to the trained vs. base assistant differ.
| Dependent Variable: | Equal-weighted recovery | |||||
|---|---|---|---|---|---|---|
| Specification | Additive | Two-way | Three-way | + Count | + Q. length | + A./output |
| Model: | (1) | (2) | (3) | (4) | (5) | (6) |
| Variables | ||||||
| Patient-trained | 0.496 ∗∗∗∗ | 0.502 ∗∗∗∗ | 0.506 ∗∗∗∗ | 0.506 ∗∗∗∗ | 0.391 ∗∗∗∗ | 0.277 ∗∗∗∗ |
| (0.024) | (0.033) | (0.034) | (0.034) | (0.048) | (0.041) | |
| Impatient-trained | 0.455 ∗∗∗∗ | 0.475 ∗∗∗∗ | 0.487 ∗∗∗∗ | 0.487 ∗∗∗∗ | 0.318 ∗∗∗∗ | 0.219 ∗∗∗∗ |
Table 6. Regressions of recovered utility in the simulated evaluation.
| Dependent Variables: | Text–text | Image–image | ||||
|---|---|---|---|---|---|---|
| Specification | Additive | Three-way | + A./output | Additive | Three-way | + A./output |
| Model: | (1) | (2) | (3) | (4) | (5) | (6) |
| Variables | ||||||
| Patient-trained | 0.171 ∗∗∗∗ | 0.157 ∗∗∗∗ | 0.120 ∗∗∗∗ | 0.053 ∗∗∗∗ | 0.050 ∗∗∗∗ | 0.032 ∗∗∗ |
| (0.014) | (0.015) | (0.016) | (0.006) | (0.007) | (0.008) | |
| Impatient-trained | 0.100 ∗∗∗∗ | 0.112 ∗∗∗∗ | 0.090 ∗∗∗∗ | 0.029 ∗∗∗∗ | 0.032 ∗∗∗∗ | 0.017 |
Table 7. Regressions of the text-to-text and image-to-image similarity gains in the simulated evaluation.
| Dependent Variables: | Text–image | Image–text | ||||
|---|---|---|---|---|---|---|
| Specification | Additive | Three-way | + A./output | Additive | Three-way | + A./output |
| Model: | (1) | (2) | (3) | (4) | (5) | (6) |
| Variables | ||||||
| Patient-trained | 0.066 ∗∗∗∗ | 0.057 ∗∗∗∗ | 0.043 ∗∗∗∗ | 0.079 ∗∗∗∗ | 0.074 ∗∗∗∗ | 0.046 ∗∗∗∗ |
| (0.007) | (0.007) | (0.008) | (0.008) | (0.008) | (0.009) | |
| Impatient-trained | 0.042 ∗∗∗∗ | 0.044 ∗∗∗∗ | 0.038 ∗∗∗∗ | 0.054 ∗∗∗∗ | 0.059 ∗∗∗∗ | 0.033 ∗∗ |
Table 8. Regressions of the text-to-image and image-to-text similarity gains in the simulated evaluation.
| Dependent Variables: | Questions | Question words | Answer words | Output words | ||
|---|---|---|---|---|---|---|
| Specification | Additive | Three-way | Additive | Three-way | ||
| Model: | (1) | (2) | (3) | (4) | (5) | (6) |
| Variables | ||||||
| Patient-trained | 2.81 ∗∗∗∗ | 218.9 ∗∗∗∗ | 219.3 ∗∗∗∗ | 82.1 ∗∗∗∗ | 82.4 ∗∗∗∗ | 4.05 |
| (0.201) | (14.8) | (16.4) | (4.62) | (5.77) | (3.30) | |
| Impatient-trained | 1.63 ∗∗∗∗ | 218.0 ∗∗∗∗ | 314.4 ∗∗∗∗ | 50.1 ∗∗∗∗ | 54.0 ∗∗∗∗ | 130.7 ∗∗∗∗ |
Table 9. Regressions of the dialogue measures in the simulated evaluation.
| Outcome | Questions | Question words | Round seconds | Sampled tokens |
|---|---|---|---|---|
| Model: | (1) | (2) | (3) | (4) |
| Variables | ||||
| Trained model | -2.27 ∗∗∗∗ | 111.9 ∗∗∗∗ | 2.00 | -1,420.4 ∗∗∗∗ |
| (0.277) | (12.2) | (30.8) | (137.9) | |
| Questions answered | 31.3 ∗∗∗∗ | 46.2 ∗∗∗∗ | 702.5 ∗∗∗∗ | |
| (2.48) | (6.10) | (28.6) |
Table 14. Human-study regressions, process measures.
Figure 14. Mean recovery on the test split by the number of hidden details, for each assistant, asking mode and simulated user. Four panels, fixed and open asking by patient and impatient user, with mean recovery against the number of missing details from 1 to 15. Under fixed asking the two trained lines rise from about 35 percent at one detail to about 90 percent at fifteen, while the base line stays between 10 and 45 percent. Under open asking the trained lines are noisier, between 30 and 100 percent, and the base line stays mostly below 25 percent.
Figure 15. Share of test episodes reaching each answered turn under fixed asking and open asking, for each assistant and simulated user. Six panels, three assistants by two simulated users, each showing the share of episodes reaching each answered turn as a step curve for fixed and open asking. Fixed-asking curves decline steadily to zero by turn 16. The base model's open-asking curve starts below 30 percent and reaches zero by turn 3. The trained open-asking curves start above 80 percent and fall below the fixed-asking curves, the impatient-trained one earliest.
More in this topic
Human-AI Interaction
Human-AI interaction studies how people and AI systems communicate, collaborate, make decisions, and shape one another over time.
See all 330 papers