LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators

Riccardo De Santi; Federico Di Gennaro; Angelo Gnazzo; Sophia Tang; Pranam Chatterjee; Fanny Yang; İsmail İlkan Ceylan; Michael M. Bronstein

cs.LGOct 12, 2026

LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators

Authors: Riccardo De Santi, Federico Di Gennaro, Angelo Gnazzo, Sophia Tang, Pranam Chatterjee, Fanny Yang, İsmail İlkan Ceylan, Michael M. Bronstein,

Organizations: ETH Zurich · ETH AI Center · Technische Universität Wien · AITHYRA ·

Abstract

Generative discovery in science requires coupling high-level scientific reasoning with the ability to generate samples respecting the structure of a design space. Yet these capabilities remain largely separated: foundation LLMs encode broad chemical and biological knowledge, while domain-specific flow- and diffusion-based generators capture the priors needed to produce valid sequences and geometries. We study how to align such scientific generators to high-level natural-language discovery goals without requiring an explicit proxy reward, which is often unavailable or poorly captured by learned predictors in experimental sciences. We introduce LLM-to-Flow Distillation (LFD), where a frozen LLM acts as an adaptive teacher for a domain-specific generator. Given a goal such as 'generate peptides with promising anticancer activity', the teacher constructs a curriculum of intermediate sub-goals aware of the flow's current capabilities, and progressively directs them toward the original discovery goal. At each round, the LLM judges sampled candidates under the current objective. These judgments induce preferences that are iteratively distilled into the generator through DPO. This procedure provides dense learning signals even when the initial model places negligible probability on designs satisfying the final goal, enabling adaptation toward poorly covered, out-of-distribution regions. Experiments on therapeutic peptides and small-molecule structure design demonstrate that LFD enables a new form of generative control: aligning arbitrary domain-specific scientific generators to scientific discovery tasks expressed via high-level natural-language goals.

Figures & tables

Figure 1: We extend the capabilities of current fine-tuning methods for specialized scientific generators (e.g., flow models), allowing them to align to natural-language goals.

Figure 2: (left): LLM $\leftrightarrow$ Flow interaction loop in LFD . (right): LFD progressively redistributes model density toward the discovery region $\Omega_{g}$ induced by the natural-language discovery goal $g$ .

Figure 3: (top) LLM estimated scores tend to be far from the ground truth (

y=x

line). (bottom) Pairwise accuracy of four elicitation mechanisms can provide a learning signal (dashed line at

50\%

Panel of Figure 4 — Figure 4: Rejected peptide LLM trace. LLM reports rationale and possible sources of uncertainty.

	Therapeutic Peptides (CPP)			Therapeutic Peptides (Anti-Cancer)
Method	pLM4CPPs [-1pt] ( $\%$ ) $\uparrow$	HemoPI2 [-1pt] ( $\%$ ) $\downarrow$	Distance [-1pt] from prior $\uparrow$	AntiCP2-alt. [-1pt] ( $\%$ ) $\uparrow$	HemoPI2 [-1pt] ( $\%$ ) $\downarrow$	Distance [-1pt] from prior $\uparrow$
Pre-trained	$8.81$ $\scriptstyle\,\pm\,3.20$	$40.84$ $\scriptstyle\,\pm\,1.96$	$0.06$ $\scriptstyle\,\pm\,0.04$	$26.43$ $\scriptstyle\,\pm\,2.91$	40.85 $\scriptstyle\,\pm\,1.89$	$0.00$ $\scriptstyle\,\pm\,0.00$
DPO	$10.39$ $\scriptstyle\,\pm\,3.19$	$40.49$ $\scriptstyle\,\pm\,1.92$	$0.07$ $\scriptstyle\,\pm\,0.04$	$25.29$ $\scriptstyle\,\pm\,3.19$	$42.77$ $\scriptstyle\,\pm\,1.81$	$0.08$ $\scriptstyle\,\pm\,0.04$
DiffusionNFT	$57.04$ $\scriptstyle\,\pm\,6.22$	$38.87$ $\scriptstyle\,\pm\,1.62$	$0.29$ $\scriptstyle\,\pm\,0.03$	$33.26$ $\scriptstyle\,\pm\,2.76$	$45.61$ $\scriptstyle\,\pm\,2.39$	$0.16$ $\scriptstyle\,\pm\,0.03$
Guidance	$8.49$ $\scriptstyle\,\pm\,3.30$	$43.45$ $\scriptstyle\,\pm\,1.85$	$0.06$ $\scriptstyle\,\pm\,0.04$	$35.94$ $\scriptstyle\,\pm\,2.33$	$47.25$ $\scriptstyle\,\pm\,1.81$	$0.29$ $\scriptstyle\,\pm\,0.03$
LFD	69.06 $\scriptstyle\,\pm\,6.36$	37.02 $\scriptstyle\,\pm\,1.69$	0.34 $\scriptstyle\,\pm\,0.03$	53.54 $\scriptstyle\,\pm\,3.32$	$42.48$ $\scriptstyle\,\pm\,2.04$	0.31 $\scriptstyle\,\pm\,0.03$

Table 1: LFD achieves the strongest held-out activity on both peptide tasks while keeping low hemolysis, and showing the largest distance from the pre-trained amino-acid distribution ( Dai et al., 2025 ) .

Figure 5: LFD generates online a sequence of scientist-interpretable intermediate sub-goals.

Figure 6: LFD -generated molecule docking to D2 Dopamine receptor, showing multiple close interactions (pink labels) below 3.5 Å (GNINA).

	Small Molecules (D2 Receptor)			Small Molecules (Kinase-3 Beta)
Method	GNINA $\uparrow$	Vina $\downarrow$	rDock $\downarrow$	GNINA $\uparrow$	Vina $\downarrow$	rDock $\downarrow$
Pre-trained	$6.11$ $\scriptstyle\,\pm\,0.33$	$-7.52$ $\scriptstyle\,\pm\,0.39$	$-7.64$ $\scriptstyle\,\pm\,1.33$	$5.80$ $\scriptstyle\,\pm\,0.17$	$-7.32$ $\scriptstyle\,\pm\,0.19$	$-8.51$ $\scriptstyle\,\pm\,0.55$
DPO	$5.60$ $\scriptstyle\,\pm\,0.05$	$-6.81$ $\scriptstyle\,\pm\,0.08$	$-8.11$ $\scriptstyle\,\pm\,0.36$	$6.29$ $\scriptstyle\,\pm\,0.17$	$-7.60$ $\scriptstyle\,\pm\,0.20$	$-8.77$ $\scriptstyle\,\pm\,0.74$
Flow-GRPO	$6.32$ $\scriptstyle\,\pm\,0.06$	$-7.83$ $\scriptstyle\,\pm\,0.09$	$-8.15$ $\scriptstyle\,\pm\,0.27$	$6.41$ $\scriptstyle\,\pm\,0.18$	$-8.06$ $\scriptstyle\,\pm\,0.25$	$-9.31$ $\scriptstyle\,\pm\,0.60$
DiffusionNFT	$6.23$ $\scriptstyle\,\pm\,0.07$	$-7.90$ $\scriptstyle\,\pm\,0.23$	$-9.39$ $\scriptstyle\,\pm\,0.26$	$6.53$ $\scriptstyle\,\pm\,0.16$	$-8.68$ $\scriptstyle\,\pm\,0.22$	$-11.71$ $\scriptstyle\,\pm\,0.50$
Guidance	$6.29$ $\scriptstyle\,\pm\,0.07$	$-7.65$ $\scriptstyle\,\pm\,0.21$	$-7.92$ $\scriptstyle\,\pm\,0.25$	$5.76$ $\scriptstyle\,\pm\,0.17$	$-7.12$ $\scriptstyle\,\pm\,0.22$	$-8.03$ $\scriptstyle\,\pm\,0.60$

Table 2: LFD outperforms reward-guided baselines on both D2 and GSK3

\beta

docking tasks, under all three held-out evaluators, despite never accessing the proxy docking scores utilized by the baselines.

Panel of Figure 7 — Figure 7: LFD outperforms reward-guided methods on held-out D2 and GSK3 $\beta$ docking scores without proxy rewards (a,c), yielding the best top-5% rDock scores and high scaffold novelty (b,d).

Figure 8: LFD -generated molecule docking to ATP pocket of GSK3 $\beta$ , showing multiple close interactions (pink labels) below 3.5 Å (GNINA).

Figure 9: LFD -generated 3D molecule satisfying Goal $5$ .

Molecule Design

See all 43 papers

Apr 8, 2026

Scientific Knowledge-driven Decoding Constraints Improving the Reliability of LLMs

Large language models (LLMs) have shown strong knowledge reserves and task-solving capabilities, but still face the challenge of severe hallucination, hindering their practical application. Though scientific theories and rules can efficiently direct the behaviors of human manipulators, LLMs still do not utilize these highly-condensed knowledge sufficiently through training or prompting. To address this issue, we propose SciDC, an LLM generation method that integrate subject-specific knowledge with strong constraints. By adopting strong LLMs to automatically convert flexible knowledge into multi-layered, standardized rules, we build an extensible framework to effectively constrain the model generation on domain tasks. Experiments on scientific tasks including industrial formulation design, clinical tumor diagnosis and retrosynthesis planning, consistently demonstrate the effectiveness of our method, achieving a 12% accuracy improvement on average compared with vanilla generation. We further discuss the potential of LLMs in automatically inductively summarizing highly-condensed knowledge, looking ahead to practical solutions for accelerating the overall scientific research process. All the code of this paper can be obtained (https://github.com/Maotian-Ma/SciDC).

Maotian Ma, Zheni Zeng, Zhenghao Liu +1

Nanjing University · Tsinghua University · Northeastern University

LLM Hallucination Mitigation Knowledge Augmentation for Language Models Constrained Decoding Scientific Reasoning in Language Models

Ask PDF

Aug 31, 2026

Language-Informed Flow Matching for Trend-Guided Structure-Based 3D Molecular Generation

Structure-based drug design (SBDD) requires ligands that satisfy both 3D target affinity and 1D chemical validity. Existing controllable generation methods often rely on task-specific fine-tuning or externally imposed sampling-time guidance, adding cost and potentially conflicting with evolving 3D geometric constraints. We propose LiFT, a language-informed cross-modal framework built on Flow Matching for trend-guided 3D molecular generation across both de novo design and scaffold hopping. LiFT uses a "Sense-Evolve-Assemble" agent to generate target-aware SMILES as intermediate chemical conditions, from which a pre-trained chemical foundation model extracts continuous semantic priors. These priors are integrated into geometric generation through a lightweight semantic projector with zero-initialized adaptive normalization for stable cross-modal conditioning. We further introduce a Self-Conditioned Decoupled Router (SCDR), which modulates the velocity field according to intermediate structural states during ODE integration. Experiments on Cross-Docked2020 show that LiFT achieves competitive distribution matching while improving medicinal chemistry metrics and maintaining competitive structural validity under task-steering settings without additional generator fine-tuning. Our results suggest that language-derived chemical priors provide effective trend-level guidance for 3D molecular generation. Code and released artifacts are available at https://github.com/kasurl/LiFT.

Tianyu Gao, Zhikai Su, Jiashu Li +5

1City University of Hong Kong · 2The University of Hong Kong · 3Stanford University

Flow Matching Structure-Based Drug Design 3D Molecular Generation Molecular Generation Conditional Generative Modeling

Ask PDF

Mar 23, 2026

Bridging the Know-Act Gap via Task-Level Autoregressive Reasoning

LLMs often generate seemingly valid answers to flawed or ill-posed inputs. This is not due to missing knowledge: under discriminative prompting, the same models can mostly identify such issues, yet fail to reflect this in standard generative responses. This reveals a fundamental know-act gap between discriminative recognition and generative behavior. Prior work largely characterizes this issue in narrow settings, such as math word problems or question answering, with limited focus on how to integrate these two modes. In this work, we present a comprehensive analysis using FaultyScience, a newly constructed large-scale, cross-disciplinary benchmark of faulty scientific questions. We show that the gap is pervasive and stems from token-level autoregression, which entangles task selection (validate vs. answer) with content generation, preventing discriminative knowledge from being utilized. To address this, we propose DeIllusionLLM, a task-level autoregressive framework that explicitly models this decision. Through self-distillation, the model unifies discriminative judgment and generative reasoning within a single backbone. Empirically, DeIllusionLLM substantially reduces answer-despite-error failures under natural prompting while maintaining general reasoning performance, demonstrating that self-distillation is an effective and scalable solution for bridging the discriminative-generative know-act gap

Jihyun Janice Ahn, Ryo Kamoi, Berk Atil +34

Penn State · UIUC · Columbia University +1

Scientific Reasoning in Language Models

Ask PDF

Abstract

Molecule Design

Explore similar work

Scientific Knowledge-driven Decoding Constraints Improving the Reliability of LLMs

Language-Informed Flow Matching for Trend-Guided Structure-Based 3D Molecular Generation

Bridging the Know-Act Gap via Task-Level Autoregressive Reasoning