Building a working mental model of a protein typically requires weeks of reading, cross-referencing crystal and predicted structures, and inspecting ligand complexes, an effort that is slow, unevenly accessible, and often requires specialized computational skills. We introduce Speak to a Protein, a new capability that turns protein analysis into an interactive, multimodal dialogue with an expert co-scientist. The AI system retrieves and synthesizes relevant literature, structures, and ligand data; grounds answers in a live 3D scene; and can highlight, annotate, manipulate and see the visualization. It also generates and runs code when needed, explaining results in both text and graphics. We demonstrate these capabilities on relevant proteins, posing questions about binding pockets, conformational changes, or structure-activity relationships to test ideas in real time. Speak to a Protein reduces the time from question to evidence, lowers the barrier to advanced structural analysis, and enables hypothesis generation by tightly coupling language, code, and 3D structures. Speak to a Protein is freely accessible at https://open.playmolecule.org.
Figures & tables
Figure 1: Overview of the system architecture. The Frontend hosts a chat panel, a 3D molecular viewer, and a Python console. The Agent Harness runs the agent loop, holds conversation memory, and executes tool calls. A search service indexes codebases, documentation, and literature. A sandboxed filesystem provides a per-project workspace and a preinstalled scientific software stack, with restricted network egress.
Group
Tool
Function
Sandbox
run_bash
Runs a bash command in the container.
cancel_task
Stops a background command.
Files
read_file
Reads a text file.
write_file
Writes a file.
edit_file
Replaces text inside a file.
view_image
Returns an image to the agent.
Table 1: Tools available to the agent. The agent calls each tool by name. Sandbox and file tools act on the per-project workspace; viewer tools act on the 3D scene in the frontend through the session WebSocket.
Figure 2: Interactive analysis of the CDK2 structure (PDB: 1AQ1) using the Speak to a Protein multimodal assistant. The user enters natural language queries in the chat panel (right), and the system responds with both textual answers and real-time updates to the 3D visualization (left). Python code executed in the viewer’s browser Python session is shown in the console (bottom), providing full transparency and reproducibility of the underlying analyses.
Figure 3: Accuracy on BixBench-Verified-50. Baseline scores are reproduced from the public leaderboard; error bars for Speak to a Protein indicate variability across repeated runs.
Figure 4: Task accuracy and cost on Common Protein Analysis Tasks , by group and ablation. Bars are the mean over three seeds and error bars their standard deviation. The visualization group has no bar for the ablations without a viewer, because those tasks cannot be attempted without it.
Figure 5: Structural workflow of the D3R/D2R selectivity case study. ( 5(a) ) D2R (PDB 6CM4, risperidone-bound, blue) overlaid on D3R (PDB 3PBL, eticlopride-bound, orange) after C α alignment, with the orthosteric ligands as licorice. ( 5(b) ) Close-up of the orthosteric and adjacent secondary pocket; selectivity-relevant residues are labelled for D2R (cyan) and D3R (yellow), including the conserved orthosteric Asp 3.32 anchor (D2R Asp114/D3R Asp110). ( 5(c) ) The Speak to a Protein interface at the end of the case study, showing the curated CSV of D3R-selective ligands (left) and the chat panel (right).
Figure 6: Figures the agent generated and embedded in its own report. (a) Annotated 3D-viewer snapshot of the cyclin E-dependent allosteric interface pocket (PDB 8VQ3), jointly framed by CDK2 (green) and cyclin E1 (blue) residues, captured by the agent during the analysis. (b) Matched-compound CDK2 selectivity relative to CDK1, CDK4, CDK6, CDK7, and CDK9 from filtered ChEMBL bioactivity; the dashed line is parity, the dotted red line marks a one-log margin in favor of CDK2.