cs.AIApr 1, 2026
SaveIn Search of the Ingredients of Open-Endedness: Replicating Picbreeder with Large Vision-Language Models
Organizations: New York University, Brooklyn, New York, USA · Work partially completed during an internship at Sakana AI. · Sakana AI, Tokyo, Japan · Independent, Dublin, Ireland ·
Abstract
We are in the midst of large-scale industrial and academic efforts to automate the processes of scientific, technological and creative production through AI-driven assistants. Historically, a fundamental property of these processes in their human form has been their open-endedness: their capacity for generating a seemingly endless supply of novel and meaningful new forms. Do artificial agents have any capacity for such fruitful unguided discovery? To answer this question, we turn to Picbreeder, the canonical exemplar of human-driven open-ended search, in which users collaboratively generated a diverse library of images through interactive evolution of small neural networks. We replicate Picbreeder, replacing human users with frontier Vision Language Models (VLMs). We observe clear qualitative differences between the output of our system and the historical human baseline, and attempt to characterize them using metrics of phylogenetic complexity and visual and semantic salience and novelty. In an effort to identify some of the causal factors contributing these differences, we study the addition of exploratory noise to the agents' selection process, of behavioral diversity between agents, and of narrative momentum in the form of memory of past actions. We make our code available at https://github.com/smearle/picbreeder-vlm.
Figures & tables
Figure 1. Large Vision-Language Models play Picbreeder and discover novel images. Cherry-picked examples.
Figure 3. Qualitative effect of varying the Number of Agents ( ), by sampling from variably-sized pools of (LLM-generated) personality traits and prepending these to system prompts during VLM-Picbreeder sessions. Archives with highest Semantic Recall ( 3(b) ) and Visual Coverage ( 3(g) ) are outlined.
Figure 4. Samples from the historical human archive after 2,000 user sessions.
Figure 5. Ancestral lineages of semantically similar images generated by human and VLM interactions with Picbreeder (only published images are displayed). Humans seem to take larger steps in semantic space (from face, to eye, to frog, to car) than VLMs (from abstract bird, to car seat, to dashboard, to hood ornament, to car).
| Sweep | Condition | Semantic Recall | Visual Coverage | Semantic Coverage | Tree Balance ( ) |
|---|---|---|---|---|---|
| Random Action Prob. ( ) | 0.0 | 0.087 0.001 | 0.614 0.002 | 0.696 0.004 | 0.235 0.026 |
| 0.05 | 0.086 0.001 | 0.619 0.011 | 0.702 0.006 | 0.246 0.023 | |
| 0.25 | 0.088 0.001 | 0.638 0.009 | 0.717 0.004 | 0.249 0.022 | |
| 0.5 | 0.085 0.001 | 0.633 0.007 | 0.709 0.005 | 0.260 0.013 | |
| 0.75 | 0.084 0.001 | 0.639 0.004 | 0.706 0.003 | 0.303 0.010 | |
| 1.0 | 0.082 0.001 | 0.610 0.013 | 0.700 0.003 | 0.275 0.030 |
Table 1. Summary of results of various interventions on VLM-driven Picbreeder in terms of their impact on our evaluation metrics (mean standard error). Overall best results highlighted in green; best results among each hyperparameter sweep appear in bold. The default setting—recurring across sweeps—is highlighted in grey.
Figure A1 . Effect of the choice of VLM model on the Semantic Recall of the Picbreeder archive. In the gemini-random setting, each agent is randomly assigned to one of the other gemini models shown in this plot. Surprisingly, gemini-2.5-pro significantly outperforms all other model choices, including gemini-3-pro-preview.
Figure A2 . Effect of the choice of VLM model on Picbreeder archives after 500 agent sessions. For each model, samples of 3 archives from different random seeds are shown. Samples are generated by selecting images at uniform intervals with respect to publication order. We note that gemini-3-pro-preview is prone to a kind of mode collapse in the collaborative archive, often obsessing over mushroom-like forms in particular. gemini-2.5-flash-lite , meanwhile, tends to flood the archive with abstract, high-frequency, psychedelic patterns (though it also discovers skull-like forms whose internal representations we evaluate in Fig. A3 resembling the—admittedly more refined—skull generated in the human baseline.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A3 . Visualization of internal representations resulting from VLM-driven evolution. We apply perturbations from to each weight in the CPPN, and display the weights that lead to greatest difference in terms of pixel distance from the initial image at the extremes of this range. These representations are not nearly as “fractured” as those resulting from SGD over a fixed-topology CPPN in [ Kumar et al., 2025 ] , but neither are they so neatly factorized as to correspond to features like mouths or eyes opening or closing.
Figure A4 . Snapshots of the original Picbreeder webpage, recovered via the Wayback Machine. We mimic a human user’s exposure to this home page by displaying, at the beginning of each VLM agent’s Picbreeder session, a sample of the archive generated thus far comprising top rated, best new, most branched, and random subsamples. Absent from our re-implementation are semantic tags, “Editor’s Picks”, user information, image titles, and the ability to further browse the site/archive.
Figure A5 . Distribution of human Picbreeder session lengths.
| Statistic | Value |
|---|---|
| Mean Generations | 21.96 |
| Median Generations | 11.00 |
| Min Generations | 1 |
| Max Generations | 765 |
| % 20 Generations | 70.9% |
Table A1 . Human Picbreeder session length statistics.
| # | Trait Description |
|---|---|
| 1 | You prefer images that evoke the smell of campfire smoke. |
| 2 | You act like a joyful dancer moving to an unseen rhythm. |
| 3 | You are searching for the visual equivalent of a wailing siren. |
| 4 | You are drawn to patterns that look like intersecting searchlights. |
| 5 | You prefer images that look like magnetic tape tracking errors. |
| 6 | You are searching for a visual representation of ’indifference’. |
Table A2 . Sample of LLM-generated personality traits used when . To generate these traits, we give gemini-3-pro-preview the Picbreeder VLM system prompt ( Fig. A17 ) and ask for personality traits that may implicitly affect an agent’s behavior on this task ( Fig. A18 ).
Figure A6 . Effect of exploration ( -greedy) on Semantic Recall within the Picbreeder archive. Forcing agents to take some random selections (i.e., with probability 0.25) can improve the quality of the archive, with Semantic Recall score approaching that of the historical human baseline. Large amounts of random parent selection ( ), and a fully random baseline (in which branching, publication, and archive rating decisions are also random) are detrimental.
Figure A8 . Effect of exploration ( -greedy) on the diversity of the Picbreeder archive. A moderate amount of noise can increase Visual and Semantic Coverage and Tree Balance, but, in excess, reduces the legibility of generated images (cf. 8(g) , Fig. A6 ).
Figure A9 . Effect of history—i.e. Context Length ( ), the number of previous actions included in an agent’s context—on Semantic Recall within the Picbreeder archive. Without any context, mode collapse is common, leading to reduced recall. proves to be a surprisingly effective sweep spot, with larger leading to overly noisy/abstract forms.
Figure A11 . Effect of history—i.e. Context Length ( ), the number of previous actions included in an agent’s context—on diversity within the Picbreeder archive. Increasing increases diversity, but with rapidly diminishing returns. is an exception; here, diversity peaks, likely because in this case the agent is prompted with an additional note encouraging its publication to be novel w.r.t. the still-visible archive sample ( 17(b) ). But the images in these archives are more noisy/abstract, as reflected in their low Semantic Recall scores ( 9(a) ). This may be due to the context window being overloaded, leading to decreased VLM performance in general; or due to the incentivization of novelty combined with forced publication steps effectively leading to the premature publication of semantically ill-defined works-in-progress.
Figure A12 . Effect of number of agents —i.e. the number of distinct personality traits (see Table A2 ) distributed among Picbreeder sessions—on Semantic Recall score and diversity metrics. Increasing improves various metrics of diversity without harming Semantic Recall.
Figure A13 . Effect of exploration ( -greedy) on the Fidelity of the Picbreeder archive. Greedy strategies can game this metric by refining a small set of images and flooding the archive with near-duplicates.
Figure A14 . Effect of history (number of previous actions included in an agent’s context/memory) on the Fidelity of the Picbreeder archive. Agents without history perform best by refining the current form, without any incentive to push past local optima.
Figure A15 . Effect of multiple agents (number of distinct personality traits assigned) on the fidelity of the Picbreeder archive. Adding agents reduces Fidelity by inhibiting mode collapse.
Figure A16 . Semantic attractors. A common failure case of VLMs when playing Picbreeder is their tendency to fall into apparent attractors (mode collapse) in CPPN-image space. We show snapshots of various archives, with images arranged according to visual embedding distance, selecting subregions that showcase such attractors.
Figure A17 . Prompt components used by the Picbreeder VLM agent.
Figure A18 . Prompt for generating personality traits used when (see Table A2 for sample output from gemini-3-pro-preview ). The VLM Picbreeder system prompt ( 17(a) ) is injected where indicated.
Figure A19 . Sample of an archive with Number of Agents , with images arranged according to visual embedding distance. We select a region of the archive that showcases the noisy, potentially adversarial images that seem to emerge at large . These might be owing to a large number of traits imbuing agents with an eye for relatively abstract properties of an image, where VLMs focused on roleplaying may be keen to project the satisfaction of such abstract inclinations onto otherwise meaningless forms. See Table A2 ; traits like 32 “You are drawn to the aesthetic of bad analog TV reception” even explicitly incentivize noise, despite our prohibiting such explicit search objectives ( Fig. A18 ).
More in this topic
Open-Ended Generation
Open-ended generation studies how AI produces coherent, useful, and varied responses when prompts allow many valid answers.
See all 41 papers