AutoRecLab: Describe the Experiment, Get the Code!
Organizations: University of Siegen Siegen, Germany · University of Göttingen Göttingen, Germany
Abstract
Empirical evaluation is central to recommender-systems (RecSys) research, but turning experimental designs into executable code remains a manual and error-prone task. We present AutoRecLab, a Python-based autonomous RecSys lab that automates RecSys experiments from natural-language prompts. Given a research idea, AutoRecLab derives explicit experiment requirements, builds and validates a prototype, and iteratively expands it into the requested full experiment. The workflow combines retrieval-augmented generation (RAG) for documentation lookup, static type verification, and execution-steered tree search. In our demonstration, AutoRecLab autonomously implements an explicit-to-implicit feedback conversion study. In a baseline comparison across six algorithms and three datasets, 8 of 9 runs succeed at an average cost of approx- imately $1 per run with GPT-5.4-mini.
Figures & tables
| Run Number | |||||||||
| Metric | 1.1 | 1.2 | 1.3 | 2.1 | 2.2 | 2.3 | 3.1 | 3.2 | 3.3 |
| Dataset | MU | MU | MU | ML1M | ML1M | ML1M | VI | VI | VI |
| Algos | 6 | 6 | 6 | 6 | 6 | 6 | 6 | 6 | 6 |
| Cost ($) | 1.09 | 1.08 | 1.10 | 0.95 | 0.98 | 0.91 | 1.04 | 1.01 | 0.90 |
| Runtime P | 4.37h | 12.23h | 40m | 23m | 58m | 1.52h | 17.32h | 7m | 19m |
| Runtime F | 30.2h | 33.65h | 16.23h | 32.33h | 32.08h | 22.43h | 5m | 24.48h | 16.58h |
Scientific Code Generation
Scientific code generation develops systems that turn scientific questions, papers, and workflows into executable, scientifically valid programs.
See all 53 papers