Project Greenhouse: Progress Toward Fully Open and Sovereign Agentic Search

Jimmy Lin; Sahel Sharifymoghaddam; Lingwei Gu; Nour Jedidi

cs.IROct 8, 2026

Project Greenhouse: Progress Toward Fully Open and Sovereign Agentic Search

Authors: Jimmy Lin, Sahel Sharifymoghaddam, Lingwei Gu, Nour Jedidi

Organizations: David R. Cheriton School of Computer Science University of Waterloo

Abstract

Project Greenhouse represents our exploration of a simple thesis: We believe that it is possible to build fully open and sovereign models for agentic search with only modest computational resources. As a first milestone, we describe how to build a competitive pointwise decoder-only reranker using a simple two-step recipe comprising pre-training from scratch followed by supervised fine-tuning, starting only from commonly available datasets. Contrary to the dominant approach in the literature, we do not rely on existing open-weight backbones from third parties, and thus we are fully in control of model training, from end to end. We were able to accomplish the bulk of our experiments using no more than a handful of GPUs. This report articulates the importance and benefits of our approach, and we share artifacts that enable transparent, independent reproduction of all aspects of model training. Beyond data, code, and configurations that capture our efforts, we also release checkpoints for our family of Gaggle models, demonstrating the feasibility of our approach and providing a first step toward validating our broader thesis.

Figures & tables

Out-of-the-box listwise rerankers
	Model	Size	DL19	DL20	DL21	DL22	DL23	Mean
1	BM25 ( Lin et al., 2021 )	—	0.506	0.480	0.446	0.269	0.263	0.393
2	Oracle reranker	—	0.892	0.871	0.835	0.678	0.669	0.789
3	Gemma-4 ( Gemma Team, 2026 )	26B	0.743	0.705	0.723	0.516	0.497	0.637
4	Qwen3.8 ( Qwen Team, 2026b )	27B	0.732	0.705	0.716	0.520	0.492	0.633
5	Jev	?	0.736	0.715	0.721	0.505	0.476	0.631

Table 1 : Headline results on TREC Deep Learning 2019–2023, reporting nDCG@10 on reranking top-100 BM25 candidates. Our pointwise reranker—Gaggle (base reranker)—is shown in row 1 . A row number annotated with the † symbol marks results reported in prior work and copied here directly; unmarked rows capture numbers from our own evaluations. Results in rows 1 – 1 were taken from RankLLM ( Sharifymoghaddam et al., 2025 ) , and those in rows 1 and 1 were taken from Jedidi et al. (2026) . Mean represents the unweighted average across collections. The highest reported score in each column, including ties, is bolded (excluding the oracle reranker).

Out-of-the-box listwise rerankers
	Model	Size	COVID	News	Robust	NFC	SciFact	SCID	FiQA	Mean
1	BM25 ( Lin et al., 2021 )	—	0.595	0.395	0.407	0.322	0.679	0.149	0.236	0.398
2	Oracle reranker	—	0.975	0.827	0.837	0.544	0.927	0.443	0.590	0.735
3	Gemma-4 ( Gemma Team, 2026 )	26B	0.846	0.492	0.662	0.387	0.805	0.208	0.439	0.548
4	Qwen3.8 ( Qwen Team, 2026b )	27B	0.849	0.505	0.647	0.390	0.803	0.225	0.462	0.554
5	GPT-6.1 Sol	?	0.837	0.526	0.668	0.398	0.821	0.237	0.520	0.572

Table 2 : Headline results on seven BEIR collections, reporting nDCG@10 on reranking top-100 BM25 candidates. Our pointwise reranker—Gaggle (base reranker)—is shown in row 2 . All results are from our own evaluations. Mean represents the unweighted average across collections. The highest reported score in each column, including ties, is bolded (excluding the oracle reranker).

Figure 1 : Validation loss on held-out ClimbMix during pre-training.

	Model	Data	Params	Tokens	CORE	MMLU	MMLU-Pro
1	gaggle-nanochat-pretrained	ClimbMix	3.3B	283B	0.458	0.493	0.178
2	gaggle-nanochat-pretrained-bidirectional ‡	ClimbMix	3.3B	287B	0.339	0.453	0.187
Open pre-training data
3	nanochat-d34	FineWeb-Edu	2.2B	89B	0.338 †	0.257	0.115
4	OPEN-1B ( Donaghy et al., 2026 )	OPEN-1B mix	1.6B	400B	0.239	0.259 †	0.117 †
5	Puro 2B ( Luo et al., 2026a )	Puro mix	2.0B	1.4T	0.464	0.552	0.235

Table 3 : Overview of a selection of recent pre-trained base models. Rows 1 and 2 are our causal model trained from scratch and its bidirectional variant, adapted from the causal checkpoint with masked next-token prediction. For all scores, higher is better. † indicates scores reported by the model developer and copied here directly; other scores are from our own evaluations. ‡ indicates scoring with masked prediction. Params include embedding tables.

Figure 2 : Prompts used for both reranker supervised fine-tuning and evaluation. (a) The standard prompt is used for Gaggle (base reranker) and all other contrastive studies by default. (b) The QPQ prompt is used for the causal query-repetition condition; both occurrences contain the same truncated query. (c) The mask-appended prompt is used for the Gaggle (MNTP) mask condition; <|output_end|> is the MNTP mask token. With the single-position readout, relevance is scored using the ␣true minus ␣false logits at the colon after Relevant: , immediately before the mask in (c).

Table 4 : Analysis of run-to-run variability and model averaging (nDCG@10). All models have 3.22B parameters; the four individual runs used the same training recipe, with differences in random seed, GPU hardware, microbatch token budget, and resulting update count. All models rerank top-100 BM25 candidates. Rows 1–4 represent individual trials; row 5 is the soup checkpoint used in our headline results. Summary rows report the mean and sample standard deviation (SD) across rows 1–4 only. The highest score in each column is marked in bold (including ties, excluding summary rows).

	Backbone	Size	LR	TREC DL	BEIR
1	gaggle-nanochat-pretrained	3.2B	$5{\times}10^{-5}$	0.641	0.544
2	nanochat-d34	2.1B	$5{\times}10^{-5}$	0.621	0.521
3	Gemma-4-E2B	4.6B	$5{\times}10^{-5}$	0.627	0.537
4	Qwen3.5-4B-Base	4.2B	$5{\times}10^{-5}$	0.626	0.530
5	MiniCPM5-2B-Base	2.2B	$5{\times}10^{-5}$	0.627	0.530
6	Gemma-4-E2B	4.6B	$1{\times}10^{-5}$	0.640	0.546

Table 5 : Effects of different backbones (mean nDCG@10). All configurations are trained with the same supervised fine-tuning recipe. LR denotes the peak learning rate; other hyperparameters follow the headline recipe, and hardware and microbatch differences are documented in Appendix C . All configurations rerank top-100 BM25 candidates. Row 1 is baseline run 1 of Gaggle (base reranker). The highest score in each column is marked in bold.

Figure 3 : Training loss on identical batches at peak learning rates

5{\times}10^{-5}

(a) and

1{\times}10^{-5}

(b); bins widen over training to smooth per-batch noise.

	Training data	Instances	TREC DL	BEIR
1	RLHN-250K	247,534	0.641	0.544
2	RLHN-680K	648,762	0.646	0.545
3	Tevatron MS MARCO	399,600	0.618	0.470

Table 6 : Effects of training data (mean nDCG@10). All rerankers have 3.22B parameters and use the same one-epoch training recipe, with differences in training data (size and source mix), GPU hardware, microbatch token budget, and resulting update count. All configurations rerank top-100 BM25 candidates. Row 1 is baseline run 1 of Gaggle (base reranker). The highest score in each column is marked in bold.

	Training queries
Benchmark	50K	100K	150K	200K	250K
TREC DL	0.628	0.616	0.634	0.637	0.638
BEIR	0.525	0.533	0.542	0.543	0.545

Table 7 : Effectiveness of baseline run 4 across RLHN-250K training checkpoints (mean nDCG@10). All configurations rerank top-100 BM25 candidates.

	Configuration	TREC DL	BEIR
1	Causal, standard prompt	0.641	0.544
2	Causal, QPQ prompt	0.647	0.551
3	Causal, mean pooling	0.642	0.543
4	MNTP, standard prompt	0.643	0.544
5	MNTP, appended mask	0.643	0.543
6	MNTP, mean pooling	0.638	0.538

Table 8 : Effects of causal versus bidirectional modeling (mean nDCG@10). All rerankers have 3.22B parameters and use the same training recipe, with differences in pre-trained initialization, attention, prompt, and readout. All configurations rerank top-100 BM25 candidates. Row 1 is baseline run 1 of Gaggle (base reranker). The highest score in each column is marked in bold.

	Training configuration	TREC DL	BEIR
1	LCE	0.641	0.544
2	LCE + self-filtering	0.641	0.548
3	LCE + self-filtering + self-distillation	0.640	0.547

Table 9 : Effects of self-filtering and self-distillation (mean nDCG@10). All rerankers have 3.22B parameters and use the same pre-trained initialization and training recipe, except for self-filtering, the additional distillation loss, and the resulting update count. All configurations rerank top-100 BM25 candidates. Row 1 is baseline run 1 of Gaggle (base reranker). The highest score in each column is marked in bold (including ties).

	Objective	TREC DL	BEIR
1	LCE	0.641	0.544
2	Pointwise CE	0.628	0.528

Table 10 : The effects of different loss functions (mean nDCG@10). Both variants start from the same 3.29B-parameter pre-trained checkpoint; the rerankers retain 3.22B parameters for LCE and 3.29B for CE. All configurations rerank top-100 BM25 candidates. Row 1 is baseline run 1 of Gaggle (base reranker). The highest score in each column is marked in bold.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

	Setting	Value
1	Fine-tuning	All retained parameters; no frozen layers or LoRA
2	Scoring parameters	Two pre-trained LM output rows: 4,352 parameters; no new head
3	Implementation	Native backbone; scaled dot-product attention (SDPA)
4	Training precision	FP32 weights, gradients, and optimizer states; TF32 matrix multiplication
5	Evaluation precision	BF16 backbone; FP32 scoring rows and scoring computation; final hidden states cast to FP32
6	Gradient checkpointing	Enabled

Appendix

Table 11 : Default settings for supervised fine-tuning and evaluating individual Gaggle rerankers. Run-specific exceptions are described in Appendix C .

Explore similar work

OpenSeeker-v2: Pushing the Limits of Search Agents with Informative and High-Difficulty Trajectories
May 5, 2026Yuwen Du, Rui Ye, Shuo Tang +4Supervised Fine-Tuning Tool-Augmented Language Model Agents
1Shanghai Jiao Tong University
Beyond Parallel Sampling: Diverse Query Initialization for Agentic Search
Jun 15, 2026Sidhaarth Murali, João Coelho, Jingjie Ning +3Agentic Search Test-Time Scaling
Carnegie Mellon University, United States · Instituto Superior Técnico and INESC-ID, University of Lisbon, Portugal · NOVA LINCS, NOVA School of Science and Technology, Portugal
T-Search: An Open Agentic Retriever and Playground for Hard Multi-Step Search
Oct 5, 2026Olga Tsymboi, Ramil Latypov, Aleksandr Medvedev +5Multi-Hop QA Agentic Search
T-Tech

May 5, 2026, cs.AIJ/K move · Enter open · S save

OpenSeeker-v2: Pushing the Limits of Search Agents with Informative and High-Difficulty Trajectories

Yuwen Du, Rui Ye, Shuo Tang, Keduan Huang+3

1Shanghai Jiao Tong University

Supervised Fine-Tuning Tool-Augmented Language Model Agents LLM Agent Trajectory Synthesis LLM Agents LLM Agent Training

Open paper AskPDF arXiv

Deep search capabilities have become an indispensable competency for frontier Large Language Model (LLM) agents, yet their development remains dominated by industrial giants. The typical industry recipe involves a highly resource-intensive pipeline spanning pre-training, continual pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning (RL). In this report, we show that when fueled with informative and high-difficulty trajectories, a simple SFT approach could be surprisingly powerful for training frontier search agents. By introducing three simple data synthesis modifications: scaling knowledge graph size for richer exploration, expanding the tool set size for broader functionality, and strict low-step filtering, we establish a stronger baseline. Trained on merely 10.6k data points, our OpenSeeker-v2 achieves state-of-the-art performance across 4 benchmarks (30B-sized agents with ReAct paradigm): 46.0% on BrowseComp, 58.1% on BrowseComp-ZH, 34.6% on Humanity's Last Exam, and 78.0% on xbench, surpassing even Tongyi DeepResearch trained with heavy CPT+SFT+RL pipeline, which achieves 43.4%, 46.7%, 32.9%, and 75.0%, respectively. Notably, OpenSeeker-v2 represents the first state-of-the-art search agent within its model scale and paradigm to be developed by a purely academic team using only SFT. We are excited to open-source the OpenSeeker-v2 model weights and share our simple yet effective findings to make frontier search agent research more accessible to the community.

Abstract

Explore similar work

OpenSeeker-v2: Pushing the Limits of Search Agents with Informative and High-Difficulty Trajectories

Beyond Parallel Sampling: Diverse Query Initialization for Agentic Search

T-Search: An Open Agentic Retriever and Playground for Hard Multi-Step Search