Benchmarking Temporal Web3 Intelligence: Lessons from the FinSurvival 2025 Challenge

Oshani Seneviratne; Fernando Spadea; Adrien Pavao; Aaron Micah Green; Kristin P. Bennett

cs.LGFeb 26, 2026

Benchmarking Temporal Web3 Intelligence: Lessons from the FinSurvival 2025 Challenge

Authors: Oshani Seneviratne, Fernando Spadea, Adrien Pavao, Aaron Micah Green, Kristin P. Bennett

Organizations: Rensselaer Polytechnic Institute Troy, NY, USA · Codabench, MLChallenges Paris, France · Vassar College Poughkeepsie, NY, USA

Abstract

Temporal Web analytics increasingly relies on large-scale, longitudinal data to understand how users, content, and systems evolve over time. A rapidly growing frontier is the Temporal Web3: decentralized platforms whose behavior is recorded as immutable, time-stamped event streams. Despite the richness of this data, the field lacks shared, reproducible benchmarks that capture real-world temporal dynamics, specifically censoring and non-stationarity, across extended horizons. This absence slows methodological progress and limits the transfer of techniques between Web3 and broader Web domains. In this paper, we present the FinSurvival Challenge 2025 as a case study in benchmarking temporal Web3 intelligence. Using 21.8 million transaction records from the Aave v3 protocol, the challenge operationalized 16 survival prediction tasks to model user behavior transitions.We detail the benchmark design and the winning solutions, highlighting how domain-aware temporal feature construction significantly outperformed generic modeling approaches. Furthermore, we distill lessons for next-generation temporal benchmarks, arguing that Web3 systems provide a high-fidelity sandbox for studying temporal challenges, such as churn, risk, and evolution that are fundamental to the wider Web.

Figures & tables

	1st Place	2nd Place	3rd Place
Team	Balancehero	AutoFinSurv	FinBoost
Focus	Feature Engineering	Optimization	Ensemble
Model	XGBoost AFT	XGBoost Cox	XGBoost Mix
Features	Hierarchical (10,000+)	Original (90)	Augmented (90+)
Average C-index	0.914	0.849	0.847

Table 1. Comparison of Top-Performing Solutions in the FinSurvival 2025 Challenge

DeFi Survival Task	Web Domain Equivalent
Deposit $\to$ Withdraw	Subscription $\to$ Churn (SaaS)
Borrow $\to$ Liquidate	Credit Line $\to$ Default (FinTech)
Repay $\to$ Borrow	Feature Engagement $\to$ Upsell (E-commerce)
Index Event $\to$ Censoring	User Session $\to$ Inactivity/Logout

Table 2. Mapping DeFi Survival Tasks to Web Analytics

Survival Analysis

Survival analysis models when events such as disease progression, failure, or death occur, accounting for incomplete follow-up and censoring.

See all 106 papers

DMind Benchmark: Toward a Holistic Assessment of LLM Capabilities across the Web3 Domain
Apr 18, 2025Enhao Huang, Pengyu Sun, Shuxun Wang +13LLM Evaluation Software Vulnerability Detection
Zhejiang University Hangzhou, China · The State Key Laboratory of Blockchain and Data Security · DMind.ai Singapore +1
From Leakage to Fidelity: Reliable Benchmarking for Temporal Cascade Prediction
Oct 29, 2025Jie Peng, Rui Wang, Qiang Wang +4Forecasting Benchmarks Temporal Data Leakage
Renmin University of China · Alibaba
Impermanent: A Live Benchmark for Temporal Generalization in Time Series Forecasting
Mar 9, 2026Azul Garza, Renée Rosillo, Rodrigo Mendoza-Smith +5Forecasting Benchmarks Time Series Forecasting
TimeCopilot · Independent Researcher · ELLIS Institute Tübingen +3

Apr 18, 2025J/K move · Enter open · S save

DMind Benchmark: Toward a Holistic Assessment of LLM Capabilities across the Web3 Domain

Enhao Huang, Pengyu Sun, Shuxun Wang, Zixin Lin+12

Zhejiang University Hangzhou, China · The State Key Laboratory of Blockchain and Data Security · DMind.ai Singapore+1

LLM Evaluation Software Vulnerability Detection LLMs for Cybersecurity

Open paper AskPDF arXiv

The Web3 ecosystem, underpinned by cryptographic primitives and decentralized consensus, represents a high-stakes environment where software vulnerabilities and incentive misalignments translate directly into financial loss. As Large Language Models (LLMs) are increasingly integrated into this domain for tasks ranging from smart contract auditing to decentralized finance analytics, ensuring their reliability is paramount. However, general-purpose benchmarks fail to capture the specialized reasoning required for these adversarial and protocol-driven settings. To bridge this gap, we introduce DMind Benchmark, a comprehensive evaluation suite designed to rigorously assess LLM proficiency across the Web3 stack. DMind Benchmark encompasses nine distinct subdomains (spanning infrastructure, smart contracts, token economics, etc.) and combines objective knowledge retrieval with complex open-ended reasoning tasks that emulate real-world operational challenges. We conduct an extensive evaluation of 31 leading proprietary and open-weights models, employing a contamination-aware pipeline and verifying the statistical robustness of our scoring protocol through rigorous cross-judge consistency checks. Our analysis reveals a critical dichotomy: while models demonstrate competence in foundational infrastructure concepts, they exhibit significant vulnerabilities in high-reasoning tasks such as security auditing. Furthermore, we provide a Pareto analysis to guide cost-effective deployment and demonstrate through adversarial experiments that high performance on DMind Benchmark necessitates genuine reasoning rather than superficial memorization. Since its open-source release in April 2025, DMind Benchmark achieved the #1 trending position on Hugging Face for nearly a week and accumulated over 13k downloads by June 2026, establishing itself as a standard for advancing secure and trustworthy AI in Web3.

Abstract

Survival Analysis

Explore similar work

DMind Benchmark: Toward a Holistic Assessment of LLM Capabilities across the Web3 Domain

From Leakage to Fidelity: Reliable Benchmarking for Temporal Cascade Prediction

Impermanent: A Live Benchmark for Temporal Generalization in Time Series Forecasting