cs.LGNov 2, 2026

When Does Design Thinking Work? A Convergence Theorem for the Iterate-Test Core

Authors: Paweł Kuraś, Adrian Michalski, Alicja Gerka, Patryk Organiściak

Organizations: Institute of Informatics, Kozminski University, ul. Jagiellońska 57/59, 03-301 Warsaw, Poland · Faculty of Mathematics and Applied Sciences, Rzeszów University of Technology, al. Powstańców Warszawy 12, 35-959 Rzeszów, Poland · Department of Complex Systems, Rzeszów University of Technology, al. Powstańców Warszawy 12, 35-959 Rzeszów, Poland

Abstract

Design thinking is widely taught as an iterative loop of ideation, prototyping, and testing, yet conditions under which such a loop must improve innovation outcomes remain unspecified. Taking problem framing as given, we formalize the iterate-test core as search over a finite prototype space whose utility is learned through noisy evaluations. We prove that the prototype a team believes to be best converges almost surely to a local optimum under five conditions: finite framing, local ideation, non-summable discovery rates, unbiased repeated testing, and retention of tested prototypes. Bounded test bias or criteria drift degrades the guarantee by a bounded amount, and multiple stakeholders yield convergence to a locally Pareto-optimal set. Finite-horizon simulations across three landscape classes (additive valuation, NK rugged landscapes, and clustered modular architectures) show that asymptotic limits are rarely reached in practice. Teams face an optimizer's curse, overestimating their chosen concept by 4-8% and frequently discarding superior prototypes. Testing every prototype once proves to be the worst allocation; optimal performance requires building one to three prototypes per five tests and repeatedly re-testing leaders. Pre-screening concepts yields greater performance gains than tuning testing policies.

Figures & tables

Explore similar work

Nov 10, 2026

Probe What-If Worlds Before You Ship: Persona Simulation for Real-World Decision Making

User research and system testing reveal whether products and services meet user needs, yet recruiting specialized participants, designing controlled what-if comparisons, and repeating studies as designs evolve often bottleneck design iteration and data-driven decision-making. We present a framework and execution infrastructure that translates real-world industry inquiries into controlled experiments with simulated users. Our methodology formally defines the target cohort, experimental stimuli, and decision-relevant metrics, while the pipeline supports large-scale concurrent runs over cohort-stimulus-environment combinations (survey, web, chatbot, and OS app), auditable trial-level logging, cohort-level outcome aggregation, and cross-cohort and cross-variant statistical reports. Across twelve industry partner cases spanning five domains, we show how constructing hard-to-reach or what-if persona cohorts supports exploratory studies and distills simulated responses into testable design hypotheses. Cross-model comparisons on selected cases largely preserve primary conclusions, whereas perturbing or ablating persona attributes alters service outcomes, effect sizes, and inter-group differences, indicating that grounding target cohorts in detailed personas materially shapes simulation results. Together, these studies show that persona-enabled agents can test specified cohorts at design speed, diagnose systems under test, and deliver directional evidence that focuses problem definition, steers design and strategic decisions, and guides further study.
May 13, 2026

Robust Sequential Experimental Design for A/B Testing

Experimental design has emerged as a powerful approach for improving the sample efficiency of A/B testing, yet existing designs rely critically on correctly specified models. We study robust sequential experimental design under model misspecification and develop a unified framework that covers both contextual bandit and dynamic settings. Theoretically, we prove that our design bounds the worst-case mean squared error of the estimated treatment effect. Empirically, we demonstrate the effectiveness of the proposed approach using synthetic and real-world datasets from a leading technology company.
Feb 17, 2026

Scenario Approach with Post-Design Certification of User-Specified Properties

The scenario approach is an established data-driven design framework that comes equipped with a powerful theory linking design complexity to generalization properties. In this approach, data are simultaneously used both for design and for certifying the design's reliability, without resorting to a separate test dataset. This paper takes a step further by guaranteeing additional properties, useful in post-design usage but not considered during the design phase. To this end, we introduce a two-level framework of appropriateness: baseline appropriateness, which guides the design process, and post-design appropriateness, which serves as a criterion for a posteriori evaluation. We provide distribution-free upper bounds on the risk of failing to meet the post-design appropriateness; these bounds are computable without using any additional test data. Under additional assumptions, lower bounds are also derived. As part of an effort to demonstrate the usefulness of the proposed methodology, the paper presents two practical examples in H2 and pole-placement problems. Moreover, a method is provided to infer comprehensive distributional knowledge of relevant performance indexes from the available dataset.