Behavioral Intelligence Platforms: From Event Streams to Autonomous Insight via Probabilistic Journey Graphs, Behavioral Knowledge Extraction, and Grounded Language Generation

Arun Patra; Bhushan Vadgave

cs.IRMar 12, 2026

Behavioral Intelligence Platforms: From Event Streams to Autonomous Insight via Probabilistic Journey Graphs, Behavioral Knowledge Extraction, and Grounded Language Generation

Authors: Arun Patra, Bhushan Vadgave

Organizations: Journium, Inc.

Abstract

Contemporary product analytics systems require users to pose explicit queries, such as writing SQL, configuring dashboards, or constructing funnels, before insights can surface. This pull-based paradigm creates a bottleneck: it requires both domain knowledge and technical fluency, and assumes practitioners know in advance which questions to ask. We argue that behavioral analytics should move from passive systems that answer queries to active systems that continuously detect and explain behavioral phenomena. We present the Behavioral Intelligence Platform (BIP), a system architecture that transforms raw event streams into automatically generated insights. BIP consists of four layers. First, Normalization and State Derivation (NSD) standardizes events and maps them to a semantic state hierarchy. Second, a Behavioral Graph Engine (BGE) models user journeys as absorbing Markov chains and computes transition probabilities, removal effects, and path quality metrics. Third, a Behavioral Knowledge Graph (BKG) and Detector System convert graph outputs into grounded behavioral facts and identify behavioral phenomena. Finally, a Grounded Language Layer constrains large language model outputs to verified facts, producing reliable narrative insights. We formalize the Behavioral Intelligence Problem, introduce a taxonomy of detectors for autonomous insight generation, and propose an interestingness score to prioritize insights under limited attention.

Figures & tables

Panel of Figure 1 — Figure 1: BIP four-layer data-flow architecture. Each layer produces a structured artifact consumed by the next: NSD derives a semantic state hierarchy from raw events; BGE builds absorbing Markov chain snapshots; BKG+DS reify graph metrics into grounded facts and emit typed findings; GLL generates faithful narrative insights constrained to verified facts.

Panel of Figure 2 — Figure 2: Example six-state funnel as an absorbing Markov chain. Single-bordered circles are transient states ( $\mathcal{S}$ ); double-bordered circles are absorbing states ( $\mathcal{A}$ ). Edge labels are empirical transition probabilities; each row sums to 1 across all outgoing edges (thin arrows indicate low-probability direct conversions). The dashed back-edge represents re-visitation from feature_used to sign_up .

Panel of Figure 3 — Figure 3: Absorption probabilities $B[s,\text{converted}]$ (Eq. ( 4 )) computed by closed form from the transition matrix in Figure 2 via $B=N\cdot R$ . Each bar is the probability that a journey starting from state $s$ ultimately reaches converted . Reproduced exactly by markov_journey_model.py ; the empirical conditional $P(\text{converted}\mid\text{reached}(s))$ from Monte Carlo trajectories agrees with $B[s,\text{converted}]$ to within sampling noise (see simulate_trajectories.py ), as required by the strong Markov property when all journeys share a common starting state.

Panel of Figure 4 — Figure 4: Removal effect ranking (Eq. ( 8 )) for the example funnel. Each bar shows the decrease in overall $P(\text{converted})$ when the corresponding state is removed from the journey graph and transition probabilities are re-normalised. import_data is the structurally most critical state, as it is the sole gateway to invite_teammate — the highest-converting transient state.

Rank	ID	Detector	Finding (summary)	Score
1	F002	ActivationDriver	feature_used is activation driver for converted (lift 4.2 $\times$ )	0.845
2	F001	TemporalRegression	email_verified $\to$ profile_complete drop-off $+$ 18 pp post-v2.3	0.736
3	F004	SegmentDivergence	Mobile conversion 22 pp below desktop	0.602
4	F003	UnexpectedLoop	Loop: profile_complete $\to$ sign_up (8% of users)	0.252
5	F005	DropOffCluster	Drop-off at sign_up $\to$ abandoned (low significance, $N{=}85$ )	0.054

Table 1: Composite interestingness scores for five synthetic detector findings (weights:

\alpha{=}0.30

\beta{=}0.25

\gamma{=}0.20

\omega{=}0.15

\varepsilon{=}0.10

). Scores computed from synthetic simulation data by interestingness_scoring.py .

User Modeling

User modeling represents people's interests, knowledge, goals, and changing behavior to personalize predictions and AI assistance.

See all 151 papers

Discovery Agents for Real-Time Analytics: Toward Proactive Insight Systems
May 26, 2026Gaetano Rossiello, Dharmashankar SubramanianMulti-Agent LLM Systems LLM Agent Orchestration
IBM · New York City, New York, USA · Yorktown Heights, New York, USA
Towards Autonomous Business Intelligence via Data-to-Insight Discovery Agent
May 8, 2026Dongming Wu, Junwen Li, Ming Lu +2AI-Assisted Decision Making RL for LLM Agents
1Rajax Network Technology(Taobao Shangou of Alibaba)
IBAD: Interpretable Behavioral Anomaly Detection on Human Mobility Data
Jun 14, 2026Bita Azarijoo, John Krumm, Cyrus ShahabiInterpretable Anomaly Detection Anomaly Detection
University of Southern California Los Angeles, California, USA

May 26, 2026J/K move · Enter open · S save

Discovery Agents for Real-Time Analytics: Toward Proactive Insight Systems

Gaetano Rossiello, Dharmashankar Subramanian

IBM · New York City, New York, USA · Yorktown Heights, New York, USA

Multi-Agent LLM Systems LLM Agent Orchestration

Open paper AskPDF arXiv

Modern analytics systems are fundamentally reactive, requiring users to define queries over increasingly complex and continuously evolving data. In real-time streaming environments, this paradigm breaks down, as the space of potential insights becomes too large to enumerate manually. We present a multi-agent architecture for autonomous insight discovery over real-time data streams. The system implements a continuous discovery loop in which agents generate hypotheses, compile them into executable analytics, validate generated artifacts, and produce visualizations and deployable applications. The architecture leverages Apache Kafka for event-driven coordination, Apache Flink for stream processing, and large language models to implement specialized agents. A key contribution is a contract-driven design based on typed intermediate artifacts, enabling modularity, observability, lineage, and safer execution of dynamically generated analytics. Through use cases in retail, finance, and public data, we show how this architecture supports a shift from query-driven analytics to proactive, discovery-driven systems.

Abstract

User Modeling

Explore similar work

Discovery Agents for Real-Time Analytics: Toward Proactive Insight Systems

Towards Autonomous Business Intelligence via Data-to-Insight Discovery Agent

IBAD: Interpretable Behavioral Anomaly Detection on Human Mobility Data