Credit risk analysis in financial institutions traditionally requires analysts to manually write SQL queries, run statistical computations, and build visualization dashboards. This is a time-consuming workflow that limits exploration to familiar segments. We introduce Oculi, a conversational platform that transforms natural language questions into comprehensive credit risk analyses, complete with data queries, statistical testing, and interactive visualizations. Oculi employs a three-layer architecture that separates reasoning (LLM-powered agent), execution (Model Context Protocol tool servers), and presentation (agentic UI), enabling analysts to discover high-risk portfolio segments. Within Oculi, a new segment discovery pipeline is proposed that combines deterministic statistical methods with LLM-guided feature selection, leveraging LLM semantic domain knowledge alongside data-driven metrics to identify meaningful, actionable portfolio segments. Evaluated on a mortgage portfolio with 200+ features, Oculi demonstrates effectiveness in discovering material risk segments previously intractable through manual exploration, reducing time-to-insight significantly while maintaining auditability and statistical rigor.
Figures & tables
Method
Segments Eval’d
Material Bivariate
Coverage
∑∣z∣
Random selection
110
34
37%
132
Top-contribution (det.)
132
38
41%
139
Decision tree (CART)
46
2
2%
8
LLM-guided
162
59
63%
222
Constrained brute force ( d=9 )
220
93
100%
351
Exhaustive ( d=200 , c=4 )
∼ 318,000
computationally infeasible
Table 1: Segment discovery quality on a real anonymized mortgage portfolio (9 features, delinquency_1_30 rate, April vs. March 2024). Coverage = fraction of material segments identified relative to constrained brute force. ∑∣z∣ = cumulative absolute z-score. The final row projects the brute-force candidate count for a production-scale portfolio ( d=200 features, average cardinality c=4 ): (2200)×42≈318,000 bivariate segments per metric per period.
Metric
Value
Total LLM invocations
20
Total feature suggestions attempted
42
Valid feature names (%)
76.2%
Hallucinated/invalid features (%)
23.8%
Unique features suggested (out of 9)
8 (89%)
Table 2: LLM feature suggestion quality across 20 invocations with varied response formats. The validation layer successfully filters hallucinated features while recovering valid picks from malformed JSON.
Stage
Time (s)
% of Total
Univariate scan (34 segments)
6.7
33%
Materiality filter
< 0.01
< 1%
LLM deepening (3 parents × 5 features)
12.2
60%
Tree discovery (join + fit)
1.4
7%
Total pipeline
20.3
100%
Table 3: Pipeline stage latency on real S3 Parquet data (9 features). Tree discovery is fast because DuckDB caches the Parquet scan from the univariate stage.
Citigroup, Irving, TX, United States · University of North Carolina at Chapel Hill, Chapel Hill, NC, United States · Georgia Institute of Technology, Atlanta, GA, United States +1