Residual-Based Causal Discovery via Pretrained In-Context Quantile Regression

Daniel Klippert; Quang Nhat Huynh; Alexander Marx

stat.MLOct 21, 2026

Residual-Based Causal Discovery via Pretrained In-Context Quantile Regression

Authors: Daniel Klippert, Quang Nhat Huynh, Alexander Marx

Organizations: Department of Statistics, TU Dortmund University, Germany · Research Center Trustworthy Data Science and Security, University Alliance Ruhr, Germany · Department of Economics, University of Münster, Germany

Abstract

Residual-based causal discovery assumes that each variable is generated from its parents and an independent noise source. It estimates this noise and tests its independence from candidate causes. For identifiable functions such as additive-noise models (ANMs) and the broader class of location-scale noise models (LSNMs), this principle identifies the causal direction, as only the causal model admits this independence. Existing LSNM methods rely on parametric assumptions to obtain standardized residuals, risking model misspecification. In contrast, the RESIT-framework allows for nonparametric estimation of residuals to iteratively identify sink nodes, but was proposed for ANMs. To close this gap, we propose Loscar, an extension of RESIT to LSNMs that estimates the conditional mean and scale via quantile regression, enabling pretrained tabular foundation models as plug-in estimators. We compare it with a feasible generalized least squares (FGLS) variant, which is consistent given suitable basis functions. Across bivariate and multivariate causal-discovery benchmarks, in-context quantile regression substantially outperforms FGLS with flexible function classes, particularly for multivariate causes.

Figures & tables

Figure 1: Accuracy in % over 19 bivariate benchmarks. Loscar and Loscar-GL use TabPFN for in-context quantile regression, computing the conditional scale using different variants (Section 3.1 ). Loscar-FGLS (Section 3.2 ) employs spline-based FGLS. Loscar has an average accuracy of $90.04\%$ . While Loscar -GL performs on par, the FGLS variant struggles to model complex location-scale relationships.

Method	Accuracy (%) $\uparrow$	AUDRC (%) $\uparrow$
Loscar	90.04 $\pm$ 10.84	94.55 $\pm$ 9.17
Loscar-GL	89.81 $\pm$ 11.26	94.36 $\pm$ 9.81
Loscar-FGLS	74.97 $\pm$ 19.49	74.57 $\pm$ 26.44
SkewD-LL	84.56 $\pm$ 19.95	89.58 $\pm$ 16.30
CAM	75.10 $\pm$ 23.17	73.90 $\pm$ 26.44
QCCD	67.64 $\pm$ 26.44	69.92 $\pm$ 33.23

Table 1: Averaged Results on Bivariate Benchmarks. Top two are in bold and the best result is underlined.

Light Tunnel ( $n=10\,000$ , $d=20$ , $\textit{\#edges}=39$ )
Method	F1 $\uparrow$	OE $\downarrow$	SHD $\downarrow$	SID $\downarrow$
Loscar	0.430	12	45	49
Loscar-GL	0.430	13	45	49
Loscar-FGLS	0.382	16	68	52
HOST	0.000	39	89	181
SkewD	NA	NA	NA	NA

Table 2: Results for Light Tunnel. Best amortized and non-amortized models marked (OE=Order Errors).

Figure 2: Comparison of Loscar and baselines in terms of F1, order errors, SHD and SID on LSNM data.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

	Tagasovska et al. (2020)					Mooij et al. (2016)					Guyon et al. (2019)			Klippert and Marx (2026)
Method	AN	ANs	LS	LSs	MNU	SIM	SIMc	SIMln	SIMG	Tübingen	Cha	Net	Multi	ANs-med	ANs-str	ANs-ext	LSs-med	LSs-str	LSs-ext
Loscar	100	100	100	99	100	79	82	72	76	71.69	75.67	85.67	85.67	99.0	98.0	100	96.5	97.5	93
Loscar-GL	100	100	99	100	100	77	81	73	76	69.34	74.67	85.33	86.00	99.0	98.0	100	98.0	97.0	93
Loscar-FGLS	95	100	54	38	50	79	82	81	82	69.78	76.67	81.67	77.33	97.5	94.0	99	37.5	58.0	72
SkewD-LL	100	100	100	100	87	54	51	81	88	45.97	54.33	73.33	72.00	100.0	100.0	100	100.0	100.0	100
CAM	100	100	100	53	86	57	60	87	81	57.77	46.67	78.33	34.67	100.0	100.0	100	82.5	70.0	33

Appendix

Table 3: Accuracy in % over All Bivariate Benchmark Datasets. Baselines are split into conventional and amortized. The best result per benchmark is bolded.

	Tagasovska et al. (2020)					Mooij et al. (2016)					Guyon et al. (2019)			Klippert and Marx (2026)
Method	AN	ANs	LS	LSs	MNU	SIM	SIMc	SIMln	SIMG	Tübingen	Cha	Net	Multi	ANs-med	ANs-str	ANs-ext	LSs-med	LSs-str	LSs-ext
Loscar	100.00	100.00	100.00	99.96	100.00	87.81	93.72	87.60	94.09	70.36	71.83	96.79	95.69	99.93	99.94	100.00	99.79	99.85	99.09
Loscar-GL	100.00	100.00	99.98	100.00	100.00	88.27	92.78	87.87	94.08	66.40	71.49	96.90	96.41	99.92	99.95	100.00	99.93	99.86	98.92
Loscar-FGLS	98.32	100.00	52.71	22.87	66.92	88.77	92.71	84.55	93.25	62.29	76.12	92.16	77.50	99.09	97.94	99.96	16.80	40.00	54.93
SkewD-LL	100.00	100.00	100.00	100.00	97.58	64.58	64.33	94.57	94.95	54.57	57.50	81.93	92.01	100.00	100.00	100.00	100.00	100.00	100.00
CAM	100.00	100.00	100.00	36.29	76.45	67.76	68.82	87.20	88.24	69.37	41.05	84.88	40.02	100.00	100.00	100.00	74.08	59.61	10.24

Appendix

Table 4: AUDRC in % over All Bivariate Benchmark Datasets. Baselines are split into conventional and amortized. The best result per benchmark is bolded.

Method	AN	ANs	LS	LSs	MNU	SIM	SIMc	SIMln	SIMG	Tübingen	Cha	Net	Multi	ANs-med	ANs-str	ANs-ext	LSs-med	LSs-str	LSs-ext
	Tagasovska et al. (2020)					Mooij et al. (2016)					Guyon et al. (2019)			Klippert and Marx (2026)
Loscar $(\tau=0.05)$	100	100	100	99	100	79	82	72	76	71.69	75.67	85.67	85.67	99.0	98.0	100	96.5	97.5	93
Loscar $(\tau=0.25)$	100	100	100	97	100	77	82	71	76	69.75	72.33	85.33	85.67	98.5	98.0	100	96.5	96.0	92
Loscar-GL ( $m=10$ )	100	100	99	100	100	77	81	73	76	69.34	74.67	85.33	86.00	99.0	98.0	100	98.0	97.0	93
Loscar-GL ( $m=100$ )	100	100	99	100	100	80	82	73	77	64.41	75.00	86.67	84.67	99.0	97.5	100	99.5	94.5	87
Loscar-FGLS	95	100	54	38	50	79	82	81	82	69.78	76.67	81.67	77.33	97.5	94.0	99	37.5	58.0	72

Appendix

Table 5: Accuracy in % over All Bivariate Benchmark Datasets. The best result per benchmark is bolded. We include variations of our main approach, ablating

\tau_{1}

from its default

0.05

and

m

from its default

10

. Further, RESIT-TabPFN is an ANM variant of Loscar performing in-context regression with TabPFN and last, RESIT-OLS is another ANM variant of our method leveraging standard spline-based regression.

	Tagasovska et al. (2020)					Mooij et al. (2016)					Guyon et al. (2019)			Klippert and Marx (2026)
Method	AN	ANs	LS	LSs	MNU	SIM	SIMc	SIMln	SIMG	Tübingen	Cha	Net	Multi	ANs-med	ANs-str	ANs-ext	LSs-med	LSs-str	LSs-ext
Loscar $(q=0.05)$	100.00	100.00	100.00	99.96	100.00	87.81	93.72	87.60	94.09	70.36	71.83	96.79	95.69	99.93	99.94	100.00	99.79	99.85	99.09
Loscar $(q=0.25)$	100.00	100.00	100.00	99.94	100.00	86.84	92.59	89.43	93.15	65.75	69.67	96.17	95.90	99.93	99.92	100.00	99.78	99.82	98.91
Loscar–GL ( $m=10$ )	100.00	100.00	99.98	100.00	100.00	88.27	92.78	87.87	94.08	66.40	71.49	96.90	96.41	99.92	99.95	100.00	99.93	99.86	98.92
Loscar-GL ( $m=100$ )	100.00	100.00	99.98	100.00	100.00	89.13	93.28	89.07	93.89	56.81	71.56	96.99	94.76	99.98	99.95	100.00	99.99	99.21	96.55
Loscar-FGLS	98.32	100.00	52.71	22.87	66.92	88.77	92.71	84.55	93.25	62.29	76.12	92.16	77.50	99.09	97.94	99.96	16.80	40.00	54.93

Appendix

Table 6: AUDRC in % over All Bivariate Benchmark Datasets. The best result per benchmark is bolded. We include variations of our main approach, ablating

\tau_{1}

from its default

0.05

and

m

from its default

10

. Further, RESIT-TabPFN is an ANM variant of Loscar performing in-context regression with TabPFN and last, RESIT-OLS is another ANM variant of our method leveraging standard spline-based regression.

Figure 3: Shown are the estimates of the conditional mean and quantiles obtained from TabPFN-3 with $m\in\{10,100\}$ and the corresponding scale computed as described in Section 3.1.2 on Pair 1 of the LSs dataset ( Tagasovska et al., 2020 ) . The outer contours of the shaded area correspond to the highest and lowest quantile considered by the corresponding instantiation. The estimates are used for Loscar-GL , where $m=10$ is the default used in our experiments.

Figure 4: Shown are the estimates of the conditional mean and quantiles obtained from TabPFN-3 (left) and TabPFN-3.5 (right) on Pair 1 of the LSs dataset ( Tagasovska et al., 2020 ) . It can be observed that the estimates for TabPFN-3 exhibit greater smoothness.

Sachs ( $n=853$ , $d=11$ , $\textit{\#edges}=17$ )
Method	F1 $\uparrow$	OE $\downarrow$	SHD $\downarrow$	SID $\downarrow$
Loscar	0.296	7	19	55
Loscar-GL	0.320	7	17	55
Loscar-FGLS	0.150	9	34	55
HOST	0.400	3	15	47
SkewD	0.296	7	19	55

Appendix

Table 7: Results on F1, Order Errors (OE), SHD and SID for the Sachs Dataset. The best result for amortized and non-amortized models each is bolded.

	F1 $\uparrow$		Precision $\uparrow$		Recall $\uparrow$		Order errors $\downarrow$		SHD $\downarrow$		SID $\downarrow$
Method	d=6	d=12	d=6	d=12	d=6	d=12	d=6	d=12	d=6	d=12	d=6	d=12
Loscar	0.740	0.616	0.749	0.605	0.742	0.638	1.13	3.17	3.40	9.96	5.43	23.46
Loscar-GL	0.745	0.620	0.753	0.608	0.747	0.643	1.08	3.11	3.33	9.88	5.31	22.40
Loscar-FGLS	0.537	0.377	0.504	0.328	0.595	0.463	2.10	5.24	6.82	19.57	9.73	33.32
HOST	0.394	0.343	0.353	0.282	0.458	0.457	3.15	6.27	9.07	22.46	12.33	38.27
SkewD	0.715	0.714	0.748	0.762	0.690	0.675	1.53	2.84	3.29	6.37	7.36	21.61

Appendix

Table 8: Results on Mean F1, Precision, Recall, Order Errors, SHD and SID on Synthetic Multivariate LSNM Data with Heterogeneous Noise. Baselines are split into conventional and amortized. The best result per benchmark is bolded.

Figure 5: Comparison of Loscar and baselines in terms of F1, order errors, SHD and SID on ANM data.

	F1 $\uparrow$		Precision $\uparrow$		Recall $\uparrow$		Order errors $\downarrow$		SHD $\downarrow$		SID $\downarrow$
Method	d=6	d=12	d=6	d=12	d=6	d=12	d=6	d=12	d=6	d=12	d=6	d=12
Loscar	0.612	0.501	0.614	0.460	0.625	0.560	1.45	3.67	5.02	14.19	6.38	24.90
Loscar-GL	0.632	0.509	0.632	0.467	0.647	0.570	1.44	3.51	4.75	13.78	6.04	23.43
Loscar-FGLS	0.718	0.589	0.716	0.539	0.730	0.667	1.08	2.68	3.56	11.89	4.93	18.84
HOST	0.354	0.271	0.303	0.196	0.442	0.454	3.18	6.31	10.10	30.09	12.44	39.67
SkewD	0.802	0.777	0.840	0.826	0.773	0.738	0.98	2.06	2.23	5.00	5.06	17.30

Appendix

Table 9: Results on Mean F1, Precision, Recall, Order Errors, SHD and SID on Synthetic Multivariate ANM Data with Heterogeneous Noise.

	LSs		ANs
Method	d=6	d=12	d=6	d=12
Loscar	17.90 $\pm$ 1.97	127.04 $\pm$ 21.30	17.61 $\pm$ 1.73	125.59 $\pm$ 22.04
Loscar-GL	32.42 $\pm$ 1.46	72.99 $\pm$ 4.41	31.41 $\pm$ 1.59	75.55 $\pm$ 3.48
Loscar-FGLS	0.58 $\pm$ 0.01	2.83 $\pm$ 0.03	0.58 $\pm$ 0.01	2.82 $\pm$ 0.03
HOST	17.70 $\pm$ 7.28	51.57 $\pm$ 35.48	13.04 $\pm$ 8.28	22.50 $\pm$ 1.13
SkewD	50.38 $\pm$ 25.91	165.84 $\pm$ 60.74	49.36 $\pm$ 23.37	215.87 $\pm$ 88.02

Appendix

Table 10: Mean

\pm

Standard Deviation Runtime (Seconds) based on Synthetic Multivariate Data with Heterogeneous Noise. Baselines are split into conventional and amortized. The best result per benchmark is bolded.

Figure 6: Average runtimes in seconds (s).

Figure 7: Comparison of Loscar and baselines in terms of F1, order errors, SHD and SID on LSNM data with exponential noise.

Figure 8: Comparison of Loscar and baselines in terms of F1, order errors, SHD and SID on LSNM data with Gaussian noise.

Figure 9: Comparison of Loscar and baselines in terms of F1, order errors, SHD and SID on LSNM data with skew-

t

noise.

Figure 10: Comparison of Loscar and baselines in terms of F1, order errors, SHD and SID on ANM data with exponential noise.

Figure 11: Comparison of Loscar and baselines in terms of F1, order errors, SHD and SID on ANM data with Gaussian noise.

Figure 12: Comparison of Loscar and baselines in terms of F1, order errors, SHD and SID on ANM data with skew-

t

noise.

Causal Discovery

Causal discovery infers cause-and-effect structures from observational and interventional data, distinguishing causal relationships from correlation.

See all 222 papers

Causal Discovery via Transformed Low-Rank Quantile Surfaces
Sep 15, 2026Ryo Kamimura, Thong PhamCausal Discovery Causal Identifiability
Shiga University · RIKEN AIP · The University of Osaka
Beyond Additivity: Causal Discovery in Location-Scale Noise Models with Hidden Variables
Jun 6, 2026Mariyam Khan, Shohei Shimizu, Thong PhamGraph Structure Learning Causal Discovery
University of Bergen · *Work conducted while a student trainee at RIKEN AIP · 2The University of Osaka +2
Fast Nonparametric Conditional Independence Testing via Two-Stage Regression
Jun 16, 2026Eric V. StroblCausal Discovery Causal Structure Learning
Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA 15213, USA

Sep 15, 2026J/K move · Enter open · S save

Causal Discovery via Transformed Low-Rank Quantile Surfaces

Ryo Kamimura, Thong Pham

Shiga University · RIKEN AIP · The University of Osaka

Causal Discovery Causal Identifiability

Open paper AskPDF arXiv

We propose Low-Rank Quantile Surfaces (LRQS), a bivariate causal model in which, in the causal direction, an unknown monotone transformation of the conditional quantile surface admits a low-rank functional decomposition. LRQS subsumes location-scale noise models and post-nonlinear heteroscedastic noise models, while allowing multiple quantile bases to represent changes beyond location-scale effects. We prove generic identifiability of LRQS: the transformed quantile surface is low rank in the causal direction, whereas reverse representability under the corresponding constraints occurs only for exceptional, fine-tuned cause marginals. We provide a nonparametric causal score using a fitting procedure that alternates between rank-constrained approximation of discretized quantile surfaces and isotonic estimation of the unknown monotone transformation. Experiments on synthetic mechanisms with higher-rank distributional shape variation and strong nonlinear distortions, together with standard bivariate benchmarks, show that LRQS is especially effective when conditional distributional shape or observation distortion goes beyond existing location-scale assumptions.

Abstract

Causal Discovery

Explore similar work

Causal Discovery via Transformed Low-Rank Quantile Surfaces

Beyond Additivity: Causal Discovery in Location-Scale Noise Models with Hidden Variables

Fast Nonparametric Conditional Independence Testing via Two-Stage Regression