Unnormalized Probability Densities

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 20

Sep 28, 2026cs.AI

Source-preserving alignment for robust evidence localization in scientific PDFS

Scientific information-extraction systems often return a claim with an evidence string, which users must locate in the original PDF. This is challenging because the extracted evidence and PDF text layer are different representations: line wrapping, Unicode variants, superscripts, citation markers, and fragmented items alter text sequences and geometry. We present a source-preserving alignment framework: normalize text for robust matching while preserving provenance for accurate localization. It aligns evidence with normalized page text, maps matches back to source-character spans, and renders only their geometry. When exact alignment fails, line-break-aware token alignment recovers supported spans while excluding unmatched noise. Experiments on 1,020 chemistry papers show that the framework achieves a 92.6% quote-level automatic localization rate, compared with 43.6% for text search and 19.1% for a precomputed bounding-box baseline. Component ablation confirms distinct contributions from normalization and approximate token alignment, while human verification assesses the visual correctness of returned highlights. Overall, these results demonstrate that reliable evidence verification requires robust matching and precise localization within a shared source-preserving alignment representation.

Zihao Liu, Wei Yang, Zixiao Dong +4

School of Computer Science and Technology, University of Science and Technology of China · State Key Laboratory of Cognitive Intelligence · University of Science and Technology of China +1

Unnormalized Probability Densities

Sep 28, 2026cs.CL

Adapt Semantics, Not Structure: Few-Instance Schema Calibration for Scientific PDF Extraction

A well-designed extraction schema is not necessarily ready for reliable LLM execution. When only limited verified extractions are available, manually tuning hundreds of field definitions through trial and error is costly. We frame this problem as few-instance schema calibration: adapting the operational semantics of an existing schema from a few annotated documents while preserving its structural contract. We introduce CPSE, a contract-preserving semantic extraction framework that jointly calibrates extraction prompts and field-level semantic descriptions from a few gold annotations. CPSE decomposes the schema into an invariant structural contract and mutable field semantics, and further separates identity discovery from record completion using manifest-conditioned resolution. On expert-annotated polymer-science documents, CPSE improves extraction by 9.93 points over an execution-matched baseline, with consistent gains under an independent judge and in a blinded expert audit. These results show that CPSE enables low-resource schema execution while preserving the output structure required downstream.

Zixiao Dong, Wei Yang, Zihao Liu +4

School of Computer Science and Technology, University of Science and Technology of China · State Key Laboratory of Cognitive Intelligence · University of Science and Technology of China +1

Extraction Schema Unnormalized Probability Densities Polymer Property Prediction

Sep 28, 2026cs.AI

Page-Aware Retrieval-Augmented Generation for EvalLLM 2026: A Five-Variant Study on French PDFs

We study retrieval-augmented generation (RAG) for questions about French PDF documents when both the answer and its supporting document pages are evaluated. Five system variants add dense retrieval, rank fusion, reranking, and query decomposition to a BM25 baseline. On 595 challenge questions, the complete system scores 0.4450 MRR@10 and 0.4013 Recall@10, compared with 0.3430 and 0.2994 for BM25. Dense retrieval alone and a simple lexical--dense fusion both underperform BM25. Reranking improves the hybrid system, whereas adding query decomposition produces the largest further gain, with higher latency and more detected output artifacts. The complete system slightly exceeds the reported anonymous overall mean on two answer metrics but falls below it on most page-retrieval metrics. These results identify accurate page selection, rather than semantic retrieval in isolation, as the main opportunity for improvement in this setting.

Abdelhak kelious

Capsens, France

Agentic Retrieval-Augmented Generation Systems Unnormalized Probability Densities Multi-Model Study

Sep 27, 2026cs.IR

From PDF to Evidence: Structure-Aware Retrieval for Clinical Practice Guidelines

Guideline documents are published as unstructured PDFs whose evidence is locked in visual structures---tables, flowcharts, and graded recommendations---that standard retrieval pipelines flatten into fixed-size text chunks. We cast evidence access as a document image analysis problem: parse each page image into typed structural elements, then retrieve structure-aware evidence units that follow the document's own layout (sections, table rows, flowchart paths, graded recommendations), each keeping its structural context so a result points to a specific element rather than a page. On 26 clinical practice guidelines from 9 sources (3,619 pages, Chinese and English) with 199 evidence queries, structure-aware units rank the gold element first under BM25, dense, and hybrid retrieval (hybrid Element Hit@1 of 0.382), with a significant element-level ranking gain over per-element OCR text (MRR_e +0.107, p=0.002; the Hit@5 gain is directional, p=0.17), while matching page-level recall (Page Hit@5 0.879 vs. 0.889, p=0.75) at 3.8x less context and clearly outperforming a ColPali visual-RAG baseline (PH@5 0.497).

Xingyu Lin, Dehui Du

Software Engineering Institute, East China Normal University Shanghai, China

Evidence-Grounded Questions Structure-Aware Unnormalized Probability Densities

Sep 22, 2026cs.CL

Layout-Guided Masking for GROBID: Lightweight Structural Gains in Large-Scale Scientific PDF Ingestion

Transforming scholarly PDFs into machine-readable fulltext remains a bottleneck for large-scale information systems. Recent vision-based parsers improve accuracy, but need GPUs and may introduce noise into the extracted text. GROBID, a modular font-stream parser running on CPU, is the de-facto standard for structuring scientific articles and underpins several of the largest open scholarly corpora. We pair it with a lightweight CPU detector localising figure, table, and paratext (header, footer, page number) regions, encoded as typed-area masks whose tokens are routed to GROBID's specialised models or discarded. On two PMC corpora, Bioinformatics (1,926 articles) and Materials Science (2,595), scored against JATS with a section-aware structural protocol, our extension improves over plain GROBID on most metrics (NS

+0.025

/

+0.013

;

+0.086

paragraph recall on Materials Science,

d_z{=}1.08

), and caption-linked figure recovery improves on both corpora. On the external Table-BRGM benchmark, table detection recovers F1

0.16 \to 0.94

and table structure follows (GriTS-Top

0.27 \to 0.78

, below the strongest GPU system). On body text, against four vision-based systems (Docling, MinerU, olmOCR, dots.ocr), it has the best paragraph precision on both corpora, the best section detection on Materials Science, and a character error rate within 0.004 of the best GPU parser. End-to-end on CPU, it costs

2.7

--

3.2\times

less than the cheapest GPU system (Docling) and

10

--

14\times

less than generative parsers.

Luca Foppiano, Sana Khamassi, Vipul Gupta

ScienciaLAB, Portugal · Helmholtz-Zentrum Hereon, Germany

Document Parsing Unnormalized Probability Densities Text Corpora Lightweight

Sep 21, 2026cs.CV

Document Retrieval-Aware Chunking (D-RAC): Universal Retrieval-Aware Ingestion of Enterprise Documents via PDF Normalization and Multimodal Markdown Conversion

Retrieval-Augmented Generation (RAG) systems over enterprise knowledge bases must ingest heterogeneous document formats -- PDFs, Word documents, presentations, and scans -- whose content is locked inside complex visual layouts, multi-column pages, and dense tables. Rule-based extraction and OCR destroy reading order, flatten tables, and lose heading hierarchy, while fully agentic chunking over extracted text incurs high token costs and hallucination risk. We present Document Retrieval-Aware Chunking (D-RAC), an extension of our Web Retrieval-Aware Chunking (W-RAC) framework to arbitrary document formats. D-RAC first normalizes any input document into PDF, exploiting the fact that virtually every format has a faithful, deterministic PDF rendering. A single multimodal LLM pass then converts rendered pages into retrieval-optimized Markdown -- rewriting tables as self-contained prose statements and preserving heading hierarchy -- after which chunking proceeds exactly as in W-RAC: deterministic parsing into ID-addressable units followed by lightweight LLM-based chunk planning over identifiers rather than text. Source text is never regenerated during chunking, preserving W-RAC's cost, determinism, and observability benefits while unlocking every renderable format as a first-class input. On the 236-document, 795-page PDF subset of the RAG-Multi-Corpus benchmark spanning five enterprise domains, D-RAC converts and chunks the entire corpus in 72 minutes with zero errors, producing 1,748 retrieval-ready chunks. Compared to agentic chunking with frontier LLMs, D-RAC reduces chunking-stage output tokens by 95.7%, cutting chunking cost by 77.8% (GPT-4.1 pricing) to 85.6% (Gemini 2.5 Pro pricing) and chunking time by 75%. D-RAC scales linearly to documents of 500+ pages.

Uday Allu, Abhivanth Sivaprakash, Pratik Singh +1

AI Research Team Yellow.ai · AI Research Team, Yellow.ai

Chunk Document Unnormalized Probability Densities

Sep 7, 2026cs.AI

AAS-RAIL: Improving Information Extraction for Asset Administration Shells through Retrieval-Augmented In-Context Learning

The Asset Administration Shell (AAS) is a cornerstone of Industry 4.0 and the Digital Product Passport, providing standardized digital representations of industrial assets. While manufacturers already maintain extensive technical product documentation, generating AAS instances from existing product datasheets remains a labor-intensive task because technical information is extracted from heterogeneous document structures and often involves company-specific terminology and conventions. In this work, we present AAS-RAIL, a retrieval-augmented information extraction (IE) approach that automatically generates Asset Administration Shells from PDF product datasheets using large language models (LLMs). Instead of relying on a fixed set of few-shot examples, the proposed retrieval-augmented in-context learning (RAIL) approach retrieves LLM-generated extraction helpers from similar Asset Administration Shells to provide instance-specific in-context learning (ICL). This enables the model to adapt its extraction behavior to company-specific naming conventions and formatting styles without fine-tuning. Our core contribution is the dynamic selection of company-specific AAS examples for each datasheet, replacing static prompting with an extraction pipeline that adapts to instances and combines semantic retrieval and structured information extraction. The proposed approach is evaluated on a collection of industrial product datasheets using a selection of open- and closed-weight LLMs. Experimental results show that RAIL consistently improves extraction quality over conventional few-shot prompting, yielding relative improvements of 30.4-52.4%. These results demonstrate that our approach provides an effective improvement for company-specific AAS generation.

Janek Groß, Jens Heidrich

University of Applied Sciences Mainz, Mainz, Rhineland-Palatinate, Germany

Information Extraction Extraction Large Language Model Pipelines Documentation Schema Unnormalized Probability Densities+1

Aug 17, 2026cs.CL

Counting Documents Is Not Counting Text: Unit Bias in Web-PDF Corpus Statistics

PDF corpora advertise their size in tokens, but every rate they publish (coverage, OCR routing, re-fetch recovery, language mix) is computed per document, and none decomposes its token total. Because PDF length is extremely skewed, the two units can describe the same corpus very differently. We ask how the headline statistics of a web-PDF corpus change when each document is weighted by the text it contributes rather than counted once. We used CC-MAIN-2021-31-PDF-UNTRUNCATED (7.9M Common Crawl PDFs, 32.6B tokens), the one public corpus that pairs the fragments Common Crawl stored with the re-fetched originals. Text mass is highly concentrated: 3.02% of text-bearing documents hold half the tokens (Gini 0.807). The clearest consequence is Common Crawl's payload cap, which truncated 23.06% of these documents but 63.08% of their text. Reconstructing the truncated fragments and extracting both versions, two widely used text-layer parsers recover only 1.4% and 11.4% of that exposed text, so roughly 55-62% of the corpus's text is unrecoverable from the crawl by such pipelines; under the 5MiB cap adopted in March 2025, 30.19% of tokens would still be exposed. We recommend that corpus statistics be reported in both units, documents and tokens.

Luca Foppiano

Common Crawl Foundation

Text Corpora Agentic Crawlers Unnormalized Probability Densities Counting Text Analysis and Detection

Aug 15, 2026cs.AI

A concentration result for multilayer feedforward neural networks

We consider for an arbitrary fixed

ρ

and for each positive integer

n

a multilayer feedforward artificial neural network with

ρ

layers,

n

neurons in the first layer (the input layer) and only one neuron, the output neuron, in the last layer. Very roughly formulated, the main result is that if the distribution of weights of connections from a layer to the next are, for all large

n

, approximated well by a fixed continuous (but otherwise arbitrary) curve which does not depend on

n

, and if the values of the

n

input neurons are independently and identically distributed with a continuous probability density function, then there is a number

ψ

such that for all

\varepsilon > 0

the probability that the value of the output neuron is in

[ψ- \varepsilon, ψ+ \varepsilon]

tends to 1 as

n

tends to infinity.

Vera Koponen

Neural Network Multilayer Perceptrons Feed-Forward Concentration Unnormalized Probability Densities Curves

Jul 21, 2026stat.ML

Algebraic Signatures for Structural Learning in Probability Tensors

Algebraic statistics characterizes statistical models through polynomial constraints, but it has mainly been used for analytically specified model classes. This paper studies the inverse problem: identifying probabilistic structure from vanishing binomials observed in empirical probability tensors. We treat the vanishing binomials of a toric model as its algebraic signature, and turn the ideal-variety correspondence of algebraic statistics into an operational procedure for structural learning that identifies a model by signature matching without parameter estimation. By restricting attention to a computationally tractable class of configuration matrices, which we call {\it the Kronecker-stack class}, we make these signatures explicitly enumerable. Within this class we define minimum invariant constraint (MIC) as the atomic unit characterizing each signature and generalizing the notion of independence. We tested this approach employing MICs on synthetic data as well as on corpus-scale real language data. The results suggested the utility of the method, revealing that the identified rank-one structures correspond to interpretable sets of words. These results open up a new avenue for applying algebraic statistics to computational linguistics.

Akihiro Maeda, Shohei Hidaka, Satoshi Aoki

Distributional Signatures Signatures Unnormalized Probability Densities

Jun 17, 2026cs.AI

Boltzmann MapReduce: A Partition-Function Reduce for Forkable Sandboxes

To leading order under local asymptotic normality (LAN), the confidence density a worker emits over a chunk of size

n

is a Gibbs--Boltzmann measure

\exp\{-βE(θ)\}

whose inverse temperature is the sample size,

β=n

. Three consequences are exact in the Gaussian/linear case and first-order otherwise: disjoint chunks carry independent Boltzmann factors, so the MapReduce \emph{reduce}, read literally, is a partition function

Z=\int\prod_k h_k\,dθ

whose mode is precision-weighted (inverse-variance) pooling; frequentist consistency is the zero-temperature limit

T=1/n\to0

Yossi Eliaz

Incredibuild · HIT CS Department, Israel

Boltzmann Distributions Unnormalized Probability Densities Sandbox

Jun 15, 2026cs.LG

Stop the Sampler! Classifier-Based Adaptive Stopping for Sampling Kernels

Sampling from complex, unnormalized probability densities is a fundamental challenge in Bayesian inference and probabilistic modeling. While Markov chain Monte Carlo (MCMC) methods provide asymptotic guarantees, they often suffer from slow mixing and high computational costs due to fixed or manually tuned trajectory lengths. In this work, we propose a novel framework that treats trajectory termination as a learnable component of the sampling dynamics. By framing MCMC within the theory of non-acyclic generative flow networks (GFlowNets), we train state-dependent neural classifiers to decide when a trajectory has reached a high-density region and should terminate. We theoretically establish the connection between optimal classifiers and the target density via detailed balance conditions and introduce a multilevel training scheme to facilitate exploration in complex geometries. Experimental results across various benchmark densities demonstrate that our approach significantly reduces average trajectory lengths while improving mode coverage and mixing compared to standard MCMC baselines.

Kirill Korolev, Nikita Morozov, Stepan Pavlenko +2

1HSE University · University of Edinburgh · 3CIFAR Fellow.

Markov Chain Monte Carlo Generative Flow Networks Unnormalized Probability Densities

Jun 8, 2026cs.LG

Operator learning for solving Fokker-Planck equations with various initial conditions

The Fokker-Planck equation (FPE) plays a pivotal role in describing the time evolution of probability density functions (PDFs) for systems governed by stochastic dynamics. In this work, we propose a conditional normalizing flow-based physics-informed neural network (PINN) framework for efficiently approximating the solution operator of the FPE for a whole range of initial conditions. Leveraging the Chapman-Kolmogorov equation for Markovian stochastic processes, the problem is reformulated into approximating a transition PDF starting at initial time from a Dirac mass centered at an arbitrary point. The PDF of an associated linearized stochastic differential equation (SDE) is employed as the base distribution for the normalizing flow, providing a good approximation of the target PDF, especially for small times, and thereby avoiding the singularity of the map associated with the Dirac delta initial distribution. Furthermore, a time-weighted loss function is introduced to mitigate numerical instabilities arising at small times, achieving a balance between causality and training difficulty as time progresses. A variety of numerical experiments are presented to illustrate the effectiveness and robustness of the proposed method.

Li Zeng, Xiaoliang Wan, Yaobin Wang +2

School of Mathematics and Statistics, Fuzhou University, Fuzhou, China · Department of Mathematics and Center for Computation and Technology, Louisiana State University, Baton Rouge 70803, USA · Faculty of Science and Technology, Beijing Normal-Hong Kong Baptist University, Zhuhai 519087, China. +2

Fokker--Planck Equation Parametric Physics-Informed Neural Network Stochastic Dynamics Unnormalized Probability Densities Loss Function

May 8, 2026eess.AS

Dependence on Early and Late Reverberation of Single-Channel Speaker Distance Estimation

Single-channel speaker distance estimation has recently achieved centimeter-level accuracy in simulated environments, yet it remains unclear which components of the room impulse response (RIR) the model exploits and how performance depends on the recording conditions. In this work, we decompose simulated RIRs into four variants (full, direct-only, no-late, and no-early) using the mixing time estimated from the echo density function as the boundary between early reflections and late reverberation. We define four calibration scenarios, from fully calibrated (synchronised capture, known source level) to fully uncalibrated (arbitrary onset, unknown level), and evaluate all combinations on a matched dataset. Results show that without time calibration, mean absolute error (MAE) increases to

1.29

m and the model extracts reverberation-based cues, with early reflections emerging as the most informative component. Further analysis against DRR,

C_{50}

, and

T_{60}

confirms that estimation accuracy improves with stronger early energy and degrades in highly reverberant environments. When time calibration is available, the model achieves a MAE of

0.14

m by extracting the propagation delay alone, regardless of the RIR content.

Michael Neri, Archontis Politis, Tuomas Virtanen

Faculty of Information Technology and Communication Sciences, Tampere University, Finland

Reverberation Impulse Response Speaker Unnormalized Probability Densities

May 5, 2026physics.ao-ph

Towards accurate extreme event likelihoods from diffusion model climate emulators

ML climate model emulators are useful for scenario planning and adaptation, allowing for cost-efficient experimentation. Recently, the diffusion model Climate in a Bottle (cBottle) has been proposed for generation of atmospheric states compatible with boundary conditions of solar position and sea surface temperatures. Crucially, cBottle can be guided to generate extreme events such as Tropical Cyclones (TCs) over locations of interest. Diffusion models such as cBottle work by approximating the probability density of the training data. Here, we show use cases of the probability density estimates of atmospheric states obtained from this climate emulator. Most importantly, these estimates allow us to calculate likelihoods of extreme events under guidance. When guiding the model towards states including TCs, comparing the probability density under the guided and unguided model enables us to quantify how much more likely the guidance has made the TC. We show how these odds ratios allow us to importance-sample from the TC distribution, reducing the standard error of the probability estimate compared to simple Monte Carlo sampling. Furthermore, we discuss results and limitations of the application of model probability densities to extreme event attribution-like experiments. We present these early but encouraging results hoping they will spur more research into probabilistic information that can be gained from diffusion models of the atmosphere.

Peter Manshausen, Noah Brenowitz, Julius Berner +2

NVIDIA, Santa Clara, CA, USA

Tropical Cyclones Atmospheric Dynamics Emulators Unnormalized Probability Densities

Mar 30, 2026cs.CR

Kill-Chain Canaries: Stage-Level Tracking of Prompt Injection Across Attack Surfaces and Five Production LLMs

Multi-agent LLM systems now read documents, web pages and tool results on behalf of users, yet their resistance to prompt injection is usually reported as one number: did the attack succeed? We introduce a kill-chain canary method that plants a unique token in every injected payload and records the furthest of four stages it reaches (Exposed -> Persisted -> Relayed -> Executed), across 950 runs, five production LLMs, six attack surfaces, and five defense conditions. Exposure was 100% among runs that called the tool; the outcomes differ downstream. Claude Haiku 4.5 and Claude Sonnet 4.5 executed none of their 164 text-surface attacks, and in the text relay the canary token never appeared in a memory write (0/40); GPT-4o-mini executed 53% of its attacks. Four findings follow. (1) A Claude writer kept the canary token out of shared memory in every relay run we report; one cross-model pairing (Claude writer, GPT-4o-mini reader, n = 3) is consistent with this protecting the reader, and other pairings were not tested. (2) As readers, the Claude models executed 0/40 raw pre-seeded injections, but Claude Haiku 4.5 executed 2/3 injections relayed by GPT-4o-mini; whether relayed injections are harder to refuse than raw ones is an open question. (3) DeepSeek Chat went from 0/24 on pre-seeded memory to 8/8 on tool results, scenarios that also differ in task and payload format; white-text PDF payloads, invisible on the rendered page, succeeded at least as often as visible ones. (4) pi_detector and write_filter failed on channels they do not inspect, spotlighting failed on content it wraps, and write_filter blocked the PDF relay but not the text relay, a difference we cannot explain. Code and run logs are publicly released: https://github.com/KevinChunye/prompt_injection

Haochuan Kevin Wang, Zechen Zhang

Massachusetts Institute of Technology · University of Chicago

Attacker Large Language Model Prompt Engineering Unnormalized Probability Densities

Feb 3, 2026cs.LG

Efficient Training of Boltzmann Generators Using Off-Policy Log-Dispersion Regularization

Sampling from unnormalized probability densities is a central challenge in computational science. Boltzmann generators are generative models that enable independent sampling from the Boltzmann distribution of physical systems at a given temperature. However, their practical success depends on data-efficient training, as both simulation data and target energy evaluations are costly. To this end, we propose off-policy log-dispersion regularization (LDR), a novel regularization framework that builds on a generalization of the log-variance objective. We apply LDR in the off-policy setting in combination with standard data-based training objectives, without requiring additional on-policy samples. LDR acts as a shape regularizer of the energy landscape by leveraging additional information in the form of target energy labels. The proposed regularization framework is broadly applicable, supporting unbiased or biased simulation datasets as well as purely variational training without access to target samples. Across all benchmarks, LDR improves both final performance and data efficiency, with sample efficiency gains of up to one order of magnitude.

Henrik Schopmans, Christopher von Klitzing, Pascal Friederich

Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, Kaiserstr. 12, 76131 Karlsruhe, Germany · Institute of Nanotechnology, Karlsruhe Institute of Technology, Kaiserstr. 12, 76131 Karlsruhe, Germany

Boltzmann Distributions Kullback-Leibler Regularization Regularization Unnormalized Probability Densities Variational

Dec 23, 2025cs.LG

Control Variate Score Matching for Diffusion Models

Sampling from unnormalized probability densities is a pervasive challenge across the computational and physical sciences. Diffusion models provide a powerful generative framework for this task, but their success relies on accurately estimating the score of the perturbed target distribution. Current approaches face a dichotomy between two standard estimation methods: the Denoising Score Identity (DSI) requires data samples and exhibits high variance at low noise levels, whereas the Target Score Identity (TSI) relies on the energy function and suffers from diverging variance at high noise levels. In this work, we reconcile both approaches by introducing the Control Variate Score Identity (CVSI), an unbiased estimator with an analytically optimal, state- and time-dependent control coefficient that theoretically minimizes variance over the entire diffusion process. CVSI serves as a robust plug-in estimator that significantly enhances performance and efficiency in data-free sampler learning and training-free diffusion sampling. These gains scale to complex, high-dimensional energy-based models.

Khaled Kahouli, Romuald Elie, Klaus-Robert Müller +3

1Google DeepMind · 2BIFOLD – Berlin Institute for the Foundations of Learning and Data · 3Machine Learning Group, Technische Universität Berlin +2

Score-Based Diffusion Model Diffusion Models Scores Unnormalized Probability Densities

Jul 29, 2025cs.LG

Edge Selection for the Effective use of Piecewise-Constant Distributions as Neural Network Outputs for Event Prediction

We study the output representation of a neural network used for next event prediction. We propose partitioning the time axis into a fixed set of intervals and having a neural network output a categorical distribution over them, which we map to a (mostly) piecewise-constant probability density. We present an optimization procedure that selects interval edges in order to maximize data likelihood under the representation. The representation is well suited to processes whose inter-event distribution is a mixture of smooth and sharply peaked components

\unicode{x2013}

a pattern we find common in event data recorded from real-world processes.

Kevin Doran, Tom Baden

School of Life Sciences, University of Sussex, UK · School of Engineering and Informatics, University of Sussex, UK · Institute of Ophthalmic Research, University of Tübingen, Germany

Unnormalized Probability Densities Neural Network

Oct 8, 2024stat.ML

Joint Bayesian Parameter and Model Order Estimation for Low-Rank Probability Mass Tensors

Obtaining a reliable estimate of the joint probability mass function (PMF) of a set of random variables from observed data is a significant objective in statistical signal processing and machine learning. Modelling the joint PMF as a tensor that admits a low-rank canonical polyadic decomposition (CPD) has enabled the development of efficient PMF estimation algorithms. However, these algorithms require the rank (model order) of the tensor to be specified beforehand. In real-world applications, the true rank is unknown. Therefore, an appropriate rank is usually selected from a candidate set either by observing validation errors or by computing various likelihood-based information criteria, a procedure that could be costly in terms of computational time or hardware resources, or could result in mismatched models which affect the model accuracy. This paper presents a novel Bayesian framework for estimating the low-rank components of a joint PMF tensor and simultaneously inferring its rank from the observed data. We specify a Bayesian PMF estimation model and employ appropriate prior distributions for the model parameters, allowing the rank to be inferred without cross-validation.We then derive a deterministic solution based on variational inference (VI) to approximate the posterior distributions of various model parameters. Numerical experiments involving both synthetic data and real classification and item recommendation data illustrate the advantages of our VI-based method in terms of estimation accuracy, automatic rank detection, and computational efficiency.

Joseph K. Chege, Arie Yeredor, Martin Haardt

Communications Research Laboratory, Ilmenau University of Technology, Ilmenau, Germany · School of Electrical and Computer Engineering, Tel Aviv University, Tel Aviv, Israel

Low-Rank Structure Variational Inference Unnormalized Probability Densities