Universal Short-Time Fourier Transform: Data-Adaptive Window-Length Selection for Time-Frequency Analysis
Authors: Manan Mittal, Kanad Sarkar, Ryan M. Corey, John R. Buck, Andrew C. Singer
Organizations: Stony Brook University · University of Illinois Urbana Champaign · University of Illinois Chicago · University of Massachusetts Dartmouth
The short-time Fourier transform (STFT) characterizes the local frequency content of time-varying signals by evaluating the discrete Fourier transform over a sliding analysis window. The STFT is governed by a fundamental resolution trade-off: a short window localizes rapid transients in time at the cost of frequency resolution, whereas a long window resolves closely spaced harmonics at the expense of temporal resolution. The Universal Short-Time Fourier Transform (U-STFT) resolves this trade-off sequentially by modeling the analysis window duration as a data-adaptive state variable that indexes the number of hops included in the current observation interval. At each frame, a bank of candidate window durations is scored using normalized order-three Rényi entropy to quantify joint time-frequency concentration. A window length trellis aggregates the resulting O(3M) candidate window schedules with linear complexity per frame, guaranteeing a cumulative mix-loss regret bound of O((k+1)logM) relative to any hindsight window schedule with k resolution changes over M frames. By linearity of the Fourier transform, posterior expectation across the candidate bank yields a single spectrum equivalent to an STFT evaluated with a data-adaptive effective window, while an envelope-normalized synthesis stream guarantees exact overlap-add reconstruction. Evaluations on synthetic and marine bioacoustic signals demonstrate adaptive resolution tracking, enhanced spectral concentration, and exact time-domain invertibility.
Figures & tables
Fig. 1: The STFT time-frequency resolution trade-off. A short analysis window provides fine temporal localization but coarse frequency resolution due to its wide spectral main lobe. A long analysis window narrows the spectral main lobe for fine frequency resolution at the expense of temporal smearing.
Fig. 2: Causal window-length trellis. State s corresponds to a window duration Ls=(s+1)H . All trajectories arriving at state s at frame m share the same physical window wLs and candidate spectrum Xm,s , collapsing O(3M) schedules into O(smax) recursive state updates per frame.
Fig. 3: Spectrogram comparison for the hyperbolic chirp on a shared decibel scale. The shortest fixed window ( L0=H ) resolves the rapid onset but broadens the quasi-stationary tail; the longest fixed window ( L15=16H ) sharpens the tail but smears the rapid onset. The U-STFT adapts its effective window duration continuously to maintain ridge concentration throughout. The right column enlarges the first 1.5 s.
Fig. 4: Diagnostic trajectories for the hyperbolic chirp: (a) local Rényi resolution losses ℓm,s with the framewise minimum, (b) posterior window distribution μm(s) with the U-STFT and hindsight schedules, (c) global 2-D Rényi entropy H3TF , (d) cumulative excess loss over the framewise minimum, (e) per-frame mix-loss regret relative to the theoretical bound, and (f) 2-D Rényi regret relative to the per-prefix optimal window schedule. Grey curves show the fixed-window bank.
Fig. 5: Controlled tone and transient experiment at 0 dB SNR. The U-STFT grows its window duration to resolve the 500 Hz and 532 Hz tones, maintains that duration across the stationary interval, and executes immediate reset transitions ( s→0 ) at t=2 s and t=3 s to localize the broadband impulses.
Fig. 6: Spectrogram comparison for five seconds of English speech (VCTK speaker p227) on a shared decibel scale: (a) the shortest fixed window ( L=H ), (c) the U-STFT, and (e) the longest fixed window ( L=16H ); (b), (d), and (f) enlarge two stop bursts and the following vowel onset between 0.3 and 0.5 s. (g) Posterior window distribution μm(s) with the dominant U-STFT window. The U-STFT holds the long window through voiced speech and resets to short windows at broadband onsets.
Fig. 7: Spectrogram comparison on measured marine bioacoustic recordings: humpback whale vocalizations (left) and snapping shrimp cavitation impulses (right) analyzed via the U-STFT, greedy framewise selection, and fixed-window STFT baselines.