Rondo: Unsupervised Discovery of Recurring Temporal Structure

Yingtian Shi; Ankith Chandra; Thomas Plötz

cs.LGSep 26, 2026

Rondo: Unsupervised Discovery of Recurring Temporal Structure

Authors: Yingtian Shi, Ankith Chandra, Thomas Plötz

Organizations: School of Interactive Computing Georgia Institute of Technology

Abstract

Many real-world time series data exhibit structural properties at multiple scales, from short, recurring units to complex sequences composed of these units. Unsupervised discovery of both these components and structure enables the design of intelligent systems that help interpret temporal data, thereby limiting the amount of costly human annotations required. Existing modeling approaches typically overlook the hierarchical structure inherent to many time series, treating recurring patterns at different temporal scales as independent structures. Moreover, most assume access to the complete data sequence and treat discovery as a static process, limiting their ability to evolve as new observations arrive. We introduce Rondo, an unsupervised approach for modeling recurring hierarchical structure in continuous temporal streams. By explicitly constructing vocabularies of reusable units and their recurring compositions, Rondo captures structure shared across complex temporal patterns while refining and expanding its discoveries as the stream evolves. Evaluations on temporal sequences spanning diverse domains and data modalities show that Rondo outperforms existing unsupervised recurrence-discovery baselines, with particularly pronounced advantages in limited-data and continual-stream settings. These capabilities provide a stronger foundation for recurring-pattern discovery, scalable behavior understanding, and adaptive intelligent systems operating on long, unlabeled temporal streams.

Figures & tables

Figure 1: RONDO discovers recurring temporal structure hierarchically. (a) Temporal streams exhibit recurring local units that compose into longer-scale motifs. (b) Current methods can fragment such structure and cause over-segmentation, while Rondo preserves the unit-motif hierarchy. (c) Rondo learns reusable units and composes recurring unit sequences into motifs.

Figure 2: Overview of Rondo . (a) A continuous temporal stream is divided into sliding windows. (b) UnitAlign encodes and adapts each window, clusters recurring local structure into a reusable unit vocabulary, and smooths unit assignments. (c) MotifFormer models unit sequences across temporal scales to discover higher-level motifs, followed by a soft coverage filter. (d) During continual operation, unmatched windows and unit subsequences are stored in novelty buffers and, once sufficient evidence is available, used to refine existing entries or introduce new units and motifs.

Method	MoF	Recall	F1@50	Edit	SegRatio	Method	MoF	Recall	F1@50	Edit	SegRatio
MoCap						Drive&Act
Time2State	88.68	91.49	84.48	83.80	1.70	Time2State	54.23	29.55	7.00	12.97	11.31
E2USD	84.97	87.45	77.95	76.48	1.97	E2USD	47.60	18.90	5.66	11.80	14.47
TICC	86.61	84.52	79.39	83.18	2.19	TICC	49.47	23.71	6.61	13.01	9.14
AutoPlait	90.27	90.28	91.18	92.05	0.97	AutoPlait	39.42	8.08	0.26	4.32	0.04
CompILE	66.45	61.05	52.19	63.14	1.97	CompILE	46.17	17.20	9.04	32.69	4.13

Table 1: Recurring structure discovery results (%). Stochastic methods report mean performance over five random seeds, with standard deviations provided in Appendix F ; deterministic or officially fixed-seed methods are evaluated once. CompILE is evaluated once due to its computational cost. Higher is better for MoF, Recall, F1@50, and Edit. For SegRatio, values closer to 1 are better, so ranked with

|\log\mathrm{SegRatio}|

and lower is better. Best in bold , second-best underlined.

Method	MoF	Recall	F1@50	Edit	SegRatio	H@40	H@60	H@80	H@100	$M_{f}$
OPPORTUNITY
Time2State@20	44.70	36.20	7.00	36.00	5.58	0.00	0.00	32.00	20.50	24.00
DBSTREAM@20	47.10	34.00	8.00	26.20	6.02	30.50	36.60	44.60	39.00	25.80
DenStream@20	59.90	45.90	15.10	23.56	8.75	11.40	37.60	57.80	58.30	12.40
Rondo @20	58.10	42.90	21.10	44.70	4.03	23.40	38.70	65.40	63.50	14.60
PAMAP2

Table 2: Continual discovery on progressively revealed streams. H@40–100 denotes H-score after observing 40–100% of the stream.

Method	MoF	Recall	F1@50	Edit	SegRatio	Method	MoF	Recall	F1@50	Edit	SegRatio
LARa						Drive&Act
Time2State	50.62	28.56	16.67	40.72	1.59	Time2State	44.80	17.53	4.99	14.07	14.37
E2USD	45.59	21.20	12.06	41.64	1.78	E2USD	43.75	14.94	6.00	13.34	14.68
Rondo	62.20	43.89	29.72	45.32	1.24	Rondo	54.27	38.11	8.97	23.42	7.98
Rondo -fit	63.48	44.05	31.35	47.83	1.25	Rondo -fit	58.66	46.50	9.84	18.94	9.28

Table 3: Cross-record transfer results on LARa and Drive&Act. Rondo -fit updates only MotifFormer on the target record.

Method	MoF	Recall	F1@50	Edit	SegRatio	Method	MoF	Recall	F1@50	Edit	SegRatio
OPPORTUNITY						LARa
w/o MF	78.70	58.20	28.90	49.48	1.13	w/o MF	76.60	48.80	36.30	47.66	1.40
w/o UA	76.90	65.40	35.00	47.11	0.86	w/o UA	77.60	45.60	37.30	44.20	1.26
Rondo	79.96	59.19	30.59	48.06	1.23	Rondo	78.44	53.01	38.50	47.34	1.31
PAMAP2						Drive&Act
w/o MF	80.30	92.60	11.12	12.94	17.99	w/o MF	55.90	35.99	7.66	13.12	11.69

Table 4: Ablation of Rondo . w/o UA removes UnitAlign while retaining MotifFormer, and w/o MF removes MotifFormer while retaining unit-level discovery.

Method	MoF	Recall	F1@50	Edit	SegRatio	Method	MoF	Recall	F1@50	Edit	SegRatio
OPPORTUNITY						LARa
TS2Vec	79.96	59.19	30.59	48.06	1.23	TS2Vec	78.44	53.01	38.50	47.34	1.31
Time-MAE	78.50	65.30	28.60	46.70	1.36	Time-MAE	79.10	53.70	46.10	57.30	2.14
MantisV2	72.30	61.00	31.30	63.80	2.40	MantisV2	74.60	51.50	39.90	58.90	2.23
PAMAP2						Drive&Act
TS2Vec	84.12	94.25	22.91	20.82	15.37	TS2Vec	68.36	62.30	14.29	22.22	8.04

Table 5: Representation sensitivity of Rondo across four datasets. Results compare TS2Vec, Time-MAE, and MantisV2 under the same discovery pipeline, showing that performance remains competitive across different self-supervised and pretrained temporal representations.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Segmentation of one recording from three representative datasets, against the ground truth (bottom). Predicted states are coloured by the class they map to; a single-colour row means every predicted state mapped to the same class.

Figure 4: Illustration of the evaluation-only many-to-one mapping. Discovered clusters are matched to ground-truth labels by their duration-weighted overlap on a common temporal axis, allowing multiple clusters to map to the same semantic label.

Dataset	Source	Subj.	Rec.	Frames	Rate (Hz)	Raw ch.	Classes
MoCap	CMU MoCap, subject 86	1	9	65,611	—	4	9
PAMAP2	PAMAP2 Protocol	8	8	2,864,056	100	40	13
ActRecTut	ActRecTut	2	4	200,693	42	24	17
Drive&Act	Drive&Act, OpenPose 3D	15	29	1,222,442	30	104	13
OPPORTUNITY	OPPORTUNITY UCI	4	24	869,387	30.3	242	5
LARa	LARa v3, MbientLab IMU	8	193	2,224,452	100	30	8

Appendix

Table 6: Summary of the six evaluation datasets. Rec. denotes the number of independent recordings over which temporal segmentation is evaluated. Raw ch. denotes the dimensionality of the original input signals before method-specific preprocessing; the actual input dimensionality used by each method is reported in Table 7 .

Dataset	Raw	Time2State, E2USD	TICC, AutoPlait	CLaP, CompILE	Window / step
MoCap	4	4 (native)	4 (native)	4 (native)	256 / 50
PAMAP2	40	9 (selection)	9 (selection)	9 (selection)	200 / 100
ActRecTut	24	24 (native)	10 (PCA)	8 (PCA)	90 / 30
Drive&Act	104	104 (native)	16 (PCA)	16 (PCA)	90 / 30
OPPORTUNITY	242	54 (PCA)	54 (PCA)	54 (PCA)	30 / 15
LARa	30	30 (native)	16 (PCA)	16 (PCA)	200 / 50

Appendix

Table 7: Preprocessing settings for each dataset. Raw is the original channel count; the next three columns give the input channels each method receives (native, the published channel selection, or the number of PCA components). The window/step applies to the sliding-window methods.

Component	Setting	Value
UnitAlign	bottleneck / output dimension	256 / 128
	initialization epochs	20
	$\lambda_{\mathrm{geom}},\lambda_{\mathrm{VIC}}$	0.2 / 0.05
	refinement epochs / max rounds	3 / 20
	$\lambda_{\mathrm{sup}},\lambda_{\mathrm{var}},\lambda_{\mathrm{cov}},\lambda_{\mathrm{geom}}$	1.0 / 1.0 / 0.04 / 0.25
	SupCon temperature	0.1

Appendix

Table 8: Key hyperparameters used in Rondo . Unless otherwise stated, all settings are fixed across datasets.

Method	Discovery mechanism
AutoPlait	MDL-based joint segmentation and regime discovery
TICC	Toeplitz inverse-covariance clustering with temporal smoothness
Time2State	causal-CNN representation followed by a DPGMM
E2USD	frequency-aware representation followed by a DPGMM
CLaP	self-supervised classification and segment merging
CompILE	variational learning of segment boundaries and latent codes

Appendix

Table 9: Baseline methods and their main discovery mechanisms.

Method	Setting	Value
AutoPlait	parameter-free	–
TICC	number of states	8
	window size	1
	switch penalty	600
	sparsity	0.11
	max iterations	100

Appendix

Table 10: Hyperparameters used for the baseline methods. Values follow the released implementations or their reference experiments unless noted otherwise.

Method	MoF	Recall	F1@50	Edit	SegRatio	Method	MoF	Recall	F1@50	Edit	SegRatio
MoCap						Drive&Act
Time2State	88.68 $\pm$ 2.07	91.49 $\pm$ 2.36	84.48 $\pm$ 2.29	83.80 $\pm$ 1.17	1.70 $\pm$ 0.13	Time2State	54.23 $\pm$ 0.39	29.55 $\pm$ 1.79	7.00 $\pm$ 0.39	12.97 $\pm$ 0.47	11.31 $\pm$ 0.13
E2USD	84.97 $\pm$ 2.81	87.45 $\pm$ 6.11	77.95 $\pm$ 5.77	76.48 $\pm$ 6.78	1.97 $\pm$ 0.28	E2USD	47.60 $\pm$ 0.48	18.90 $\pm$ 0.81	5.66 $\pm$ 0.29	11.80 $\pm$ 0.19	14.47 $\pm$ 0.42
TICC	86.61 $\pm$ 0.00	84.52 $\pm$ 0.00	79.39 $\pm$ 0.00	83.18 $\pm$ 0.00	2.19 $\pm$ 0.00	TICC	49.47 $\pm$ 0.28	23.71 $\pm$ 1.39	6.61 $\pm$ 0.65	13.01 $\pm$ 0.70	9.14 $\pm$ 0.09
AutoPlait	90.27	90.28	91.18	92.05	0.97	AutoPlait	39.42	8.08	0.26	4.32	0.04
CompILE	66.45	61.05	52.19	63.14	1.97	CompILE	46.17	17.20	9.04	32.69	4.13

Appendix

Table 11: Complete results (%), mean

\pm

standard deviation over five seeds. For SegRatio, values closer to 1 are better.

Unsupervised Learning

Unsupervised learning finds structure, useful representations, or generative models in data without task labels.

See all 57 papers

Discovering Temporal Structure: An Overview of Hierarchical Reinforcement Learning
Jun 16, 2025Martin Klissarov, Akhil Bagaria, Ziyan Luo +3Hierarchical RL
Mila, McGill University · Amazon · Amii, University of Alberta Canada CIFAR AI Chair
TStruct: Learning Shared Temporal Structures for Long-Term Time-Series Forecasting
Oct 13, 2026Yangze Li, Zhengyang Zhou, Qihe Huang +3Long-Term Time Series Forecasting Multivariate Time Series Forecasting
University of Science and Technology of China (USTC), Hefei, China · Suzhou Institute for Advanced Research, USTC, Suzhou, China
SHARP: Sleep-based Hierarchical Accelerated Replay for Long Range Non-Stationary Temporal Pattern Recognition
May 30, 2026Jayanta Dey, Shikhar Srivastava, Itamar Lerner +2Memory-Augmented Neural Networks Continual Learning
University of Texas at San Antonio, USA · University of Rochester, USA

Jun 16, 2025J/K move · Enter open · S save

Discovering Temporal Structure: An Overview of Hierarchical Reinforcement Learning

Martin Klissarov, Akhil Bagaria, Ziyan Luo, George Konidaris+2

Mila, McGill University · Amazon · Amii, University of Alberta Canada CIFAR AI Chair

Hierarchical RL

Open paper AskPDF arXiv

Developing agents capable of exploring, planning and learning in complex open-ended environments is a grand challenge in artificial intelligence (AI). Hierarchical reinforcement learning (HRL) offers a promising solution to this challenge by discovering and exploiting the temporal structure within a stream of experience. The strong appeal of the HRL framework has led to a rich and diverse body of literature attempting to discover a useful structure. However, it is still not clear how one might define what constitutes good structure in the first place, or the kind of problems in which identifying it may be helpful. This work aims to identify the benefits of HRL from the perspective of the fundamental challenges in decision-making, as well as highlight its impact on the performance trade-offs of AI agents. Through these benefits, we then cover the families of methods that discover temporal structure in HRL, ranging from learning directly from online experience to offline datasets, to leveraging large language models (LLMs). Finally, we highlight the challenges of temporal structure discovery and the domains that are particularly well-suited for such endeavours.

Abstract

Unsupervised Learning

Explore similar work

Discovering Temporal Structure: An Overview of Hierarchical Reinforcement Learning

TStruct: Learning Shared Temporal Structures for Long-Term Time-Series Forecasting

SHARP: Sleep-based Hierarchical Accelerated Replay for Long Range Non-Stationary Temporal Pattern Recognition