Mix&Fix-Net: A Dual-Stage Trajectory Prediction Model for AIS and Vision-Derived Vessel Data

Md Mahmuddun Nabi Murad; Bora San Turgut; Yasin Yilmaz

cs.CVAug 20, 2026

Mix&Fix-Net: A Dual-Stage Trajectory Prediction Model for AIS and Vision-Derived Vessel Data

Authors: Md Mahmuddun Nabi Murad, Bora San Turgut, Yasin Yilmaz

Organizations: Department of Electrical Engineering, University of South Florida, Tampa, FL, 33647, USA · College of Transportation and Logistics, Istanbul University, Istanbul, Turkey

Abstract

Vessel trajectory prediction is critical for maritime safety and accident prevention. While most existing trajectory prediction models rely on Automatic Identification System (AIS) data due to its precision and availability, small vessels mostly operate without AIS, resulting in a significant monitoring gap. To address this, we propose Mix&Fix-Net, a dual-stage mixer-based trajectory prediction model designed to handle vessel trajectory time-series data derived from both AIS and (non-AIS) vision data. Our architecture integrates a Primary Trajectory Predictor with a Residual Trajectory Adjuster, enabling more refined trajectory prediction. Additionally, we introduce a new video-based dataset derived from webcam streams, from which vessel trajectories are extracted to represent non-AIS data. Extensive evaluations on both AIS and non-AIS datasets across six metrics (mean squared error, mean absolute error, symmetric mean absolute percentage error, final displacement error, Frechet distance, and average Euclidean distance) demonstrate that Mix&Fix-Net consistently outperforms existing baselines across most metrics and datasets.

Figures & tables

Figure 1: Overall methodological framework of our vessel trajectory prediction pipeline. The framework supports two alternative input modalities: non-AIS (webcam video) processed through a vessel detector and tracker, or AIS data. Mix&Fix-Net operates on either the non-AIS or the AIS path independently (the two modalities are not fused or used simultaneously). The selected modality produces time-series trajectories that are passed through a data preprocessing stage and fed into Mix&Fix-Net to predict the trajectory.

Figure 2: Architecture of our proposed Mix&Fix-Net, an MLP-mixer-based model, for vessel trajectory prediction.

Panel of Figure 3 — Figure 3: Vessel trajectory maps of the study areas for the VVR, M3, and TampaBay datasets.

Panel of Figure 4 — Figure 4: Vessel tracking in the VVR dataset in different weather conditions. Each vessel is assigned a new vessel-ID.

S/N	Recording Date	Start time	Duration	Purpose
0	2/12/2025	13:02:00	6 Hours	Detector training
1	11/19/2024	8:01:00	4 Hours	Detector training
2	11/19/2024	12:17:00	4 Hours	Trajectory Prediction
3	11/19/2024	16:26:00	38 Minutes	Trajectory Prediction
4	11/20/2024	8:12:00	4 Hours	Trajectory Prediction
5	2/12/2025	7:50:00	5 Hours	Trajectory Prediction

Table 1: Details of the VVR dataset. The dataset consists of six video recordings. Vessels are manually annotated in two videos to train the vessel detector, while the remaining four videos are used to convert vessel trajectories into time series representations. Times reported in the table correspond to local time.

	Setting-1			Setting-2
	M3	TampaBay	VVR	M3	TampaBay	VVR
Train	367039	133740	119312	369845	200719	117421
Validation	127752	49651	37361	123282	66940	39140
Test	121618	60817	39029	123282	67046	39141
Total	616409	244208	195702	616409	334705	195702

Table 2: Number of observation-prediction sequence pairs.

		Mix&Fix-Net	WPMixer	P_sLSTM	PatchTST	Dlinear	FusionRNN
VVR	MSE	5.22E-04	7.76E-04	6.71E-04	8.18E-04	9.83E-04	9.22E-04
	MAE	1.58E-02	1.61E-02	1.58E-02	1.66E-02	2.05E-02	2.35E-02
	SMA.	4.60E-02	5.22E-02	4.78E-02	5.44E-02	6.31E-02	7.01E-02
	FDE	2.69E-02	3.05E-02	2.95E-02	3.27E-02	3.99E-02	3.60E-02
	FD	3.21E-02	3.29E-02	3.13E-02	3.36E-02	4.06E-02	3.96E-02
	AED	1.28E-02	1.30E-02	1.30E-02	1.35E-02	1.63E-02	1.88E-02

Table 3: Performance comparison under data split setting-1. For each model, the best results are reported. The best-performing model is highlighted in bold, while the second-best is underlined.

		Mix&Fix-Net	WPMixer	P_sLSTM	PatchTST	Dlinear	FusionRNN
VVR	MSE	3.86E-06	7.38E-04	5.33E-04	8.33E-04	1.16E-03	9.43E-06
	MAE	1.45E-03	1.59E-02	1.45E-02	1.72E-02	2.20E-02	2.58E-03
	SMA.	4.08E-03	5.04E-02	4.40E-02	5.47E-02	6.76E-02	7.47E-03
	FDE	1.64E-03	3.01E-02	2.68E-02	3.29E-02	4.15E-02	3.13E-03
	FD	3.72E-03	3.25E-02	2.85E-02	3.41E-02	4.23E-02	5.35E-03
	AED	1.16E-03	1.29E-02	1.19E-02	1.39E-02	1.75E-02	2.06E-03

Table 4: Performance comparison under data split setting-2. For each model, the best results are reported. The best-performing model is highlighted in bold, while the second-best is underlined. Note that this setting involves a risk of temporal data leakage due to overlapping trajectory segments; consequently, these results do not reflect the realistic generalization performance on unseen vessels.

Panel of Figure 5 — Figure 5: Visual comparison of the prediction among different models. Trajectories are randomly selected. The first two rows refer to the VVR dataset, while the last two rows refer to the M3 dataset. Predicted trajectories for the TampaBay dataset are shown in Figure 6 in the Appendix.

		Mix&Fix-Net		WPMixer		P_sLSTM
		Mean	Std.	Mean	Std.	Mean	Std.
VVR	MSE	4.12E-06	2.09E-07	7.65E-04	1.89E-05	5.33E-04	4.51E-06
	MAE	1.46E-03	3.29E-05	1.61E-02	1.69E-04	1.43E-02	1.46E-04
	SMAPE	4.11E-03	9.49E-05	5.12E-02	5.68E-04	4.37E-02	2.59E-04
	FDE	1.64E-03	2.39E-05	3.09E-02	5.94E-04	2.67E-02	9.26E-05
	FD	3.71E-03	4.89E-05	3.35E-02	7.89E-04	2.84E-02	1.05E-04

Table 5: Robustness comparison. Each model is evaluated using four random seed values.

	Train Time	Test time	GFLOPs
Mix&Fix-Net	0.086	0.030	1136
WPMixer	0.011	0.005	11
P_sLSTM	0.029	0.010	3
PatchTST	0.010	0.006	35
Dlinear	0.008	0.004	0.03
FusionRNN	0.008	0.003	14

Table 6: Computational complexity measured by average iteration time (in seconds) for NVIDIA 4090 GPU and GFLOPs for the VVR dataset.

Case	RTA	Emb. Mx	Tm. Mx	Head	Padding	RevIN	$\Delta$ MSE	$\Delta$ MAE	$\Delta$ SMAPE	$\Delta$ FDE	$\Delta$ FD	$\Delta$ AED
1	✓	✓	✓	$\spadesuit$	✓	$D$	0	0	0	0	0	0
2	$\times$	✓	✓	$\spadesuit$	✓	$D$	14	5	4	7	2	5
3	✓	✓	$\times$	$\spadesuit$	✓	$D$	157906	4562	3932	5312	2365	4565
4	✓	$\times$	✓	$\spadesuit$	✓	$D$	322	131	140	163	87	131
5	✓	✓	✓	$\spadesuit$	✓	$ND$	231	67	66	98	44	68
6	✓	✓	✓	$\spadesuit$	✓	-	39	23	26	27	14	22

Table 7: Relative change in six metrics with respect to the base case (Case-1). All values are reported as percentages, while the corresponding absolute values are shown in Table 12 . Symbol

\spadesuit

in the Head column refers to our proposed head layer design, while symbol

\diamondsuit

refers to the regular head layer design for time series forecasting models. The "

D

" in the RevIN column refers only to denormalization, whereas "

ND

" refers to both normalization and denormalization. RTA refers to the Residual Trajectory Adjuster module.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Description	Value
hsv_h	Image HSV-Hue augmentation (fraction)	0.015
hsv_s	Image HSV-Saturation augmentation (fraction)	0.7
hsv_v	Image HSV-Value augmentation (fraction)	0.4
translate	Image translation (+/- fraction)	0.1
scale	Image scale (+/- gain)	0.5
fliplr	Image flip left-right (probability)	0.5

Appendix

Table 8: Data augmentation parameters used for YOLOv11 fine-tuning on the VVR dataset.

Parameter	Description	Value
track_high_thresh	Threshold for the first association (high score)	0.5
track_low_thresh	Threshold for the second association (low score)	0.1
new_track_thresh	Threshold for initiating a new track	0.5
track_buffer	Number of frames to retain a lost track	600
match_thresh	Intersection over Union (IoU) matching threshold	0.95

Appendix

Table 9: Experimental settings of the ByteTrack algorithm. Parameter definitions follow the standard implementation in [ 34 ] .

Setting	Data	$L$	$T$	Initial lr.	Batch	$d$	$t_{f}$	$d_{f}$	drop.	Epochs	Patience
1	VVR	60	120	8.13E-05	128	16	5	2	0.05	50	10
	M3	60	120	0.004887	512	16	5	1	0.05	50	10
	TampaBay	60	120	6.72E-05	128	16	1	3	0.05	50	10
2	VVR	60	120	0.001172	128	256	5	5	0	50	10
	M3	60	120	0.001392	512	256	3	5	0	50	10
	TampaBay	60	120	8.18E-05	128	128	5	2	0.1	50	10

Appendix

Table 10: Hyperparameter settings of the proposed model across benchmark datasets under both experimental settings. Here,

L

and

T

denote the lookback window length and the prediction window length, respectively. Additionally,

d

t_{f}

d_{f}

, drop. refer to the embedding dimension, temporal expansion factor, embedding expansion factor, and dropout value, respectively.

Panel of Figure 6 — Figure 6: Visual comparison of the predicted trajectory among different models for the TampaBay dataset. Trajectories are randomly selected

	$C_{i}$	2	7	9
VVR	MSE	2.16E-05	4.63E-06	3.86E-06
	MAE	3.50E-03	1.49E-03	1.45E-03
	SMAPE	1.02E-02	4.18E-03	4.08E-03
	FDE	4.92E-03	1.70E-03	1.64E-03
	FD	7.08E-03	3.79E-03	3.72E-03
	AED	2.82E-03	1.19E-03	1.16E-03

Appendix

Table 11: Impact of the non-positional features.

C_{i}

denotes the number of input features:

C_{i}=2

(positional only),

C_{i}=7

(all features excluding

D_{sin}/D_{cos}

), and

C_{i}=9

(full feature set).

Figure 7: Impact of embedding dimension $d$ on prediction performance and computational cost for the VVR dataset.

Figure 8: Impact of observation window $L$ on prediction performance for the VVR dataset.

Panel of Figure 9 — Figure 9: Histogram of the Vessel trajectory length.

Case	RTA	Emb. Mx	Tm. Mx	Head	Padding	RevIN	MSE	MAE	SMAPE	FDE	FD	AED
1	✓	✓	✓	$\spadesuit$	✓	$D$	3.86E-06	1.45E-03	4.08E-03	1.64E-03	3.72E-03	1.16E-03
2	$\times$	✓	✓	$\spadesuit$	✓	$D$	4.41E-06	1.52E-03	4.26E-03	1.75E-03	3.79E-03	1.22E-03
3	✓	✓	$\times$	$\spadesuit$	✓	$D$	6.10E-03	6.77E-02	1.64E-01	8.88E-02	9.17E-02	5.40E-02
4	✓	$\times$	✓	$\spadesuit$	✓	$D$	1.63E-05	3.35E-03	9.79E-03	4.31E-03	6.96E-03	2.67E-03
5	✓	✓	✓	$\spadesuit$	✓	$ND$	1.28E-05	2.42E-03	6.79E-03	3.25E-03	5.35E-03	1.95E-03
6	✓	✓	✓	$\spadesuit$	✓	-	5.37E-06	1.78E-03	5.13E-03	2.08E-03	4.24E-03	1.42E-03

Appendix

Table 12: Absolute values of six metrics for each ablation case, corresponding to the relative percentage changes reported in Table 7 . Symbol

\spadesuit

in the Head column refers to our proposed head layer design, while symbol

\diamondsuit

refers to the regular head layer design for time series forecasting models. The "

D

" in the RevIN column refers only to denormalization, whereas "

ND

" refers to both normalization and denormalization. RTA refers to the Residual Trajectory Adjuster module.

Explore similar work

CmIVTP: Cross-modal Interaction-based Vessel Trajectory Prediction for Maritime Intelligence
May 26, 2026Yuxu Lu, Dong Yang, Xiaoyu Li +2Intelligent Transportation Systems Vessel Trajectory Prediction
Department of Logistics and Maritime Studies, the Hong Kong Polytechnic University, Hong Kong 999077 · Department of Logistics and Maritime Studies and the Research Centre for ESG Advancement (RCESGA), the Hong Kong Polytechnic University, Hong Kong 999077 · Department of Logistics and Maritime Studies, the Hong Kong Polytechnic University, Hong Kong 999077, and also with the School of Navigation, Wuhan University of Technology, Wuhan 430063
AIS-Based Vessel Trajectory Prediction Using Memory-Augmented Neural Networks
Jun 4, 2026Wonmo Koo, Sanha Chang, Heeyoung KimMemory-Augmented Neural Networks Vessel Trajectory Prediction
Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.
NaviAIS: A Scenario-Level Vessel Trajectory Prediction Dataset withVectorized Lane Priors and the NaviLane Forecasting Framework
Jul 21, 2026Yuan Gui, Hongchen Luo, Liqi Qu +2Vessel Trajectory Prediction Multimodal Trajectory Prediction
Northeastern University, Shenyang 110819, China

May 26, 2026J/K move · Enter open · S save

CmIVTP: Cross-modal Interaction-based Vessel Trajectory Prediction for Maritime Intelligence

Yuxu Lu, Dong Yang, Xiaoyu Li, Mengwei Bao+1

Department of Logistics and Maritime Studies, the Hong Kong Polytechnic University, Hong Kong 999077 · Department of Logistics and Maritime Studies and the Research Centre for ESG Advancement (RCESGA), the Hong Kong Polytechnic University, Hong Kong 999077 · Department of Logistics and Maritime Studies, the Hong Kong Polytechnic University, Hong Kong 999077, and also with the School of Navigation, Wuhan University of Technology, Wuhan 430063

Intelligent Transportation Systems Vessel Trajectory Prediction Multimodal Trajectory Prediction Trajectory Prediction Multimodal Sensor Fusion

Open paper AskPDF arXiv

Maritime intelligent transportation systems (MITS) are essential for ensuring navigation safety and efficiency in busy waterways. However, accurate vessel trajectory prediction remains challenging due to the limitations of single-source data. Automatic identification system (AIS) data is often sparse or unavailable for small vessels, while closed-circuit television (CCTV) data alone cannot fully capture dynamic vessel behavior. To mitigate these challenges, we propose a cross-modal interaction-based vessel trajectory prediction (named CmIVTP) framework to model the intricate interactions between vessel dynamics and environmental constraints. Specifically, we introduce a target-aware scene encoder to extract scene semantic features, effectively capturing vessel-environment interactions and enhancing trajectory prediction accuracy. In addition, we propose a cross-modal interaction transformer, which integrates AIS-derived motion features, CCTV-based environmental features, and scene representations. It leverages cross-modal attention mechanisms to simultaneously capture intra-modal semantics and inter-modal interactions, ensuring dynamically consistent and environmentally feasible predictions. Furthermore, we construct a vessel group trajectory bank by clustering historical AIS trajectories into representative motion patterns, providing an efficient and scalable approach for candidate trajectory generation. Additionally, we introduce the maritime multimodal dataset plus (named Maritime-MmD

^+

), a large-scale dataset that synchronizes AIS data and CCTV video data, providing robust support for multimodal trajectory prediction research. Extensive experiments demonstrate that CmIVTP achieves better performance on multimodal-driven vessel trajectory prediction benchmarks. The code resources for this work can be available at https://github.com/LouisYxLu/CmIVTP.

Abstract

Explore similar work

CmIVTP: Cross-modal Interaction-based Vessel Trajectory Prediction for Maritime Intelligence

AIS-Based Vessel Trajectory Prediction Using Memory-Augmented Neural Networks

NaviAIS: A Scenario-Level Vessel Trajectory Prediction Dataset withVectorized Lane Priors and the NaviLane Forecasting Framework