DUGAE: Unified Geometry and Attribute Enhancement via Spatiotemporal Correlations for G-PCC Compressed Dynamic Point Clouds
Authors: Pan Zhao, Hui Yuan, Chang Sun, Chongzhen Tian, Raouf Hamzaoui, Sam Kwong
Organizations: School of Control Science and Engineering, Shandong University, Jinan 250061, China, and also with the Key Laboratory of Machine Intelligence and System Control, Ministry of Education, Ji’nan, 250061, China · School of Engineering, Infrastructure and Sustainability De Montfort University, LE1 9BH Leicester, UK · Department of Computing and Decision Science, Lingnan University, Hong Kong
Existing post-decoding quality enhancement methods for point clouds are designed for static data and typically process each frame independently. As a result, they cannot effectively exploit the spatiotemporal correlations present in point cloud sequences.We propose a unified geometry and attribute enhancement framework (DUGAE) for G-PCC compressed dynamic point clouds that explicitly exploits inter-frame spatiotemporal correlations in both geometry and attributes. First, a dynamic geometry enhancement network (DGE-Net) based on sparse convolution (SPConv) and feature-domain geometry motion compensation (GMC) aligns and aggregates spatiotemporal information. Then, a detail-aware k-nearest neighbors (DA-KNN) recoloring module maps the original attributes onto the enhanced geometry at the encoder side, improving mapping completeness and preserving attribute details. Finally, a dynamic attribute enhancement network (DAE-Net) with dedicated temporal feature extraction and feature-domain attribute motion compensation (AMC) refines attributes by modeling complex spatiotemporal correlations. On seven dynamic point clouds from the 8iVFB v2, Owlii, and MVUB datasets, DUGAE significantly enhanced the performance of the latest G-PCC geometry-based solid content test model (GeS-TM v10). For geometry (D1), it achieved an average BD-PSNR gain of 11.03 dB and a 93.95% BD-bitrate reduction. For the luma component, it achieved a 4.23 dB BD-PSNR gain with a 66.61% BD-bitrate reduction. DUGAE also improved perceptual quality (as measured by PCQM) and outperformed V-PCC. Our source code will be released on GitHub at: https://github.com/yuanhui0325/DUGAE
Figures & tables
Fig. 1: Relationship between geometry error and color error.
Fig. 2: Overall architecture of DUGAE for G-PCC compressed dynamic point clouds.
Fig. 3: DGE-Net and DAE-Net architectures.
Rate level
R01
R02
R03
R04
R05
GeS-TM PQS
0.125
0.25
0.50
0.75
0.875
GeS-TM QP
51
46
40
34
28
V-PCC QP-G
32
28
24
20
16
V-PCC QP-A
42
37
32
27
22
TABLE I: Quantization Parameters of GeS-TM and V-PCC for All Dynamic Point Cloud Test Sequences.
Point cloud
BD-BR (%)
BD-PSNR (dB)
1 -PCQM
D1
D2
Y
U
V
D1
D2
Y
U
V
BD-BR (%)
BD-PCQM ( 10−3 )
redandblack
-92.59
-89.68
-63.60
-55.15
-57.84
11.30
10.30
3.85
2.00
3.56
-78.15
16.68
soldier
-95.19
-92.83
-65.29
-41.69
-55.97
11.51
10.61
3.36
1.64
2.37
-84.70
16.90
exercise
-95.59
-93.61
-65.20
-63.72
-66.20
12.78
11.58
3.30
1.48
2.14
-79.24
8.47
model
-94.54
-91.71
-66.06
-67.34
-70.98
12.34
10.67
4.35
1.50
3.02
-80.90
13.32
phil
-92.68
-85.82
-70.58
-58.61
-69.96
9.69
8.01
4.91
1.49
2.19
-82.48
18.93
TABLE II: Quantitative gains of DUGAE compared to GeS-TM v10.
Point cloud
BD-BR (%)
BD-PSNR (dB)
1 -PCQM
D1
D2
Y
U
V
D1
D2
Y
U
V
BD-BR (%)
BD-PCQM ( 10−3 )
redandblack
-8.58
0.37
-4.30
-11.72
-7.32
0.45
0.37
0.13
0.18
0.26
-1.32
0.03
soldier
-10.76
-4.45
-1.76
-34.66
-25.41
0.51
-4.45
0.04
0.66
0.51
-3.66
0.11
exercise
-11.32
1.79
-1.82
-16.15
-12.81
0.58
1.79
0.03
0.08
0.12
-3.32
0.15
model
-8.15
1.47
-2.69
-19.20
-11.72
0.35
1.47
0.08
0.26
0.20
-10.38
0.66
phil
-4.02
2.21
-2.85
-9.18
-8.39
0.15
2.21
0.09
0.15
0.16
1.49
-0.06
TABLE III: Quantitative gains of DUGAE compared to UGAE.
Fig. 4: D1-PSNR vs. geometry bitrate for DUGAE, UGAE, V-PCC, GeS-TM, and DPCGC.
Fig. 5: YUV PSNR vs. attribute bitrate for DUGAE, UGAE, V-PCC, and Ge-S-TM.
Fig. 6: Comparison of visual reconstruction quality. The bitrates are expressed in bpip.
Point Cloud
DGE (GPU) (s)
DGE (CPU) (s)
DAE (GPU) (s)
Redandblack
4.90
30.46
1.69
Soldier
5.29
42.03
1.76
Exercise
4.97
122.74
1.67
Model
5.51
117.41
1.81
Phil
5.50
63.83
1.71
Ricardo
6.05
36.92
1.81
TABLE IV: AVERAGE PROCESSING TIME PER FRAME for Different Components of DUGAE.
Fig. 7: R-D Curves for DUGAE with and without the GMC, AMC, and ATFE components on Redandblack and Soldier Dynamic Point Cloud Sequences.
Fig. 8: Subjective quality comparison and R–D curves for Redandblack. From left to right: (a) GeS-TM reconstructed attributes using compressed geometry (GeS-TM), (b) GeS-TM reconstructed attributes using enhanced geometry (DUGAE), (c) R-D curves of DUGAE and GeS-TM.
University of Southern California, 3740 McClintock Ave., Los Angeles, CA 90089-2564, USA · KDDI Research, Inc., 2-1-15 Ohara, Fujimino, Saitama, 356-8502, Japan