DUGAE: Unified Geometry and Attribute Enhancement via Spatiotemporal Correlations for G-PCC Compressed Dynamic Point Clouds
Authors: Pan Zhao, Hui Yuan, Chang Sun, Chongzhen Tian, Raouf Hamzaoui, Sam Kwong
Organizations: School of Control Science and Engineering, Shandong University, Jinan 250061, China, and also with the Key Laboratory of Machine Intelligence and System Control, Ministry of Education, Ji’nan, 250061, China · School of Engineering, Infrastructure and Sustainability De Montfort University, LE1 9BH Leicester, UK · Department of Computing and Decision Science, Lingnan University, Hong Kong
Existing post-decoding quality enhancement methods for point clouds are designed for static data and typically process each frame independently. As a result, they cannot effectively exploit the spatiotemporal correlations present in point cloud sequences.We propose a unified geometry and attribute enhancement framework (DUGAE) for G-PCC compressed dynamic point clouds that explicitly exploits inter-frame spatiotemporal correlations in both geometry and attributes. First, a dynamic geometry enhancement network (DGE-Net) based on sparse convolution (SPConv) and feature-domain geometry motion compensation (GMC) aligns and aggregates spatiotemporal information. Then, a detail-aware k-nearest neighbors (DA-KNN) recoloring module maps the original attributes onto the enhanced geometry at the encoder side, improving mapping completeness and preserving attribute details. Finally, a dynamic attribute enhancement network (DAE-Net) with dedicated temporal feature extraction and feature-domain attribute motion compensation (AMC) refines attributes by modeling complex spatiotemporal correlations. On seven dynamic point clouds from the 8iVFB v2, Owlii, and MVUB datasets, DUGAE significantly enhanced the performance of the latest G-PCC geometry-based solid content test model (GeS-TM v10). For geometry (D1), it achieved an average BD-PSNR gain of 11.03 dB and a 93.95% BD-bitrate reduction. For the luma component, it achieved a 4.23 dB BD-PSNR gain with a 66.61% BD-bitrate reduction. DUGAE also improved perceptual quality (as measured by PCQM) and outperformed V-PCC. Our source code will be released on GitHub at: https://github.com/yuanhui0325/DUGAE
Figures & tables
Fig. 1: Relationship between geometry error and color error.
Fig. 2: Overall architecture of DUGAE for G-PCC compressed dynamic point clouds.
Fig. 3: DGE-Net and DAE-Net architectures.
Rate level
R01
R02
R03
R04
R05
GeS-TM PQS
0.125
0.25
0.50
0.75
0.875
GeS-TM QP
51
46
40
34
28
V-PCC QP-G
32
28
24
20
16
V-PCC QP-A
42
37
32
27
22
TABLE I: Quantization Parameters of GeS-TM and V-PCC for All Dynamic Point Cloud Test Sequences.
Point cloud
BD-BR (%)
BD-PSNR (dB)
1 -PCQM
D1
D2
Y
U
V
D1
D2
Y
U
V
BD-BR (%)
BD-PCQM ( 10−3 )
redandblack
-92.59
-89.68
-63.60
-55.15
-57.84
11.30
10.30
3.85
2.00
3.56
-78.15
16.68
soldier
-95.19
-92.83
-65.29
-41.69
-55.97
11.51
10.61
3.36
1.64
2.37
-84.70
16.90
exercise
-95.59
-93.61
-65.20
-63.72
-66.20
12.78
11.58
3.30
1.48
2.14
-79.24
8.47
model
-94.54
-91.71
-66.06
-67.34
-70.98
12.34
10.67
4.35
1.50
3.02
-80.90
13.32
phil
-92.68
-85.82
-70.58
-58.61
-69.96
9.69
8.01
4.91
1.49
2.19
-82.48
18.93
TABLE II: Quantitative gains of DUGAE compared to GeS-TM v10.
Point cloud
BD-BR (%)
BD-PSNR (dB)
1 -PCQM
D1
D2
Y
U
V
D1
D2
Y
U
V
BD-BR (%)
BD-PCQM ( 10−3 )
redandblack
-8.58
0.37
-4.30
-11.72
-7.32
0.45
0.37
0.13
0.18
0.26
-1.32
0.03
soldier
-10.76
-4.45
-1.76
-34.66
-25.41
0.51
-4.45
0.04
0.66
0.51
-3.66
0.11
exercise
-11.32
1.79
-1.82
-16.15
-12.81
0.58
1.79
0.03
0.08
0.12
-3.32
0.15
model
-8.15
1.47
-2.69
-19.20
-11.72
0.35
1.47
0.08
0.26
0.20
-10.38
0.66
phil
-4.02
2.21
-2.85
-9.18
-8.39
0.15
2.21
0.09
0.15
0.16
1.49
-0.06
TABLE III: Quantitative gains of DUGAE compared to UGAE.
Fig. 4: D1-PSNR vs. geometry bitrate for DUGAE, UGAE, V-PCC, GeS-TM, and DPCGC.
Fig. 5: YUV PSNR vs. attribute bitrate for DUGAE, UGAE, V-PCC, and Ge-S-TM.
Fig. 6: Comparison of visual reconstruction quality. The bitrates are expressed in bpip.
Point Cloud
DGE (GPU) (s)
DGE (CPU) (s)
DAE (GPU) (s)
Redandblack
4.90
30.46
1.69
Soldier
5.29
42.03
1.76
Exercise
4.97
122.74
1.67
Model
5.51
117.41
1.81
Phil
5.50
63.83
1.71
Ricardo
6.05
36.92
1.81
TABLE IV: AVERAGE PROCESSING TIME PER FRAME for Different Components of DUGAE.
Fig. 7: R-D Curves for DUGAE with and without the GMC, AMC, and ATFE components on Redandblack and Soldier Dynamic Point Cloud Sequences.
Fig. 8: Subjective quality comparison and R–D curves for Redandblack. From left to right: (a) GeS-TM reconstructed attributes using compressed geometry (GeS-TM), (b) GeS-TM reconstructed attributes using enhanced geometry (DUGAE), (c) R-D curves of DUGAE and GeS-TM.
Plenoptic point clouds (PPC) are novel data structures that represent the light from different viewing directions in order to provide a higher degree of realism to regular point clouds. This is achieved by associating each point to multiple colors instead of a single one. Here, we present a method to efficiently compress the attributes of a PPC, consisting of a Karhunen-Loève transform over the color attributes followed by multiple attribute coders with intra prediction capability. This compression scheme can be incorporated within the MPEG's geometry-based PCC (G-PCC) standard, using any of G-PCC's existing solutions for attribute coding. Compression performance assessment using PPCs of different spatial resolutions reveals competitive results in comparison to existing methods, such as RAHT-based or video-based PCC solutions. We believe our coder to be the new state of the art.
Davi R. Freitas, Gustavo L. Sandri, Ricardo L. de Queiroz
Inria - Rennes Bretagne-Atlantique Rennes, France · Instituto Federal de Bras´ılia Bras´ılia, Brazil · Universidade de Bras´ılia Bras´ılia, Brazil
Point cloud compression relies on techniques to compress both geometry and attributes. Motion-based approaches for dynamic solid point cloud geometry compression within the geometry-based point cloud compression (G-PCC) framework have achieved significant reductions in geometry rate. However, motion-based techniques for attribute compression remain underexplored, making it challenging to achieve significant reductions in the temporal redundancy of attributes. Firstly, this paper proposes a geometry-based inter-coding scheme to compress the attributes of dynamic solid point clouds. Secondly, a graph-based motion-estimation scheme for point-cloud attribute compression is proposed. Thirdly, an interpolation-free fractional-voxel motion estimation method is proposed to refine motion accuracy to fractional-voxel precision. Our experimental results on the MPEG point cloud dataset show that the proposed scheme outperforms G-PCC, GeS-TM, and V-PCC in lossless and lossy geometry conditions. We achieve average bitrate savings of 55.3%, 42.3%, and 16.5% over G-PCC, GeS-TM, and V-PCC, respectively, under lossy-geometry conditions.
Haoran Hong, Eduardo Pavez, Antonio Ortega +2
University of Southern California, 3740 McClintock Ave., Los Angeles, CA 90089-2564, USA · KDDI Research, Inc., 2-1-15 Ohara, Fujimino, Saitama, 356-8502, Japan
Scalable compression is essential for bandwidth-adaptive transmission, yet most learned codecs are optimized for a fixed rate-distortion point, making rate adaptation costly due to re-encoding or maintaining multiple bitstreams. In this work, we propose TAFA-GSGC, a scalable learned point cloud geometry codec that enables multi-quality decoding from a single bitstream and a single trained model. TAFA-GSGC combines layered residual refinement with channel-group entropy coding, and introduces a Target-Aligned Feature Aggregation module to reduce cross-layer redundancy in enhancement residuals. Our framework supports up to 9 decodable quality levels with monotonic quality improvement as more subbitstreams are received, while maintaining strong compression efficiency. Compared with the PCGCv2 baseline, TAFA-GSGC demonstrates improved RD performance, achieving average BD-rate reductions of 4.99% and 5.92% in terms of D1-PSNR and D2-PSNR, respectively.
Xiumei Li, Alexander Kopte, André Kaup
Chair of Multimedia Communications and Signal Processing Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) Erlangen, Germany