This repository contains the implementation of the ADL-PCC codec - a Point Cloud geometry coding solution based on Deep Learning.
- Authors: André F. R. Guarda1, Nuno M. M. Rodrigues2, Fernando Pereira1
- Affiliations:
- 1 Instituto Superior Técnico - Universidade de Lisboa, and Instituto de Telecomunicações, Lisbon, Portugal
- 2 ESTG, Politécnico de Leiria and Instituto de Telecomunicações, Leiria, Portugal
- Source code: In
src, the full source code to run the ADL-PCC codec, including for the training of the Deep Learning (DL) coding models. - Trained DL coding models: In
models, all the trained DL coding models used in the ADL-PCC paper, namely 5 DL coding models trained with different loss function parameters (α = 0.5, 0.6, 0.7, 0.8 and 0.9) for each of the 5 target rate-distortion trade-offs (λ = 20000, 5000, 1500, 900 and 500), for a total of 25 DL coding models. - Experimental data: Compressed bitstream files and reconstructed Point Clouds (PCs) obtained with ADL-PCC, for the JPEG Pleno PC coding dataset described in the Common Test Conditions of the Call for Evidence.
- RD performance results: In
results/rd_charts.xlsx, rate-distortion results and accompanying charts for each PC in the JPEG Pleno PC coding dataset, corresponding to the previously mentioned experimental data. The rate-distortion charts compare the performance of ADL-PCC with the MPEG G-PCC standard, following the JPEG Pleno PC coding Common Test Conditions.
The prerequisites to run the ADL-PCC software, and the DL coding models in particular, are:
- Python 3.6.9
- Tensorflow 1.15 with CUDA Version 10.0.130 and cuDNN 7.6.5
- tensorflow-compression 1.3
- Python packages in
requirements.txt
Using a Linux distribution (e.g. Ubuntu) is recommended.
The main script src/ADLPCC.py is used to encode and decode a PC using the ADL-PCC codec, or to train a DL coding model.
python ADLPCC.py {train,compress,decompress} [OPTIONS]
python ADLPCC.py train TRAIN_DATA CHECKPOINT_DIR [OPTIONS]
positional arguments:
train_data Directory containing PC data for training. Filenames
should be provided with a glob pattern that expands
into a list of PCs: data/*.ply
checkpoint_dir Directory where to save model checkpoints. For training,
a single directory should be provided: ../models/test
optional arguments:
-h, --help show this help message and exit
--batchsize BATCHSIZE
Batch size for training. (default: 8)
--last_step LAST_STEP
Train up to this number of steps. (default: 1000000)
--lambda LMBDA Lambda for rate-distortion trade-off. (default: 1000)
--fl_alpha FL_ALPHA Class balancing weight for Focal Loss. (default: 0.75)
--fl_gamma FL_GAMMA Focusing weight for Focal Loss. (default: 2.0)
The loss function parameters can be selected depending on the desired rate-distortion trade-off, as well as the target PC block characteristics. Usage example:
python ADLPCC.py train "train_data_path/*.ply" "../models/3000/75" --lambda 3000 --fl_alpha 0.75
python ADLPCC.py compress INPUT_FILE CHECKPOINT_DIR [OPTIONS]
positional arguments:
input_file Input Point Cloud filename (.ply).
checkpoint_dir Directory where to load model checkpoints. For
compression, a glob pattern that expands into a list of
directories (each corresponding to a different trained
DL coding model) should be provided: ../models/*
optional arguments:
-h, --help show this help message and exit
--blk_size BLK_SIZE Size of the 3D coding block units. (default: 64)
--lambda LMBDA Lambda for RD trade-off when selecting best DL coding
model. (default: 0)
The desired number of DL coding models and the specific trained DL coding models available for selection can be specified by providing a directory path with a glob pattern (e.g., using wildcard characters such as * or ?).
Usage example:
python ADLPCC.py compress "test_data_path/longdress.ply" "../models/3000/*" --blk_size 64
python ADLPCC.py decompress INPUT_FILE CHECKPOINT_DIR [OPTIONS]
positional arguments:
input_file Input bitstream filename (.gz).
checkpoint_dir Directory where to load model checkpoints. For decompression,
a glob pattern that expands into a list of directories (each
corresponding to a different trained DL coding model) should
be provided: ../models/*
optional arguments:
-h, --help show this help message and exit
The desired number of DL coding models and the specific trained DL coding models available for selection can be specified by providing a directory path with a glob pattern (e.g., using wildcard characters such as * or ?).
Usage example:
python ADLPCC.py decompress "../results/3000/longdress/longdress.pkl.gz" "../models/3000/*"
The script rescaling.py can be used to re-scale a point cloud for a target bit depth precision.
python rescaling.py INPUT_FILE BITDEPTH
positional arguments:
input_file Input Point Cloud filename (.ply).
bitdepth Desired Point Cloud bit depth precision.
optional arguments:
-h, --help show this help message and exit
Usage example:
python rescaling.py Staue_Klimt_vox12.ply 9
The script estimateNormals.sh can be used to estimate the normal vectors of a point cloud, using quadric fitting with a neighborhood radius of 5. It requires CloudCompare 2.11.1 or a compatible version.
Usage example:
./estimateNormals.sh longdress.ply
A. F. R. Guarda, N. M. M. Rodrigues and F. Pereira, "Adaptive Deep Learning-based Point Cloud Geometry Coding," in IEEE Journal on Selected Topics in Signal Processing (J-STSP), vol. 15, no. 2, pp. 415–430, Feb. 2021. doi: 10.1109/JSTSP.2020.3047520.
@article{Guarda2021,
author = {A.F.R. Guarda; N.M.M. Rodrigues; F. Pereira},
title = {Adaptive Deep Learning-Based Point Cloud Geometry Coding},
journal = {IEEE Journal on Selected Topics in Signal Processing},
volume = {15},
issue = {2},
pages = {415-430},
month = {2},
year = {2021},
issn = {19410484},
doi = {10.1109/JSTSP.2020.3047520},
}