polimi-ispl / deep_learning_soundfield_synthesis_irregular_array

Soundfield synthesis using Irregular Loudspeaker Arrays and Convolutional Neural Networks

#Compensation of Driving Signals for Soundfield Synthesis through Irregular Loudspeaker Arrays Based on Convolutional Neural Network

Code repository for the paper Synthesis of Soundfields through Irregular Loudspeaker Arrays Based on Convolutional Neural Networks [1].

• Dependencies
• Data Generation
• Network Training
• Results Computation

Dependencies

• Python, it has been tested with version 3.6.9
• Numpy, scikit-image, scikit-learn, tqdm, matplotlib
• Tensorflow 2.+
• sfs
• cvnn

Data generation

There are three different scripts to generate the data for the circular, linear array and real measurements scenarios, namely generate_data_circular_array.py, generate_data_linear_array.py and generate_data_real_array.py. The parameters used for data generation (e.g. sources position, array position) are defined into data_lib/params_circular, data_lib/params_linear and data_lib/params_real, respectively.

The command-line arguments are the following

• gt_soundfield: bool, True if want to generate also data related to ground truth soundfield
• n_missing: Int, number of missing loudspeaker from full regular setup
• dataset_path: String, folder where to store dataset

Network training

There are three different scripts to generate the data for the circular, linear array and real measurements scenarios, namely train_circular_array.py, train_linear_array.py and train_real_array.py. The parameters used for data generation (e.g. sources position, array position) are defined into data_lib/params_circular, data_lib/params_linear and data_lib/params_real, respectively.

• epochs: Int, number of epochs
• batch_size: Int, dimension of batches used for training
• log_dir: String, folder where store training logs accessible via Tensorboard
• gt_soundfield_dataset_path: String, path to numpy array where ground truth soundfield data are contained
• learning_rate: Float, learning rate used for the optimizer
• green_function: String, path to numpy array where green function between secondary sources and evaluation points is contained

Results computation

There are three different scripts to compute the results included in the paper for the circular, linear array and real measurements scenarios, namely generate_test_data_circular_array.py, generate_test_data_linear_array.py and generate_test_data_real_array.py. The parameters used for data generation (e.g. sources position, array position) are defined into data_lib/params_circular, data_lib/params_linear and data_lib/params_real, respectively.

• dataset_path: String, folder where data are saved
• models_path: String, folder where network model are saved
• • n_missing: Int, Number of missing loudspeakers
• gt_soundfield: Bool, compute and save ground-truth soundfield
• pwd: Bool, compute and save results using MR method
• pwd_cnn: Bool, compute and save results using CNN method
• wfs: Bool,compute and save results using WFS method
• awfs: Bool, compute and save results using AWFS method
• pwd_apwd: Bool,compute and save results using AMR method

To plot both Normalized Reproduction Error (NRE) and Structural Similarity Index (SSIM) run the code contained into plot_results.py with the following arguments

• dataset_path: String, path to folder where to save results stored into arrays.
• array_type: String, type of array, i.e. linear or circular
• n_missing: Int, number of missing loudspeaker from full regular setup

N.B. pre-trained models used to compute the results shown in [1] can be found in folder models

References

[1] L.Comanducci, F.Antonacci, A.Sarti, Synthesis of Soundfields through Irregular Loudspeaker Arrays Based on Convolutional Neural Networks [arXiv preprint].