Parallel faceted imaging in radio interferometry via proximal splitting

Parallel faceted imaging in radio interferometry via proximal splitting

Description

Faceted-HyperSARA is a fully documented MATLAB library for radio-interferometric wideband intensity imaging. The library offers a collection of utility functions and scripts from data extraction from an RI measurement set MS Table to the reconstruction of a wideband intensity image over the field of view and frequency range of interest. The library contains an implementation of algorithms from the "SARA" family (SARA, HyperSARA and Faceted HyperSARA), and more specifically described in the following publications.

P.-A. Thouvenin, A. Abdulaziz, A. Dabbech, A. Repetti, Y. Wiaux, Parallel faceted imaging in radio interferometry via proximal splitting (Faceted HyperSARA): I. Algorithm and simulations, submitted, preprint available online, March 2022.

P.-A. Thouvenin, A. Dabbech, M. Jiang, A. Abdulaziz, J.-P. Thiran, A. Jackson, Y. Wiaux, Parallel faceted imaging in radio interferometry via proximal splitting (Faceted HyperSARA): II. Code and real data proof of concept, submitted, April 2022.

The following features, crucial to achieve high precision imaging from large data volumes, are supported:

data dimensionality reduction via visibility gridding (Kartik2017);
estimation of the effective noise level when reliable noise estimates are not available, due to imperfect calibration.
correction of the w-term via w-projection (Dabbech2017);
incorporation of available compact Fourier models of the direction dependent effects (DDEs) in the measurement operator (Dabbech2021).

Dependencies

Faceted-HyperSARA relies on two auxiliary submodules (see installation section):

RI-measurement-operator for the formation of the radio-interferometric measurement operator;
SARA-dictionary for the implementation of the sparsity priors.

These modules contains codes associated with the following publications

A. Dabbech, L. Wolz, L. Pratley, J. D. McEwen and Y. Wiaux, The w-effect in interferometric imaging: from a fast sparse measurement operator to superresolution, Mon. Not. Roy. Astron. Soc., 471(4):4300-4313, 2017.

J. A. Fessler and B. P. Sutton, Nonuniform Fast Fourier Transforms Using Min-Max Interpolation, IEEE Trans. Image Process., vol. 51, n. 2, pp. 560--574, Feb. 2003.

A. Onose, A. Dabbech and Y. Wiaux, An accelerated splitting algorithm for radio-interferometric imaging: when natural and uniform weighting meet, Mon. Not. Roy. Astron. Soc., 469(1):938-949, 2017.

Z. Prusa, Segmentwise discrete wavelet transform, Brno university of technology, 2012.

Installation

To properly clone the project with the submodules, you may need to choose one of following set of instructions.

Cloning the project

If you plan to clone the project from https, you will first need to edit the .gitmodules file accordingly, replacing the git@github.com addresses with the https counterpart. That is

[submodule "lib/SARA-dictionary"]
	path = lib/SARA-dictionary
	url = https://github.com/basp-group/SARA-dictionary.git
[submodule "lib/RI-measurement-operator"]
	path = lib/RI-measurement-operator
	url = https://github.com/basp-group/RI-measurement-operator.git

Cloning the repository from scratch. If you used https, issue the following command

git clone --recurse-submodules https://github.com/basp-group/Faceted-HyperSARA.git

If you are using an SSH key for github rather than a personal token, then you will need to clone the repository as follows instead:

git clone --recurse-submodules git@github.com:basp-group/Faceted-HyperSARA.git

You will then also need to update the local repository configuration to use this approach for the sub-modules and update the submodules separately as detailed below. If you have problems updating the submodule access configuration open a issue on this repository.

Submodules update: updating from an existing Faceted-HyperSARA repository.

git pull
git submodule sync --recursive # update submodule address, in case the url has changed
git submodule update --init --recursive # update the content of the submodules
git submodule update --remote --merge # fetch and merge latest state of the submodule

Generating a synthetic dataset to test the library

To generate the synthetic data cubes used in the spatial and spectral experiments, you will need to download the S_DDE_MODEL.fits image file associated with the following paper.

A. Dabbech, A. Repetti, R. A. Perley, O. M. Smirnov, Y. Wiaux, Cygnus A jointly calibrated and imaged via non-convex optimization from VLA data, Mon. Not. Roy. Astron. Soc., 506(4):4855-4876, Oct. 2021.

To do so, issue the following commands in a terminal.

# if on MAC: 
# brew install wget
cd path/to/Faceted-HyperSARA
cd data
wget -P . https://researchportal.hw.ac.uk/files/43645966/S_DDE_MODEL.fits

Please refer to the online documentation to configure and run the imaging script (see documentation section if you prefer building the documentation locally).

Documentation

Full library documentation is accessible online.

To build the documentation on your machine, make sure the following Python packages have been installed, and issue the appropriate build command.

# setup conda environment to build the documentation
conda env create --name fhs-doc --file environment.yml

## or using conda/pip
# conda create -n fhs-doc
# conda activate fhs-doc
# conda install pip
# pip install -r requirement.txt

# building the documentation in html format
cd docs
make html

All the generated .html files are contained in the docs/build folder.

If needed, you can delete the conda environment as follows

conda env remove -n fhs-doc

Note to contributors

Pushing the documentation online

Add a worktree from the master branch to be able to push the documentation, once built locally.

# make sure the folder html does not exist before running the command
git worktree add docs/build/html gh-pages
cd docs/build/html
git add .
git commit -m "Build documentation as of $(git log '--format=format:%H' master -1)"
git push origin gh-pages
# delete the worktree
cd ../
git worktree remove html

Pull request: Matlab code formatting

Make sure codes in any pull request have been properly formatted with the miss_hit package using the miss_hit.cfg file provided

# activate fhs-doc environment (see previous paragraph)
conda activate fhs-doc
# run the following command from the root of the package (where the miss_hit.cfg file is)
mh_style --fix .

basp-group / Faceted-HyperSARA