β¨ Method | π© Paper | π§ Install | π¨ Datasets | π» Training | π° Model Zoo | β‘ Inference| π Results
We provide a spatial redundancy denoising transformer (SRDTrans) to remove noise from fluorescence images in a self-supervised manner. First, a sampling strategy based on spatial redundancy is proposed to extract adjacent orthogonal training pairs, which eliminates the dependence on high imaging speed. SRDTrans is complementary to our previously proposed DeepCAD and DeepCAD-RT. Second, to break the performance bottleneck of convolutional neural networks (CNNs), we designed a lightweight spatiotemporal transformer architecture to capture long-range dependencies and high-resolution features at a low computational cost. SRDTrans can overcome the inherent spectral bias of CNNs and restore high-frequency information without producing over-smoothed structures and distorted fluorescence traces. Finally, we demonstrate the state-of-the-art denoising performance of SRDTrans on single-molecule localization microscopy (SMLM) and two-photon volumetric calcium imaging. SRDTrans does not contain any assumptions about the imaging process and the sample, thus can be easily extended to a wide range of imaging modalities and biological applications.
This repository is for SRDTrans introduced in the following paper:
- Python >= 3.6
- PyTorch >= 1.7
-
Clone the repository and delete placeholder files.
git clone https://github.com/cabooster/SRDTrans.git cd SRDTrans rm datasets/noisy/__init__.py pth/__init__.py results/__init__.py
-
Create a virtual environment and install PyTorch and other dependencies. In the 3rd step, please select the correct Pytorch version that matches your CUDA version from https://pytorch.org/get-started/previous-versions/.
$ conda create -n srdtrans python=3.6 $ conda activate srdtrans $ pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html $ pip install tifffile einops timm tqdm scikit-image
You can use your own data or download one of the demo data below (*.tif file). The raw low-SNR files should be put into ./datasets/noisy/
.
# Simulated SMLM & Simulated Calcium imaging data at 30hz
python train.py --datasets_folder noisy --n_epochs 20 --GPU 0,1 --train_datasets_size 6000 --patch_x 128 --patch_t 128
# Key parameters:
--datasets_folder: the folder containing your training data (one or more *.tif stacks)
--n_epochs: the number of training epochs
--GPU: specify the GPU(s) used for training. (e.g., '0', '0,1', '0,1,2')
--train_datasets_size: how many patches will be extracted for training
--patch_x, --patch_t: patch size in three dimensions (xy and t), should be divisible by 8.
-
In the vast majority of cases, good denoising models can be trained with these default parameters. If not necessary, you do not need to modify these parameters. You just need to change
--datasets_folder
or--GPU
. -
The folders containing training files are in
./datasets
. The checkpoint (model) of each epoch will be saved in./pth
. -
Hyperstacks with multiple channels and even one channel should be split and saved into single-channel stacks. Otherwise, the following error may occur (
ValueError: num_samples should be a positive integer value, but got num_samples=0
).
Models | Modality | Download |
---|---|---|
SRDTrans | Two-photon calcium imaging | Zenodo repository |
SRDTrans | Single-molecule localization microscopy (SMLM) | Zenodo repository |
Before inference, you should have trained your own model or downloaded our pre-trained model.
# Simulated calcium imaging data sampled at 0.3 Hz
python test.py --datasets_folder noisy --denoise_model cad_03hz --GPU 0,1 --patch_x 128 --patch_t 128
# Key parameters:
--datasets_folder: the folder containing the data to be processed (one or more *.tif stacks)
--denoise_model: the subfolder (under pth/) containing pre-trained models (e.g., ad_03hz).
--GPU: specify the GPU(s) used for inference. (e.g., '0', '0,1', '0,1,2')
--patch_x, --patch_t: patch size in three dimensions (xy and t), should be divisible by 8.
-
In the vast majority of cases, good denoising results can be obtained with these default parameters. If not necessary, you do not need to modify these parameters. You just need to change
--datasets_folder
,--denoise_model
or--GPU
. -
For testing, the patch size in t (
--patch_t
) should be consistent with that used for training. -
The denoising results will be saved in
./results
. If there are multiple models in--denoise_model
, only the last one will be used for denoising.
These are videos and you can click on the pictures to play them.
1. SRDTrans enhances large-scale volumetric calcium imaging in the mouse cortex.
2. The denoising performance of different methods on calcium imaging data sampled at 0.3 Hz.
3. Comparing the performance of DeepCAD and SRDTrans on fast-moving objects.