pybbbc
This is a python interface to the BBBC021 dataset of cellular images (Caie et al., Molecular Cancer Therapeutics, 2010), available from the Broad Bioimage Benchmark Collection (Ljosa et al., Nature Methods, 2012).
The dataset is made of 13,200 fields of view imaged in three channels of resolution 1024x1280 and the corresponding metadata: site, well, replicate, plate, compound, concentration and mechanism of action (MoA).
The images are of MCF-7 breast cancer cells treated for 24 h with a collection of 113 small molecules at eight concentrations. The cells were fixed, labeled for DNA, F-actin, and Β-tubulin, and imaged by fluorescent microscopy. The complete description of the dataset can be found here.
Installation
With pip
pip install git+https://github.com/giacomodeodato/pybbbc.git
For development (with conda)
git clone git+https://github.com/giacomodeodato/pybbbc.git
cd pybbbc
conda env update
conda activate pybbbc
pre-commit install
Usage
An instance of the dataset can be easily created with the following code:
from pybbbc import BBBC021
bbbc021 = BBBC021()
Indexing
Standard indexing (i.e. bbbc021[3]
or bbbc021[2:5]
) can be used to access samples and subsets. A single sample is a tuple (image, metadata)
where the metadata is a tuple itself of plate metadata and compound metadata using the following structure:
metadata = (
( # plate metadata
site,
well,
replicate,
plate
),
( # compound metadata
compound,
concentration,
moa
)
)
Filtering
The instance of a dataset can be intuitively filtered during the initiation using metadata keyword arguments as follows:
# get the samples without MoA
bbbc021 = BBBC021(moa='null')
or using a list of metadata values:
# get the samples with the colchicine and latrunculin B compounds
bbbc021 = BBBC021(compound=['colchicine', 'latrunculin B'])
# get the samples with only a selection of Mechanisms of Action
bbbc021 = BBBC021(moa=['Actin disruptors', 'DMSO', 'Microtubule destabilizers'])
The dataset can be filtered using any metadata keyword: site
, well
, replicate
, plate
, compound
, concentration
and moa
. The unique values of the most relevant keywords are stored as static attributes: BBBC021.PLATES
, BBBC021.COMPOUNDS
and BBBC021.MOA
. Other useful static attributes are BBBC021.IMG_SHAPE
, BBBC021.CHANNELS
and BBBC021.N_SITES
.
Sub-datasets
The individual sub-datasets and their corresponding samples can be accessed as well after initiation:
bbbc021 = BBBC021()
img = bbbc021.images[0]
# bbbc021.wells
# bbbc021.sites
# bbbc021.replicates
# bbbc021.plates
# bbbc021.compounds
# bbbc021.concentrations
# bbbc021.moa
Data download
The raw data can be downloaded after importing the BBBC021 dataset as follows:
from pybbbc import BBBC021
BBBC021.download()
This will automatically create the data folder in ~/.cache/
(by default) and download images and metadata (this may take several hours). Alternatively, it is possible to specify the data_path
parameter to set the destination folder.
Dataset creation
After downloading the raw data, the preprocessed dataset can be created by calling the static method make_dataset()
.
from pybbbc import BBBC021
# BBBC021.download()
BBBC021.make_dataset()
This function will execute the following steps:
- Merge the metadata regarding the compound and the Mechanism of Action (MoA)
- Compute and apply the illumination correction function to the images
- Scale pixel intensities in the [0, 1] range discarding the top and bottom 0.1 percentiles
- Create a virtual HDFS dataset (using h5py) associating the images to the corresponding metadata
Illumination correction
In order to compute the illumination correction function, the average of all the images taken at the same site and belonging to the same plate is calculated. This image is then smoothed using a Gaussian filter with sigma=500. A robust minimum is calculated as the 0.02 percentile of all the positive values of the average image, and all the pixel with smaller values are truncated. This image is finally divided by the robust minimum in order to have only values greater or equal to one so that the illumination correction can be applied by dividing the original image by the illumination function.
def correct_illumination(images, sigma=500, min_percentile=0.02):
# calculate average of images belonging to the same site and plate
img_avg = images.mean(axis=0)
# apply Gaussian filter
img_mask = gaussian_filter(img_avg, sigma=sigma)
# calculate robust minimum
robust_min = np.percentile(img_mask[img_mask > 0], min_percentile)
# clip and scale pixel intensities
img_mask[img_mask < robust_min] = robust_min
img_mask = img_mask / robust_min
# return corrected images
return images / img_mask