mrbles.pipeline module

Pipeline Classes and functions

This files contains the pipeline for the MRBLEs analysis.

class mrbles.pipeline.Analyze(dataframe, seq_list=None, images=None, masks=None)

Bases: mrbles.data.TableDataFrame

Analyze data MRBLE data and return per-code statistics.

Parameters:

• seq_list (list, Pandas DataFrame) – List (one column) or Pandas DataFrame with additional per-code information. In the case of a Pandas DataFrame all provided columns will be added, based on the row number, which corresponds to its code.
• flag_filt (boolean) – Sets if the flagged data is automatically filtered out. Defaults to True.

Variables:

functions (dict) –

Dictionary of functions and their corresponding names. Default: {‘mean’: np.mean,

’median’: np.median, ‘sd’: np.std, ‘se’: sp.stats.sem, ‘N’: len, ‘CV’: sp.stats.variation}

analyze(assay_channel, min_prob=None, bkg_data=None)

Calculate per-code statisics.

assay_channel : str

Assay channel name (column name) to calculate per-code statistics for.

min_prob: float

Filter by minimum probability interval, e.g. 0.95. Defaults to None.

bkg_data : float/int, string, list, NumPy array, Pandas DataFrame

If float/value is used, this value be subtracted. If string is used, column from internal dataframe is used. Otherwise, the data provided is subtracted, which needs to be the same size as the per-bead data. In the case of Pandas DataFrame, do slice the single exact column. Row number of the data is the code. Defaults to None.

data_per_bead

Return (normalized) data per bead.

data_per_code

Return (normalized) data per code.

mrble_report(assay_channel, filename, set_name=None, codes=None, files=None, sort=True, image_names=None, mask_names=None)

Generate per-MRBLE PDF image report.

Parameters:

• assay_channel (str) – Assay channel name, e.g. ‘Cy5_FF’
• filename (str) – Filename for generated PDF file.
• set_name (str) – Name of set, e.g. ‘Set A’
• codes (int, list of int) – Integer or list of integers with selected codes. Defaults to None.
• files (int, list of int) – Integer or list of integers with selected files. Defaults to None.
• sort (boolean) – Sort by code. Defaults to True.
• image_names (list of str) – List of image names. Defaults to [‘Dy’, ‘Sm’, ‘Tm’, ‘bkg’, ‘Eu’, assay_channel].
• mask_names (list of str) – List of mask names. Defaults to [‘mask_ring’, ‘mask_inside’, ‘mask_full’, ‘mask_bkg’].

norm_data

Return normalized data.

normalize(norm_data, scaled=True)

Normalize data.

Parameters:

• norm_data (Pandas DataFrame) – Dataframe with the per-bead normalization data. Single set only!
• scaled (boolean) – Scale maximum value normalization data to 1. Defaults to True.

qc_report(filename, savefig=False)

Generate Quality Control PDF report.

Parameters:

• filename (str) – Filename to save QC PDF report to.
• savefig (boolean) – Save figures separately to ‘report’ folder. Defaults to False.

class mrbles.pipeline.Decode(target, seq_list=None, decode_channels=None)

Bases: mrbles.data.TableDataFrame

Decode MRBLEs.

Parameters:

• target (list) – List of target ratios
• seq_list (Pandas DataFrame) – Columns with sequence information. Defaults to None.
• decode_channels (list) – Channel names. Defaults to None.

decode(data, combine_data=None)

Decode MRBLEs.

Parameters:

• data (Pandas DataFrame) – Data that contains decoding data.
• combine_data (Pandas DataFrame) – Data to be combined from previous pipeline steps. Defaults to None.

plot_clusters_2D(colors, ci_trace=None, min_prob=None)

Plot ratio clusters in 2D.

Parameters:

• colors (list) – List of color (NPL) names.
• ci_trace (float) – Set trace confidence line around clusters. Defaults to None.
• min_prob (float) – Set minimal probability level from 0 to 1, e.g. 0.95. Defaults to None.

plot_clusters_3D(min_prob=None)

Plot ratio clusters in 3D.

Parameters:min_prob (float) – Set minimal probability level from 0 to 1, e.g. 0.95. Defaults to None.

class mrbles.pipeline.Extract(function=None)

Bases: mrbles.data.TableDataFrame

Extract data from images using masks.

Parameters:function (function) – This is the function used to extract data from each bead region. This can be replaced with any 1 parameter function, e.g. np.mean. Defaults to np.median.

background_subtract(assay_channel, bkg_data)

filter(bkg_factor=2.0, ref_factor=2.0, bkg='bkg.mask_full', ref='Eu.mask_inside')

Filter and flag based on unmixed background and reference signal.

Parameters:

• bkg_factor (float) – Filter factor of Standard Deviations away from mean background signal. Defaults to 2.0
• ref_factor (float) – Filter factor of Standard Deviations within mean of reference signal. Defaults to 2.0
• bkg (string) – Background signal column name. Defaults to ‘bkg.mask_full’
• ref (string) – Reference signal column name. Defaults to ‘Eu.mask_inside’

get(images, masks, combine_data=None)

Extract data from images using masks.

Parameters:

• images (Xarray DataArray, mrbles ImageDataFrame) – Images of which to take values from using provided labeled masks. Must be same number of dimensions as masks.
• masks (Xarray DataArray, mrbles ImageDataFrame) – Labeled mask of regions to take values from the provided images. Must be same number of dimensions as images, even if one dimension is only singular. Therefore, use [] around mask selection, even when only using a single channel: [‘inside’].

Examples

>>> per_mrble_data = mrbles.Extract(np.median)
>>> per_mrble_data.run(images['100  nM', :, ['Cy5', 'Cy3']],
                       masks['100 nM', :, ['ring']])
>>> per_mrble_data.data

class mrbles.pipeline.Find(bead_size, pixel_size=None, border_clear=True, circle_size=None, min_r=None, max_r=None, annulus_width=None, **kwargs)

Bases: mrbles.data.ImageDataFrame

Find MRBLEs in brightfield images.

This class provides the method to find MRBLEs and segment regions for each MRBLE in brightfield images, provided by mrbles.Images.

Mask output: mask_full, mask_ring, mask_inside, mask_outside, mask_bkg, mask_check.

Examples

>>> find_mrbles = mrbles.Find(bead_size=18, pixel_size=3.5, border_clear=True, circle_size=350)
>>> find_mrbles.settings.parallelize = True
>>> find_mrbles.find(assay_images[:, : , 'Brightfield'])
    Bead diameter AVG: 54.36
    Bead diameter SD: 2.00
    Bead diameter CV: 3.68%
    Total number of beads: 2386
>>> find_mrbles
    {'Set 1': <xarray.DataArray (f: 12, c: 6, y: 1024, x: 1024)>
        array([[[[0, ..., 0],
                  ...,
                 [0, ..., 0]]]], dtype=uint16)
        Coordinates:
        * c        (c) <U12 'mask_full' 'mask_ring' 'mask_inside' 'mask_outside' ...
        Dimensions without coordinates: f, y, x,
    'Set 2': ...
>>> find_mrbles['Set 2', 3, 'mask_ring']
    <xarray.DataArray (y: 860, x: 860)>
    array([[0, 0, 0, ..., 0, 0, 0],
            ...,
           [0, 0, 0, ..., 0, 0, 0]], dtype=uint16)
    Coordinates:
        c        <U12 'mask_ring'
    Dimensions without coordinates: y, x

Super-Class wrapper for mrbles.core.FindBeadsImaging. Please see this class documentation for more information.

Parameters:

• bead_size (int) – Approximate width of beads (circles) in pixels.
• pixel_size (float) – Optional setting for image pixel size. Outputs dimensions using provided conversion value. Adds additional converted to database with affic ‘_conv’. Defaults to None.
• border_clear (boolean) – Beads touching border or ROI will be removed. Defaults to True.
• circle_size (int) – Set circle size for auto find circular ROI.
• min_r (int) – Minimum size of beads. This is set only when using Circle Finding method.
• max_r (int) – Maximum size of beads. This is set only when using Circle Finding method.
• annulus_width (int) – Annulus width size. This is set only when using Circle Finding method.

Variables:

• bead_dims (Pandas DataFrame) – Dataframe with individual MRBLE dimensions.
• beads_total (int) – Total number of MRBLEs found.
• beads_per_set (list of int) – Number of MRBLEs per set.
• settings (property) – Property provinding access to settings of mrbles.core.FindBeadsImaging. See property documentation for more information.

bead_dims

Return MRBLEs dimensions.

beads_per_set

Return total bead count in set(s).

beads_total

Return total bead count in set(s).

find(object_images, combine_data=None)

Execute finding images.

inspect(set_name=None, fig_num=3)

Display random images from set for inspection.

plot_size_dist()

Plot size distribution.

sets

Return list of sets.

settings

Return FindBeadsImaging object for settings purposes.

See FindBeadsImaging documentation for detailed settings.

Examples

>>> find_object = Find(bead_size=18)
>>> find_object.settings.area_min = 20
>>> find_object.settings.area_max = 350
>>> find_object.settings.eccen_max = 0.65
>>> find_object.settings.circle_size = 350

class mrbles.pipeline.Images(folders=None, file_patterns=None, data=None, channels=None)

Bases: mrbles.data.ImageDataFrame

Load images into mrbles dataframe.

There are two methods to load images into the mrbles dataframe: Method 1 - Provide dictionaries with folder and filename patterns. Method 2 - Provide a dictionary with numpy arrays.

This class facilitates loading images into the mrbles.data.ImageDataFrame. Please see this class documentation for more information.

For loading OME-TIFF image it uses mrbles.data.ImageSetRead. Please see this class documentation for more information.

>>> assay_folder = '../data'
>>> assay_folders = {'Set 1': '../data1',
                     'Set 2': '../data2'}
>>> assay_files = {'Set 1': 'image_set1_(0-9)(0-9).ome.tif',
                   'Set 2': 'image_set2_(0-9)(0-9).ome.tif'}
>>> assay_images = mrbles.Images(assay_folder, assay_files)
    Found 12 files in Set 1
    Found 11 files in Set 2

>>> image_arrays = {'Set 1': numpy_array_1,
                    'Set 2': numpy_array_2}
>>> channel_names = ['Brightfield', 'Cy5', 'l-435', 'l-546', 'l-620']
>>> assay_images = mrbles.Images(data=image_arrays, channels=channel_names)

>>> assay_images
    {'Set 1': <xarray.DataArray (f: 12, c: 5, y: 1024, x: 1024)>
        array([[[[169., ..., 166.],
                ...,
                [101., ..., 121.]]]])
        Coordinates:
        * c        (c) object 'Brightfield', 'Cy5', 'l-435', 'l-546', ...
        Dimensions without coordinates: f, y, x,
    'Set 2': ...
>>> assay_images['Set 2', 3, 'Brightfield']
    <xarray.DataArray (y: 1024, x: 1024)>
    array([[188., 163., 182., ..., 188., 174., 170.],
            ...,
           [144., 141., 142., ..., 162., 162., 154.]])
    Coordinates:
        c        <U11 'Brightfield'
    Dimensions without coordinates: y, x

Parameters:

• folders (str, dict) – String of single folder or dict of multiple folders. The folder(s) provided will searched recursively. Dict keys must match file_patterns dict.
• file_patterns (dict) – Dict of multiple file patterns. Dict keys must match folders keys, if multiple folders.
• data (NumPy array) – Alternative method to loading OME-Tif files. Load a dict of NumPy arrays with dimension order (f, c, y, x): files (f), channels (c), Y-dimension (y), X-dimension (x)
• channels (list of str) – List of channel names. Defaults to None. Channels will be numbers.

Variables:

• crop_x (slice) – Used for setting ROI slice in the X dimension.
• crop_y (slice) – Used for setting ROI slice in the Y dimension.

add_images(images)

Add images to dataframe.

flat_field(ff_image, channel, affix='_FF')

Flat-Field correction.

Parameters:

• ff_image (str, NumPy array) – Flat-field image file or NumPy array.
• channel (str) – Channel to apply flat-field correction.
• affix (str) – Affix for channel name to be appended. Defaults to ‘_FF’

load(series=0)

Load images in memory.

rename_channel(old_name, new_name)

Rename channel name.

Parameters:

• old_name (str) – Original name of channel.
• new_name (str) – New name for old_name channel.

static scan_path(path, pattern='*.tif')

Scan folder recursively for files matching the pattern.

Parameters:

• path (string) – Folder path as string, e.g. r’C:/folder/file.tif’.
• pattern (string) – File extenstion of general file pattern as search string, e.g. ‘peptides_x_([0-9][0-9])_MMStack_Pos0.ome.tif’ Defaults to ‘*.tif’.

class mrbles.pipeline.Ratio(reference_spectra, background='bkg')

Bases: mrbles.data.ImageDataFrame

Generate spectrally unmix ratio images.

Parameters:

• reference_spectra (list, ndarray, References object) – List, array, or References object of reference spectra for linear spectral unmixing.
• background (string) – Background key label. Defaults to ‘bkg’.

get(image_sets, reference, combine_data=None)

Calculate unmixed and ratio images.

Parameters:

• image_set (Xarray DataArray, mblres ImageDataFrame) – These are the images (emission channels) to be unmixed.
• reference (string) – Reference key label, e.g. ‘Eu’, used for ratio images.
• comb_data (Xarray DataArray, mblres ImageDataFrame) – These are the images that are combined with ‘image_set’. Must be same dimensions and type as ‘image_set’.

class mrbles.pipeline.References(folders, files, object_channel, reference_channels, bead_size=16, dark_noise=99)

Bases: mrbles.data.TableDataFrame

Create reference spectra.

There are three methods to load images into the this class: Method 1 - Provide a dictionaries with folders and filenames. Method 2 - Provide a dictionary with numpy arrays. Method 3 - Provide a file with spectra data.

Parameters:

• folders (dict) – Dictionary with keywords (e.g. ‘Dy’, ‘bkg’) and folders. Must correspond with files parameter.
• files (dict) – Dictionary with keywords (e.g. ‘Dy’, ‘bkg’) and filenames. Must correspond with folders parameter.
• object_channel (str) – Channel to be used for finding MRBLEs
• reference_channels (list) – List of channel names to be used for generating reference spectra.
• bead_size (int) – Bead diamater set in pixels. Defaults to 16.
• dark_noise (int) – Dark noise of the camera used for the images. Defaults to 99 (median dark noise of Andor Zyla 4.2 PLUS sCMOS).

Variables:

• background (str) – Name of the background spectrum. Defaults to ‘bkg’.
• crop_x (slice) – Crop X slice. Set with slice().
• crop_y (slice) – Crop Y slice. Set with slice().

static get_spectrum(dark_noise, channels, mask)

Get spectrum from image set using mask.

The median of the masked area is extracted, the camera dark noise subtracted, and normalized.

Parameters:

• dark_noise (int) – Intrinsic dark noise of camera. Image taken when shutter closed.
• channels (slice, list) – Slice of channels
• mask (NumPy array) – Labeled mask.

load()

Process all images and generate reference spectra.

plot(dpi=75)

Plot Reference spectra.

class mrbles.pipeline.Settings(objects, object_names)

Bases: future.types.newobject.newobject

Settings object.