Are model-based tools still relevant for bioimage analysis?

New methods for background estimation and denoising fight back


Mauro Silberberg

Departamento de Física
Universidad de Buenos Aires
Argentina 🇦🇷 ⭐️⭐️⭐️
IFIBA-CONICET

Model-based vs AI

AI has taken the field by storm


Publishing model-based methods in:

  • 2017 ✅
  • Now: have you compared with AI?


It promises to learn the model from data.

Is it the end for model-based science?

Model-based vs AI

Model-based leverage prior knowledge of the measurement process.

Super-resolution achieved by modelling the illumination.

The diffraction limit applies when we ignore it.


Pragmatic advantages:

  • avoid the training process, which requires big datasets
  • are explainable, based on hypotheses to understand a priori their applicability
  • provide better generalization for out-of-distribution samples

Outline: two model-based methods

  1. SMO: a background estimation method
    • Short intro to background estimation
    • Comparison with standard methods
    • Tecnical part (transformation, thresholding, properties)
  2. binlets: a (multichannel) denoising method
    • Short intro to wavelets for denoising
    • Multi-channel and transformation-based thresholding
    • Extra: a (reviewer-required) comparison with Deep Learning

Estimating the background

in fluorescence microscopy image.

  • Calculate ratio:

    • ❌ \(\quad \text{cell}_i \,/\, \text{cell}_j\)
    • ✅ \(\quad (\text{cell}_i - \text{bg}) \,/\, (\text{cell}_j - \text{bg})\)

  • Simple segmentation: \[\text{cell} \leftarrow \text{intensity} > \text{background}\]

  • AI-based segmentation:

    • ❌ \(\quad \text{cell} \leftarrow \text{AI}(\text{intensity})\)
    • ✅ \(\quad \text{cell} \leftarrow \text{AI}(\text{intensity} - \text{bg})\)

Standard methods to estimate the background

Otsu

Splits intensity histogram to minimize intra-class variance

Variants: Li, Yen, isodata, etc.

They ignore spatial information.

Rolling ball

“Rolls a ball” below the intensity profile to obtain a “local minimum”.

Default method in ImageJ/FIJI

Process

> Subtract background

Standard methods to estimate the background

Manual selection

To be used as ground truth.

Comparison of the methods

SMO provides a fair sample of the background

not a segmentation.

Outline

  1. SMO: a background estimation method
    • Short intro to background estimation
    • Comparison with standard methods
    • Tecnical part (transformation, thresholding, properties)
  2. binlets: a (multichannel) denoising method
    • Short intro to wavelets for denoising
    • Multi-channel and transformation-based thresholding
    • Extra: a (reviewer-required) comparison with Deep Learning

SMO transformation

Hypothesis: background is flat compared to the noise

It distinguishes “flat” from “non-flat” regions

where flat is compared to the noise

Thresholding on a simulated “cell”

“Cell” \(\rightarrow\) a gaussian intensity profile.

Intensity thresholding recovers a biased distribution.

SMO thresholding recovers an unbiased distribution.

SMO properties: independence and non-parametricness

What happens when we apply it to “background” (flat-regions)?

  • Independence: we can sample any slice of SMO values
  • Non-parametric: we can select a threshold a priori

Non-uniform background

Does SMO still work here?

SMO generates an unbiased local sampling

Non-uniform background

What can we do with that?

Example: moving median on the SMO selection to estimate the background

Outline

  1. SMO: a background estimation method
    • Short intro to background estimation
    • Comparison with standard methods
    • Tecnical part (transformation, thresholding, properties)
  2. binlets: a (multichannel) denoising method
    • Short intro to wavelets for denoising
    • Multi-channel and transformation-based thresholding
    • Extra: a (reviewer-required) comparison with Deep Learning

Binlets

adaptive binning for multichannel signals

  • based on Haar wavelet \(\rightarrow\) adaptive binning or averaging
  • Hypothesis \(\rightarrow\) similarity in neighboring points
    • across channels:
      • RGB \(\rightarrow\) (\(R_i \approx R_j\) and \(G_i\approx G_j\) and …)
    • on a target transformation:
      • ratio \(r = \frac{ch_1}{ch_2}\) \(\rightarrow\) \(r_i \approx r_j\)
  • Requires \(\rightarrow\) test to compare neighboring points
    • Poisson noise \(\rightarrow\) \(\frac{x-y}{\sqrt{x + y}} < 1\) (Z-test)
    • phasor FLIM \(\rightarrow\) pawFLIM

Denoising with wavelets

  • Wavelet: Haar

  • Forward transform: \[ \begin{cases} a = (x + y) \,/\, 2 \\ d = (x - y) \,/\, 2 \end{cases} \]

  • Inverse transform: \[ \begin{cases} x = a + d \\ y = a - d \end{cases} \]

This process is recursively repeated with the \(a\) coefficients.

Thresholding as hypothesis testing

How to select a proper threshold?

Normalizations:

\[ \begin{cases} a = (x + y) \,/\, 2 \\ d = (x - y) \,/\, 2 \end{cases} \]

\[ \begin{cases} a = (x + y) \,/\, \sqrt{2} \\ d = (x - y) \,/\, \sqrt{2} \end{cases} \]

\[ \begin{cases} a = (x + y) \\ d = (x - y) \end{cases} \]

Useful when:

\[ x, y \sim N(\mu, \sigma) \]

\[ x, y \sim Poisson(\mu_{\{x,y\}}) \]

We obtain:

\[ \begin{cases} a \sim N(\mu, \sigma) \\ d \sim N(0, \sigma) \end{cases} \]

\[ \begin{cases} a \sim Poisson(\mu_x + \mu_y) \\ d \sim \quad ? \end{cases} \]

Threshold:

Global for all \(d\)

Define test for \(x=y\).

Single vs multichannel denoising

Example: two channel signal alternating between for values.

Multichannel information prevents wrongly averanging the values near 0.

Thresholding based on a target transformation

Example: ratio between two channels

Transformation information allows to average “different” values.

Based on a target transformation

Example: ratio between two channels

Binlets vs Deep Learning

  • Single-channel fluorescence image
  • Why? Reviewers…
  • Models: Noise2Noise and DnCNN
  • Dataset: 50 images per FOV

\(2-\sigma\) in bias in the nucleus or cytoplasm for Deep Learning

Summary

  • two model-based methods
    • no training required, explainable and generalizable
    • hypothesis testing based on prior knowledge

SMO

  • for background sampling
  • Hypothesis: (locally) flat background

binlets

  • for (multichannel) denoising
  • Hypothesis: smooth underlying signal
  • published as articles and Python libraries
  • SMO plugins for napari, ImageJ/FIJI and CellProfiler

This work was part of my PhD with Dr. Hernán E. Grecco