What exactly is deconvolution doing?

[Jiayun]

TL;DR Summary

Is deconvolution removing the effect of illumination or detection PSF or both?

The widely recognized theory for the detected image in microscope (I believe) is calculated by taking the convolution of the object and a point spread function (PSF). Deconvolution tries to reverse this operation to get back the original object. This explanation sounds easy and intuitive but after some digging, I found that I don't understand what deconvolution is doing anymore.

So this PSF is actually made up of two components: detection (PSF_det) and illumination PSF (PSF_ill). If we talk about widefield microscopy, illumination is constant so PSF_ill=1 which means the PSF in the original theory is just PSF_det. This is still ok but a quick question will be why doesn't this operation return the image of the object (image with infinite resolution)?

If we talk about confocal microscopy, PSF_ill is not 1 and we can basically approximate to PSF_det giving PSF=PSF_det². So we can see that the convolution with this PSF actually gives higher resolution. My main question with this is what happens if I deconvolve this image? Am I removing the effect of PSF_det? If so, do I end up with PSF=PSF_det (without the square) which has lower resolution then I started with?

 

[Ibix]

I was more into astronomical instrumentation, so I'm not sure about the microscopy aspects of your question, but the first part is easy enough. The image you see is being modelled as the light from the object convolved with the PSF of your instrument. If you Fourier transform that it can be written as the FT of the object multiplied by the FT of the PSF. So you Fourier transform the light you see, divide by the Fourier transform of the PSF, and inverse FT and you should get the object.

The problem is in the division. When the PSF is zero, dividing by it is undefined - and even for an ideal optical system the PSF is the Airy disc, $J_0(r)$, which has many zeroes. If your Fourier transformed object has any power at all where the zeroes fall then you cannot perform the inversion because some information has been lost. Furthermore, if you have any noise in your input that falls near a zero of the PSF (stray photons in your detector can cause this, for example) then that can be heavily magnified by being divided by a very small number and have a large effect on your input.

 

[Jiayun]

I see, so basically we can't get infinite solution even theoretically due to the zeros in the Airy disk? Does this mean theoretically deconvolution doesn't help diffraction limited system since they already reach the highest spatial frequency allowed by the first Airy disk? Or deconvolution can help by increasing the contrast of the higher frequency components since the Airy disk is tapering these components.

 

[Ibix]

I think I said something unclear above due to early morning confusion on my part. Apologies.

Remember that you aren't dividing by the Airy disc, you're dividing by its Fourier transform. The Fourier transform of an Airy disc is just a disc: 1 for all $\omega_r<\omega_\mathrm{max}$ and 0 otherwise. So it isn't where the zeroes of the Airy disc are that matters, but where that cutoff frequency is (obviously there's a relationship between the size of the disc in Fourier space and the size of the Airy disc, but it's the Fourier space size that matters firectly). So deconvolution doesn't help you here - where the division works, you're dividing by 1, and everywhere else (those higher frequency components above the cutoff) the information is gone.

Deconvolution helps when the Fourier transform of the PSF has a gentler shape than the 1-or-0 hard edge of the diffraction limited case. This is for exactly the reason you say, amplifying the frequency components that were suppressed by the tail of the Fourier transformed PSF. You can still end up amplifying noise, though, so you need to take care where the Fourier transformed PSF is very low compared to its peak value.

 

[Jiayun]

Oh yes I was thinking of Gaussian when actually it is supposed to be Airy disk I am guessing PSF in practical situations are not Airy disks since deconvolution is shown to improve resolutions? So basically deconvolution increases resolution by amplifying high frequency components?

 

[Ibix]

So basically deconvolution increases resolution by amplifying high frequency components?

Or whatever components it is that are missing, yes.

 

[Jiayun]

I see. This makes sense. Thank you very much

 

[Gleb1964]

The Airy disc is described by Bessel function, which Fourier transform has limited cut-off frequency $\lambda / D$. All information above cut-off frequency is not passing through the optical system.
In confocal microscopy one need to scan object field with illumination spot or pattern.

 

[Jiayun]

Yeah I am aware of the single spot illumination in confocal microscopy. I am wondering whether deconvolution increases the resolution for confocal images.

 

[Gleb1964]

Deconvolution cannot overcome the diffraction limit, but it can bust the contact transfere up to diffraction cut-off frequency. Optical system can be characterized by a Modulation Transfer Function (MTF), which is 1 at zero frequency and going down to 0 at the maximum cut off frequency which means increasing contrast lost at higher spatial frequency. If you have a good signal to noise relation, you can gain contrast back and even bust it above initial level by doing agressive deconvolution.
Some algorithms may claim to go above the diffraction limit, but that ether "guess", ether algorithm takes some additional assumption (for example, that object consists from only point sources).
In confocal illumination the diffraction limit is higher because coherent illumination squares the amplitude of the point spread function making it more sharp.

 

[HAYAO]

TL;DR Summary: Is deconvolution removing the effect of illumination or detection PSF or both?

The widely recognized theory for the detected image in microscope (I believe) is calculated by taking the convolution of the object and a point spread function (PSF). Deconvolution tries to reverse this operation to get back the original object. This explanation sounds easy and intuitive but after some digging, I found that I don't understand what deconvolution is doing anymore.

So this PSF is actually made up of two components: detection (PSF_det) and illumination PSF (PSF_ill). If we talk about widefield microscopy, illumination is constant so PSF_ill=1 which means the PSF in the original theory is just PSF_det. This is still ok but a quick question will be why doesn't this operation return the image of the object (image with infinite resolution)?

If we talk about confocal microscopy, PSF_ill is not 1 and we can basically approximate to PSF_det giving PSF=PSF_det². So we can see that the convolution with this PSF actually gives higher resolution. My main question with this is what happens if I deconvolve this image? Am I removing the effect of PSF_det? If so, do I end up with PSF=PSF_det (without the square) which has lower resolution then I started with?

I work with both wide-field and confocal fluorescence microscopes.

This depends on the context.

If you're using wide-field, when people say "deconvolution" it usually refers to post-processing of the 3D image stack using PSF_det that is either theoretical (Airy disk for uniformly emitting point source, or dipole-emitter PSF for single molecules) or experimental (single emitter measurement that is done prior to the sample imaging), so that the final images are not blurred due to emission occurring away from the focal plane of each z-distance acquisition.

To answer your question, infinite resolution is impossible. The fact that your detector (which most likely is a CCD or CMOS camera) has finite resolution means that there will be errors associated with deconvoluting finite resolution images. However, the higher the magnification of the objective, and the smaller each of the capacitors are (translates to pixles), you will get fewer errors.

If you're using confocal microscopy, when people say "deconvolution" it usually refers to post-processing of the 3D image stack using PSF_ill. In many cases, emission from out-of-plane emitters is minimized because of the pinhole in the detector side, so you wouldn't need to worry too much about the PSF_det. Nonetheless, you can still do a second deconvolution with PSF_det if you think it is necessary. PSF_ill is not PSF_det (the PSF of the single emitter is not the same as the PSF of the excitation source). Also, you cannot do PSF=PSF_det². You have to deconvolute subsequently.

 

[Gleb1964]

To answer your question, infinite resolution is impossible. The fact that your detector (which most likely is a CCD or CMOS camera) has finite resolution means that there will be errors associated with deconvoluting finite resolution images. However, the higher the magnification of the objective, and the smaller each of the capacitors are (translates to pixles), you will get fewer errors.

So, what would be your conclusion about resolution limit, under assumption that sampling frequency can be optimazed whatever to get the best outcome?
My answer is simple, resolution cannot overcome the diffraction limit, deconvolution does not help.
Remark: Resolution is defined as possibility to resolve sinusoidal mira with contrast 1.0

 

[HAYAO]

So, what would be your conclusion about resolution limit, under assumption that sampling frequency can be optimazed whatever to get the best outcome?

Resolution cannot be infinitely good, as it is bound by the inherent error of the mathematical operation when deconvoluting a finite element data. That's my conclusion.

My answer is simple, resolution cannot overcome the diffraction limit, deconvolution does not help.

In conventional fluorescence microscopy, no. You're right. But there are techniques to overcome the diffraction limit. Google Super-resolution microscopy.

 

[Gleb1964]

.. But there are techniques to overcome the diffraction limit. Google Super-resolution microscopy.

Yes, that what I do periodicaly for my long professional life, but new methods coming are still surprizing me. Super-resolution microscopy works, indeed, over difraction limit without breaking the difraction limitation.
Looking on a number of techiches of sample manipulation, one can mention, that super-resolution is achived by bringing additional assumption, for example, that the object consist only of one or a very limited amount of point sources, which makes possible to apply resonable deconvolution bejond the diffraction limit. That wound not be possible to produce on coventional type of image where all points of continuous image are exposed simultaneously.