Light Microscopy: Structured Illumination Microscopy Basics & Math PDF

Light microscopy #14a structured illumination microscopy Basics & math 1 Marcel Müller, Wolfgang Hübner Image-scanning microscopes “ f...

Light microscopy #14a structured illumination microscopy Basics & math 1 Marcel Müller, Wolfgang Hübner Image-scanning microscopes “ f ”. focussed onto a spot, scanned over the sample. Using a pixelated detector instead of a pinhole, this instrument can retrieve information beyond the Abbe limit. 𝐷 𝑟Ԧ = 𝑆 𝑟Ԧ ⋅ 𝐼 𝑟Ԧ ∗ ℎ෨ 𝑟Ԧ Clearly, 𝐼 𝑟Ԧ ≠ const for a confocal system, and the illumination is structured. However, and for mostly historic reasons, we do not tend to call confocal “ d ” Structured illumination illustrated Simulation of resolution limit. Fluorophores on the left indistinguishable in wide-field. Now, apply a fine structure to the illumination. More precisely, a sinusoidal illumination distribution, with a spacing close to the diffraction limit. A static image does not reveal more (but at least different) information. However, shifting the phase of the pattern reveals extra information.  Some details on terminology Gustaffson-(Heintzmann)-SIM: Use sinusoidal modulation of illumination light Allows for „direct“ reconstruction Number of phases set by sinusoids (2xN + 1), allows to disentangle effects of illumination from sample structure Based on direct decomposition in Fourier space (sinusoids become delta peaks, with allows to explicitly invert the sample/illumination convolution) Other approaches (using deconvolution-like algorithms) exist, often combined with more complex illumination …w ’ f w w w k ISIM, MSIM, etc.: A “ ” , w I ? T w “I ” f w SIM math 5  Reminder: OTFs are PSFs in Fourier space 2 2 𝑘 𝑘 𝑘 ℎ 𝑘 = arccos − 1− 𝜋 𝑘𝑚𝑎𝑥 𝑘𝑚𝑎𝑥 𝑘𝑚𝑎𝑥 𝜆 With Abbe: = 𝑘𝑚𝑎𝑥 2 𝑁𝐴 Keep in mind: ℎ෨ 𝑥 = 𝐹𝑇 ℎ 𝑘 It is the same function! (in a different basis) 6 1. Camera image D of sample S under constant Microscope Sample illumination I (widefield), blurred by PSF ℎ෨ 𝐷 𝑟Ԧ = 𝑆 𝑟Ԧ ⋅ 𝐼 𝑟Ԧ ∗ ℎ෨ 𝑟Ԧ 𝐼 𝑟Ԧ = I0 → const. 2. Microscopy in Fourier space: fold with PSF → multiplication with OTF ෩ 𝑘 = 𝑆ሚ 𝑘 ∗ 𝐼ሚ 𝑘 𝐷 ⋅ℎ 𝑘 OTF shows two effects Band-limit, i.e. higher frequencies do not transmit ෩ 𝑘 = 𝑆ሚ 𝑘 ∗ 𝛿 𝑘 𝐼0 ⋅ ℎ 𝑘 𝐷 d f q “d d” → that is w w “ d” What this omits (at least, besides all practicalities) ෩ 𝑘 = 𝐼0 ⋅ 𝑆ሚ 𝑘 ⋅ ℎ(𝑘) 𝐷 Area integration of camera pixels Light quantisation (in SIM: every time someone says “filtering”) → Shot noise  Visualisation widefield 𝐷 𝑟Ԧ ෩ 𝑘 𝐷 𝑘𝑦 𝑘𝑥 U2OS cells, stained for Actin, illuminated at 488nm Power spectrum 8 (pseudo)-widefield data  Structured illumination 1. Camera image D of sample S, 2. Fourier space but now under sinusoidal SIM illumination 𝐼 𝑟Ԧ , blurred by PSF ℎ෨ 𝑀 𝐷𝑞,𝑛 𝑟Ԧ = 𝑆 𝑟Ԧ ⋅ I0 ෍ 𝑎𝑞,𝑚 cos 2 𝜋 𝑚 (𝑟Ԧ 𝑝Ԧ𝑞 + 𝜙𝑛 ) ∗ ℎ෨ 𝑟Ԧ 𝐷𝑞,𝑛 𝑟Ԧ = 𝑆 𝑟Ԧ ⋅ 𝐼𝑞,𝑛 𝑟Ԧ ∗ ℎ෨ 𝑟Ԧ 𝑚=0 𝑀 𝑀 𝐼𝑞,𝑛 𝑟Ԧ = I0 ෍ 𝑎𝑞,𝑚 cos 2 𝜋 𝑚 (𝑟Ԧ 𝑝Ԧ𝑞 + 𝜙𝑛 ) ෩𝑞,𝑛 𝑘 = I0 ෍ 𝑎𝑞,𝑚 𝑒 ±𝑖𝑚𝜙𝑛 𝛿 𝑘 ∓ 𝑚 ⋅ 𝑝Ԧ𝑞 ∗ 𝑆ሚ 𝑘 𝐷 ⋅ ℎ(𝑘) 𝑚=0 𝑚=0 𝑝Ԧ𝑞 Wave vector, spacing and rotation of your SIM pattern 𝑞: Pattern orientation 3. Convolution with delta peak → shift of information 𝜙𝑛 Phase shift of the SIM pattern 𝑀 𝑚 Number of harmonics ෩𝑞,𝑛 𝑘 = I0 ෍ 𝑎𝑞,𝑚 𝑒 ±𝑖𝑚𝜙𝑛 𝑆ሚ 𝑘 ∓ 𝑚𝑝Ԧ𝑞 𝐷 ⋅ ℎ(𝑘) 𝑎𝑞,𝑚 Modulation strength of a certain harmonic 𝑚=0 Side note: 𝑚 0. T “ ” d f fact that light intensities physically cannot be negative.  Raw data SIM w/o harmonics ෩𝑞,𝑛 𝑘 𝐷 𝐷𝑞,𝑛 𝑟Ԧ , for q=1, n=1..3 (animation), and M=1 𝑘𝑦 𝑝Ԧ1 𝑘𝑥 MitoTracker red (568nm excitation) in U2OS, Power spectrum 10 using a sinusoidal intensity pattern  SIM pattern visibility in raw data The apparent low contrast of the SIM pattern is an effect of this observation through a transfer function that has low contrast at the spatial frequency of the pattern Apparent(!) contrast of SIM pattern Spatial frequency of SIM pattern 11  Raw data SIM w/o harmonics 𝐷𝑞,𝑛 𝑟Ԧ , for q=1, n=1..3 (animation), and M=1 ෩𝑞,𝑛 𝑘 𝐷 𝑘𝑦 𝑝Ԧ1 𝑘𝑥 Fluorescent beads, illuminated at 647nm, Power spectrum 12 using a sinusoidal intensity pattern SIM with 1st harmonic 𝑀 𝐼𝑞,𝑛 𝑟Ԧ = I0 ෍ 𝑎𝑞,𝑚 cos 2 𝜋 𝑚 (𝑟Ԧ 𝑝Ԧ𝑞 + 𝜙𝑛 ) 𝑚=0 M=2 here means there is a base frequency and a 1st harmonic in the illumination. 𝑝Ԧ𝑞 now is typically a lower base frequency. Illustration: 400nm base frequency, 1st harmonic at 200nm, and thus close to the resolution limit. 13  Raw data SIM with 1st harmonic 𝐷𝑞,𝑛 𝑟Ԧ , for q=1, n=1..3 (animation), and M=2 𝑘𝑦 2 ⋅ 𝑝Ԧ1 𝑝Ԧ1 𝑘𝑥 U2OS cells, stained for Actin, illuminated at 488nm Power spectrum 14 using sinusoidal illumination intensities with 1. harmonic  SIM pattern visibility in raw data In 3-beam SIM, using a base Contrast of frequency and a 1st SIM pattern harmonic, the base frequency is well visible, as the OTF transmits it with high contrast Contrast of 1st harmonic 1st harmonic of SIM pattern SIM pattern 15  Reconstructing SIM data: band separation 4. Construct a linear equation system. Vectors: Band-separation matrix: 𝑀 ෩𝑞,𝑛 𝑘 = I0 ෍ 𝑎𝑞,𝑚 𝑒 ±𝑖𝑚𝜙𝑛 𝑆ሚ 𝑘 ∓ 𝑚𝑝Ԧ𝑞 𝐷 ⋅ ℎ(𝑘) ഥ → 𝐸ത𝑚𝑛 = 𝑎𝑞,𝑚 𝑒 𝑖𝑚𝜙𝑛 ⋅ 𝑬 𝑚=0 Equation on the left becomes: 𝑁 𝑵 𝑴 ෩′ → 𝑇෨𝑚 𝑘 = 𝑆ሚ 𝑘 − 𝑚𝑝Ԧ𝑞 ℎ 𝑘 𝐼0 𝑻 ෩𝑞,𝑛 (𝑘) = ෍ ෍ 𝐸ത𝑚𝑛 𝑇෨𝑚′ (𝒌) ෍𝐷 𝑛=1 𝒏=𝟏 𝒎=−𝑴 Those entries are the so- d “bands” with entries m = -M.. -0,+0.. M Also absorb contributions of ℎ 𝑘 and 𝐼0 ෩ ′𝒒 (𝒌) = 𝑬 𝑫 ഥ𝑻෩′(𝒌) Multiple Measurements ෩ ′𝒒 → 𝐷 𝑫 ෩𝑞,𝑛 (𝑘) with n varying phases 𝜙𝑛  Reconstructing SIM data: band separation 5. Solve the equation system: “Band separation” right-hand side provides information beyond the ෩′ 𝑫 𝒒 ഥ𝑻 𝑘 =𝑬 ෩𝒒 ′ 𝒌 diffraction limit, along 𝑝Ԧ1 ෩𝒒 ′ 𝒌 = 𝑬 ഥ −𝟏 𝑫 ෩ ′𝒒 𝑘 ‘q’ f 2 𝑻 Can be extended to higher harmonics M>1, by :W ’ f d f : providing 2 extra phase shifts per harmonic ෩′ → 𝑇෨𝑚 (𝑘) = 𝑆ሚ 𝑘 − 𝑚𝑝Ԧ𝑞 ℎ 𝑘 𝐼0 𝑻 This is the first step in the SIM reconstruction N :W d ’ d k w 𝑝Ԧ𝑛 at this point Less obvious: We can correct for a global Example for M=1, N=3 (3-phases 2-beam SIM) phase shift (𝜙𝑛 = 𝜙global + 𝜙𝑛′ ) later → Here we only need to know relative phases −1 2 𝑎𝑞,0 𝑎𝑞,1 𝑒 𝑖𝜙1 𝑎𝑞,2 𝑒 −𝑖𝜙1 ෩𝑞,𝜙 𝐷 1 𝑆ሚ𝑞 (𝑘)ℎ 𝑘 𝐼0 Why care? This is what makes parameter 2 𝑎𝑞,0 𝑎𝑞,1 𝑒 𝑖𝜙2 𝑎𝑞,2 𝑒 −𝑖𝜙2 ෩𝑞,𝜙 𝐷 = 𝑆ሚ𝑞 (𝑘 − 𝑝Ԧ1 )ℎ 𝑘 𝐼0 extraction work 2 2 𝑎𝑞,0 𝑎𝑞,1 𝑒 𝑖𝜙3 𝑎𝑞,2 𝑒 −𝑖𝜙3 ෩𝑞,𝜙 𝐷 𝑆ሚ𝑞 (𝑘 + 𝑝Ԧ1 )ℎ 𝑘 𝐼0 3  Separated bands 𝑆ሚ1,0 𝑘 𝐼0 ~ 𝑆1,0 (𝑟) Ԧ 𝐼0 We can obtain 𝑆ሚ𝑞,0 𝑘 ℎ 𝑘 𝐼0 𝑆ሚ𝑞,1 𝑘 ∓ 𝑝Ԧ𝑞 ℎ 𝑘 𝐼0 𝑆ሚ𝑞,2 𝑘 ∓ 2 ⋅ 𝑝Ԧ𝑞 ℎ 𝑘 𝐼0 for each pattern orientation q Note that ℎ 𝑘 has (approximately) been divided out “f d” data 18  Separated bands 𝑆ሚ1,1 𝑘 𝐼0 ~ 𝑆1,1 (𝑟) Ԧ 𝐼0 We can obtain 𝑆ሚ𝑞,0 𝑘 ℎ 𝑘 𝐼0 𝑆ሚ𝑞,1 𝑘 ∓ 𝑝Ԧ𝑞 ℎ 𝑘 𝐼0 𝑆ሚ𝑞,2 𝑘 ∓ 2 ⋅ 𝑝Ԧ𝑞 ℎ 𝑘 𝐼0 for each pattern orientation q Note that ℎ 𝑘 has (approximately) been divided out “f d” data Note that the shift 𝑝Ԧ𝑞 has been compensated for. 19  Separated bands 𝑆ሚ1,2 𝑘 𝐼0 ~ 𝑆1,2 (𝑟) Ԧ 𝐼0 We can obtain 𝑆ሚ𝑞,0 𝑘 ℎ 𝑘 𝐼0 𝑆ሚ𝑞,1 𝑘 ∓ 𝑝Ԧ𝑞 ℎ 𝑘 𝐼0 𝑆ሚ𝑞,2 𝑘 ∓ 2 ⋅ 𝑝Ԧ𝑞 ℎ 𝑘 𝐼0 for each pattern orientation q Note that ℎ 𝑘 has (approximately) been divided out “f d” data Note that the shift 𝑝Ԧ𝑞 has been compensated for. With less information at 𝑘 = 0, smooth structured vanish 20  Multiple angles q q=1 q=2 q=3 + + = 21  SIM vs. Widefield Widefield filtered SIM reconstruction 22  SIM in a nutshell “A w ” Resolution limit independent of light going in or out of the sample Structure the illumination light (fine pattern at res. limit) Take 9 or 15 images Reconstruct into super-resolved image Next parts: Parameter extraction & filtering Resolution, PSFs & OTFS How to build a SIM microscope 24