High Throughput Fourier Light Field imaging of 3D Biological Dynamics

Abstract

Since the mid-20th century, Computational Optics has combined the design of advanced microscopes and imaging systems with novel image processing and machine learning algorithms to extract meaningful information from complex samples. As computational speed and power continue to accelerate, so too does our ability to rapidly analyze and derive insights from large datasets. However, there are still constraints on the throughput of imaging systems themselves, or the volume of data they can generate and transmit. Specifically, it is challenging to simultaneously maximize the speed, resolution, and field-of-view of imaging systems. This is especially pronounced in the 3D imaging of biological samples, which are often highly dynamic and may move rapidly during observation. Depending on the technique employed, 3D imaging also requires significantly more data throughput than 2D approaches to retain the same imaging view and resolution. In this dissertation, we combine Fourier Light Field imaging -- an established approach for dynamic 3D microscopy -- with high throughput optical components, advanced imaging sensors, and image processing algorithms to demonstrate novel 3D imaging systems for dynamic, large field-of-view, and high resolution imaging of biological samples. First, we demonstrate an ultra-high speed, Fourier Light Field 3D imaging system for the analysis of exoskeleton deformations in snapping trap-jaw ants. Paired with a custom image processing pipeline, we use this system to derive novel insights about the energy storage and release mechanisms behind trap-jaw ant mandible strikes, by revealing micron-scale deformations over the $mm$-scale, moving ant head -- all captured at 100$kHz$. Next, we use the previously developed Multi-Camera Array Microscope (MCAM) to develop a high throughput Fourier Light Field Array (FLFA) microscope, which can capture high resolution images of a sample from 48 unique perspectives simultaneously, while operating at video rates. We use a self-supervised 3D reconstruction algorithm to demonstrate the utility of this system for 3D visualization in micro-surgical settings. We additionally perform an ablation study to understand the impact of viewpoint configuration on 3D reconstruction from multi-angle images. Finally, we adapt the FLFA system for multi-channel fluorescence guided surgery. We perform a proof-of-concept study using this system to simultaneously image three types of fluorescent agents accumulated in tumor growths within ex-vivo rat brains. Collectively, these systems demonstrate how Fourier Light Field imaging can be used alongside high throughput optical components and advanced image processing algorithms to gain new insights into and visualizations of dynamic biological systems.