Physical Designs in Artificial Neural Imaging

dc.contributor.advisor

Brady, David J

dc.contributor.author

Huang, Qian

dc.date.accessioned

2023-03-28T21:41:23Z

dc.date.available

2023-03-28T21:41:23Z

dc.date.issued

2022

dc.department

Electrical and Computer Engineering

dc.description.abstract

Artificial neural networks fundamentally shift the paradigm of computational imaging. Powerful neural processing is not only taking place of the conventional algorithms, but also embracing radical and physically plausible forward models that better sample the high dimensional light field. Physical designs of sampling in turn tailor simulation and neural algorithms for optimal inverse estimation. Sampling, simulation and neural algorithms as three essential components compose a novel imaging paradigm -- artificial neural imaging, in which they interact and improve themselves in an upward spiral.

Here we present three concrete examples of artificial neural imaging and the important roles physical designs play. In all-in-focus imaging, we see autofocus, sampling and fusion algorithms are redefined for optimizing the image quality of a camera with limited depth of field. Image-based neural autofocus acts 5-10x faster than traditional algorithms. The focus control based on the rule or reinforcement learning dynamically estimates the environment and optimizes the focus trajectory. Along with the neural fusion algorithm, the pipeline outperforms traditional focal stacking approaches in static and dynamic scenes. In scatter ptychography, we show imaging the secondary scatters reflected by a remote target under coherent illumination can create a synthetic aperture on the scatterer. The reconstruction of the object through phase retrieval algorithms can drastically exceed the resolution of directly viewing the target. In the lab experiment we demonstrate 32x resolution improvement relative to direct imaging using error-reduction and plug-and-play algorithms. In array camera imaging, we demonstrate heterogeneous multiaperture designs that have better sampling structures and physics-aware transformers for feature-based data fusion. The proposed transformer incorporates the physical information of the camera array as its receptive fields, demonstrating the superior ability of image compositing on array cameras with diverse resolutions, focal lengths, focal planes, color spaces, and exposures. We also demonstrate a scalable pipeline of data synthesis through computer graphics software that empowers the transformers.

The examples above justify artificial neural imaging and the physical designs interweaved. We expect better designs in sampling, simulation, neural algorithms and eventually better estimation of the light field.

dc.identifier.uri

https://hdl.handle.net/10161/26805

dc.subject

Electrical engineering

dc.subject

Computer science

dc.subject

Computer engineering

dc.subject

Artificial Neural Network

dc.subject

Computational Imaging

dc.subject

Computer Vision

dc.subject

Deep Learning

dc.title

Physical Designs in Artificial Neural Imaging

dc.type

Dissertation

Files

Original bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
Huang_duke_0066D_17005.pdf
Size:
37.28 MB
Format:
Adobe Portable Document Format

Collections