Learned sensing: jointly optimized microscope hardware for accurate image classification.
Abstract
Since its invention, the microscope has been optimized for interpretation by a human
observer. With the recent development of deep learning algorithms for automated image
analysis, there is now a clear need to re-design the microscope's hardware for specific
interpretation tasks. To increase the speed and accuracy of automated image classification,
this work presents a method to co-optimize how a sample is illuminated in a microscope,
along with a pipeline to automatically classify the resulting image, using a deep
neural network. By adding a "physical layer" to a deep classification network, we
are able to jointly optimize for specific illumination patterns that highlight the
most important sample features for the particular learning task at hand, which may
not be obvious under standard illumination. We demonstrate how our learned sensing
approach for illumination design can automatically identify malaria-infected cells
with up to 5-10% greater accuracy than standard and alternative microscope lighting
designs. We show that this joint hardware-software design procedure generalizes to
offer accurate diagnoses for two different blood smear types, and experimentally show
how our new procedure can translate across different experimental setups while maintaining
high accuracy.
Type
Journal articleSubject
Science & TechnologyLife Sciences & Biomedicine
Physical Sciences
Biochemical Research Methods
Optics
Radiology, Nuclear Medicine & Medical Imaging
Biochemistry & Molecular Biology
FOURIER PTYCHOGRAPHY
HIGH-RESOLUTION
ILLUMINATION
Permalink
https://hdl.handle.net/10161/19766Published Version (Please cite this version)
10.1364/BOE.10.006351Publication Info
Muthumbi, Alex; Chaware, Amey; Kim, Kanghyun; Zhou, Kevin C; Konda, Pavan Chandra;
Chen, Richard; ... Horstmeyer, Roarke (2019). Learned sensing: jointly optimized microscope hardware for accurate image classification.
Biomedical optics express, 10(12). pp. 6351-6369. 10.1364/BOE.10.006351. Retrieved from https://hdl.handle.net/10161/19766.This is constructed from limited available data and may be imprecise. To cite this
article, please review & use the official citation provided by the journal.
Collections
More Info
Show full item recordScholars@Duke
Roarke Horstmeyer
Assistant Professor of Biomedical Engineering
Roarke Horstmeyer is an assistant professor within Duke's Biomedical Engineering Department.
He develops microscopes, cameras and computer algorithms for a wide range of applications,
from forming 3D reconstructions of organisms to detecting neural activity deep within
tissue. His areas of interest include optics, signal processing, optimization and
neuroscience. Most recently, Dr. Horstmeyer was a guest professor at the University
of Erlangen in Germany and an Einstein postdoctoral fellow at Ch
Kevin Zhou
Affiliate
Alphabetical list of authors with Scholars@Duke profiles.

Articles written by Duke faculty are made available through the campus open access policy. For more information see: Duke Open Access Policy
Rights for Collection: Scholarly Articles
Works are deposited here by their authors, and represent their research and opinions, not that of Duke University. Some materials and descriptions may include offensive content. More info