Learned sensing: jointly optimized microscope hardware for accurate image classification.


Since its invention, the microscope has been optimized for interpretation by a human observer. With the recent development of deep learning algorithms for automated image analysis, there is now a clear need to re-design the microscope's hardware for specific interpretation tasks. To increase the speed and accuracy of automated image classification, this work presents a method to co-optimize how a sample is illuminated in a microscope, along with a pipeline to automatically classify the resulting image, using a deep neural network. By adding a "physical layer" to a deep classification network, we are able to jointly optimize for specific illumination patterns that highlight the most important sample features for the particular learning task at hand, which may not be obvious under standard illumination. We demonstrate how our learned sensing approach for illumination design can automatically identify malaria-infected cells with up to 5-10% greater accuracy than standard and alternative microscope lighting designs. We show that this joint hardware-software design procedure generalizes to offer accurate diagnoses for two different blood smear types, and experimentally show how our new procedure can translate across different experimental setups while maintaining high accuracy.





Published Version (Please cite this version)


Publication Info

Muthumbi, Alex, Amey Chaware, Kanghyun Kim, Kevin C Zhou, Pavan Chandra Konda, Richard Chen, Benjamin Judkewitz, Andreas Erdmann, et al. (2019). Learned sensing: jointly optimized microscope hardware for accurate image classification. Biomedical optics express, 10(12). pp. 6351–6369. 10.1364/BOE.10.006351 Retrieved from https://hdl.handle.net/10161/19766.

This is constructed from limited available data and may be imprecise. To cite this article, please review & use the official citation provided by the journal.




Kevin Zhou


Roarke Horstmeyer

Assistant Professor of Biomedical Engineering

Roarke Horstmeyer is an assistant professor within Duke's Biomedical Engineering Department. He develops microscopes, cameras and computer algorithms for a wide range of applications, from forming 3D reconstructions of organisms to detecting neural activity deep within tissue. His areas of interest include optics, signal processing, optimization and neuroscience. Most recently, Dr. Horstmeyer was a guest professor at the University of Erlangen in Germany and an Einstein postdoctoral fellow at Charitè Medical School in Berlin. Prior to his time in Germany, Dr. Horstmeyer earned a PhD from Caltech’s electrical engineering department in 2016, a master of science degree from the MIT Media Lab in 2011, and a bachelors degree in physics and Japanese from Duke University in 2006.