Digital Stack Photography and Its Applications
This work centers on digital stack photography and its applications.
A stack of images refer, in a broader sense, to an ensemble of
associated images taken with variation in one or more than one various
values in one or more parameters in system configuration or setting.
An image stack captures and contains potentially more information than
any of the constituent images. Digital stack photography (DST)
techniques explore the rich information to render a synthesized image
that oversteps the limitation in a digital camera's capabilities.
This work considers in particular two basic DST problems, which had
been challenging, and their applications. One is high-dynamic-range
(HDR) imaging of non-stationary dynamic scenes, in which the stacked
images vary in exposure conditions. The other
is large scale panorama composition from multiple images. In this
case, the image components are related to each other by the spatial
relation among the subdomains of the same scene they covered and
captured jointly. We consider the non-conventional, practical and
challenge situations where the spatial overlap among the sub-images is
sparse (S), irregular in geometry and imprecise from the designed
geometry (I), and the captured data over the overlap zones are noisy
(N) or lack of features. We refer to these conditions simply as the
S.I.N. conditions.
There are common challenging issues with both problems. For example,
both faced the dominant problem with image alignment for
seamless and artifact-free image composition. Our solutions to the
common problems are manifested differently in each of the particular
problems, as a result of adaption to the specific properties in each
type of image ensembles. For the exposure stack, existing
alignment approaches struggled to overcome three main challenges:
inconsistency in brightness, large displacement in dynamic scene and
pixel saturation. We exploit solutions in the following three
aspects. In the first, we introduce a model that addresses and admits
changes in both geometric configurations and optical conditions, while
following the traditional optical flow description. Previous models
treated these two types of changes one or the other, namely, with
mutual exclusions. Next, we extend the pixel-based optical flow model
to a patch-based model. There are two-fold advantages. A patch has
texture and local content that individual pixels fail to present. It
also renders opportunities for faster processing, such as via
two-scale or multiple-scale processing. The extended model is then
solved efficiently with an EM-like algorithm, which is reliable in the
presence of large displacement. Thirdly, we present a generative
model for reducing or eliminating typical artifacts as a side effect
of an inadequate alignment for clipped pixels. A patch-based texture
synthesis is combined with the patch-based alignment to achieve an
artifact free result.
For large-scale panorama composition under the S.I.N. conditions, we
have developed an effective solution scheme that significantly reduces
both processing time and artifacts. Previously existing approaches can
be roughly categorized as either geometry-based composition or feature
based composition. In the former approach, one relies on precise
knowledge of the system geometry, by design and/or calibration. It
works well with a far-away scene, in which case there is only limited
variation in projective geometry among the sub-images. However, the
system geometry is not invariant to physical conditions such as
thermal variation, stress variation and etc.. The composition with
this approach is typically done in the spatial space. The other
approach is more robust to geometric and optical conditions. It works
surprisingly well with feature-rich and stationary scenes, not well
with the absence of recognizable features. The composition based on
feature matching is typically done in the spatial gradient domain. In
short, both approaches are challenged by the S.I.N. conditions. With
certain snapshot data sets obtained and contributed by Brady et al,
these methods either fail in composition or render images with
visually disturbing artifacts. To overcome the S.I.N. conditions, we
have reconciled these two approaches and made successful and
complementary use of both priori and approximate information about
geometric system configuration and the feature information from the
image data. We also designed and developed a software architecture
with careful extraction of primitive function modules that can be
efficiently implemented and executed in parallel. In addition to a
much faster processing speed, the resulting images are clear and
sharper at the overlapping zones, without typical ghosting artifacts.
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.
Rights for Collection: Duke Dissertations