| dc.description.abstract |
<p>Statistical machine learning becomes a more important workload for computing systems
than ever before. Probabilistic computing is a popular approach in statistical machine
learning, which solves problems by iteratively generating samples from parameterized
distributions. As an alternative to Deep Neural Networks, probabilistic computing
provides conceptually simple, compositional, and interpretable models. However, probabilistic
algorithms are often considered too slow on the conventional processors due to sampling
overhead to 1) computing the parameters of a distribution and 2) generating samples
from the parameterized distribution. A specialized architecture is needed to address
both the above aspects. </p><p>In this dissertation, we claim a specialized architecture
is necessary and feasible to efficiently support various probabilistic computing problems
in statistical machine learning, while providing high-quality and robust results.</p><p>We
start with exploring a probabilistic architecture to accelerate Markov Random Field
(MRF) Gibbs Sampling by utilizing the quantum randomness of optical-molecular devices---Resonance
Energy Transfer (RET) networks. We provide a macro-scale prototype, the first such
system to our knowledge, to experimentally demonstrate the capability of RET devices
to parameterize a distribution and run a real application. By doing a quantitative
result quality analysis, we further reveal the design issues of an existing RET-based
probabilistic computing unit (1st-gen RSU-G) that lead to unsatisfactory result quality
in some applications. By exploring the design space, we propose a new RSU-G microarchitecture
that empirically achieves the same result quality as 64-bit floating-point software,
with the same area and modest power overheads compared with 1st-gen RSU-G. An efficient
stochastic probabilistic unit can be fulfilled using RET devices. </p><p>The RSU-G
provides high-quality true Random Number Generation (RNG). We further explore how
quality of an RNG is related to application end-point result quality. Unexpectedly,
we discover the target applications do not necessarily require high-quality RNGs---a
simple 19-bit Linear-Feedback Shift Register (LFSR) does not degrade end-point result
quality in the tested applications. Therefore, we propose a Stochastic Processing
Unit (SPU) with a simple pseudo RNG that achieves equivalent function to RSU-G but
maintains the benefit of a CMOS digital circuit. </p><p>The above results bring up
a subsequent question: are we confident to use a probabilistic accelerator with various
approximation techniques, even though the end-point result quality ("accuracy") is
good in tested benchmarks? We found current methodologies for evaluating correctness
of probabilistic accelerators are often incomplete, mostly focusing only on end-point
result quality ("accuracy") but omitting other important statistical properties. Therefore,
we claim a probabilistic architecture should provide some measure (or guarantee) of
statistical robustness. We take a first step toward defining metrics and a methodology
for quantitatively evaluating correctness of probabilistic accelerators. We propose
three pillars of statistical robustness: 1) sampling quality, 2) convergence diagnostic,
and 3) goodness of fit. We apply our framework to a representative MCMC accelerator
(SPU) and surface design issues that cannot be exposed using only application end-point
result quality. Finally, we demonstrate the benefits of this framework to guide design
space exploration in a case study showing that statistical robustness comparable to
floating-point software can be achieved with limited precision, avoiding floating-point
hardware overheads. </p>
|
|
