||<p>This thesis naturally divides itself into two sections. The first two chapters
concern</p><p>the development of Bayesian semi-parametric models for arrival times.
Chapter 2</p><p>considers Bayesian inference for a Gaussian process modulated temporal
inhomogeneous Poisson point process, made challenging by an intractable likelihood.
The intractable likelihood is circumvented by two novel data augmentation strategies
which result in Gaussian measurements of the Gaussian process, connecting the model
with a larger literature on modelling time-dependent functions from Bayesian non-parametric
regression to time series. A scalable state-space representation of the Matern Gaussian
process in 1 dimension is used to provide access to linear time filtering algorithms
for performing inference. An MCMC algorithm based on Gibbs sampling with slice-sampling
steps is provided and illustrated on simulated and real datasets. The MCMC algorithm
exhibits excellent mixing and scalability.</p><p>Chapter 3 builds on the previous
model to detect specific signals in temporal point patterns arising in neuroscience.
The firing of a neuron over time in response to an external stimulus generates a temporal
point pattern or ``spike train''. Of special interest is how neurons encode information
from dual simultaneous external stimuli. Among many hypotheses is the presence multiplexing
- interleaving periods of firing as it would for each individual stimulus in isolation.
Statistical models are developed to quantify evidence for a variety of experimental
hypotheses. Each experimental hypothesis translates to a particular form of intensity
function for the dual stimuli trials. The dual stimuli intensity is modelled as a
dynamic superposition of single stimulus intensities, defined by a time-dependent
weight function that is modelled non-parametrically as a transformed Gaussian process.
Experiments on simulated data demonstrate that the model is able to learn the weight
function very well, but other model parameters which have meaningful physical interpretations
less well.</p><p>Chapters 4 and 5 concern mathematical optimization and theoretical
properties of Bayesian models for variable selection. Such optimizations are challenging
due to non-convexity, non-smoothness and discontinuity of the objective. Chapter 4
presents advances in continuous optimization algorithms based on relating mathematical
and statistical approaches defined in connection with several iterative algorithms
for penalized linear</p><p>regression. I demonstrate the equivalence of parameter
mappings using EM under</p><p>several data augmentation strategies - location-mixture
representations, orthogonal data augmentation and LQ design matrix decompositions.
I show that these</p><p>model-based approaches are equivalent to algorithmic derivation
via proximal</p><p>gradient methods. This provides new perspectives on model-based
and algorithmic</p><p>approaches, connects across several research themes in optimization
and statistics,</p><p>and provides access, beyond EM, to relevant theory from the
proximal gradient</p><p>and convex analysis literatures.</p><p>Chapter 5 presents
a modern and technologically up-to-date approach to discrete optimization for variable
selection models through their formulation as mixed integer programming models. Mixed
integer quadratic and quadratically constrained programs are developed for the point-mass-Laplace
and g-prior. Combined with warm-starts and optimality-based bounds tightening procedures
provided by the heuristics of the previous chapter, the MIQP model developed for the
point-mass-Laplace prior converges to global optimality in a matter of seconds for
moderately sized real datasets. The obtained estimator is demonstrated to possess
superior predictive performance over that obtained by cross-validated lasso in a number
of real datasets. The MIQCP model for the g-prior struggles to match the performance
of the former and highlights the fact that the performance of the mixed integer solver
depends critically on the ability of the prior to rapidly concentrate posterior mass
on good models.</p>