Browsing by Subject "Ensemble"

The resurgence of Convolutional Neural Networks (CNNs) from the early foundational work is largely attributed to the advent of extensive manually labeled datasets, which has made it possible to train high-capacity models with strong generalization capabilities. However, the annotation cost for these datasets is often prohibitive, and so training CNNs on limited data in a fully-supervised setting remains a crucial problem. Data augmentation is a promising direction for improving generalization in scarce data settings.

We study foundational augmentation techniques, including Mixed Sample Data Augmentations (MSDAs) and a no-parameter variant of RandAugment termed Preset-RandAugment, in the fully supervised scenario. We observe that Preset-RandAugment excels in limited-data contexts while MSDAs are moderately effective. In order to explain this behaviour, we refine ideas about diversity and realism from prior work and propose new ways to measure them. We postulate an additional property when data is limited: augmentations should encourage faster convergence by helping the model learn stable and invariant low-level features, focusing on less class-specific patterns. We explain the effectiveness of Preset-RandAugment in terms of these properties and identify low-level feature transforms as a key contributor to performance.

Building on these insights, we introduce a novel augmentation technique called RandMSAugment that integrates complementary strengths of existing methods. It combines low-level feature transforms from Preset-RandAugment with interpolation and cut-and-paste from MSDA. We improve image diversity through added stochasticity in the mixing process. RandMSAugment significantly outperforms the competition on CIFAR-100, STL-10, and Tiny-Imagenet. With very small training sets (4, 25, 100 samples/class), RandMSAugment achieves compelling performance gains between 4.1\% and 6.75\%. Even with more training data (500 samples/class) we improve performance by 1.03\% to 2.47\%. We also incorporate RandMSAugment augmentations into a semi-supervised learning (SSL) framework and show promising improvements over the state-of-the-art SSL method, FlexMatch. The improvements are more significant when the number of labeled samples is smaller. RandMSAugment does not require hyperparameter tuning, extra validation data, or cumbersome optimizations.

Finally, we combine RandMSAugment with another powerful generalization tool, ensembling, for fully-supervised training with limited samples. We show additonal improvements on the 3 classification benchmarks, which range between 2\% and 5\%. We empirically demonstrate that the gains due to ensembling are larger when the individual networks have moderate accuracies \ie outside of the low and high extremes.Furthermore, we introduce a simulation tool capable of providing insights about the maximum accuracy achievable through ensembling, under various conditions.

As non-coding RNAs are increasingly implicated in cellular regulatory functions and disease states, there is a need to deepen our understanding of RNA structure-function relationships as well as to develop methods targeting RNA with small molecules. The transactivation response element (TAR) RNA from human immunodeficiency virus type 1 (HIV-1) is an established drug target for the development of anti-HIV therapeutics and has served as a model system for understanding RNA dynamics and RNA:ligand interactions. Like many RNAs, HIV-1 TAR is a highly flexible molecule that experiences dynamics ranging from local fluctuations in base orientation and interhelical angles to higher-order dynamics that transiently alter base pairing away from the ground state (GS) secondary structure. The work presented in this thesis is aimed at developing approaches targeting TAR with small molecules that integrate its broad range of structural dynamics.

First, nuclear magnetic resonance (NMR) chemical shift mapping is applied in concert with fluorescence binding assays and computational docking to efficiently characterize the TAR-binding modes of a focused library of amiloride derivatives. Through this work, amiloride is established as a novel RNA binding scaffold with interesting structure-activity relationships. Ultimately, this approach yielded ten novel TAR binders with demonstrated selectivity for TAR over tRNA and with up to a 100-fold increase in activity over the parent dimethyl amiloride compound.

Next, we demonstrate that ensemble-based virtual screening (EBVS) is a powerful approach to predict ligand binding for flexible RNA targets. Here, we generate a library to evaluate EBVS enrichment by subjecting HIV-1 TAR to experimental high-throughput screening against ~100,000 drug-like small molecules. EBVS against a dynamic ensemble of the TAR GS determined previously by combining NMR spectroscopy data and molecular dynamics (MD) simulations scores hits and non-hits with an area under the receiver operator characteristic curve of ~0.85-0.94 and with ~40-75% of all hits falling within the top 2% of scored molecules. Importantly, the enrichment was shown to depend on the accuracy of the ensemble.

Finally, we explore the novel strategy of specifically targeting non-native RNA excited state conformations inspired by the fact that their altered secondary structures are likely functionally inactive and highly unique. We use a mutational stabilize-and-rescue approach to demonstrate that TAR ES2 dramatically inhibits TAR activity in cells, suggesting that stabilizing the ES conformation with small molecules would similarly inhibit activity. To pursue TAR ES2 as a potential target, we have determined the first-ever dynamic ensemble of an RNA ES using a combination of MD and NMR residual dipolar couplings (RDCs) measured on a highly accurate ES2-stabilizing mutant. This dynamic ensemble was subjected to our validated EBVS approach to identify small molecules that bind and stabilize TAR ES2. Using NMR chemical shift fingerprinting, we have identified molecules that bind the TAR ES2 structure, including two that induce significant broadening in wtTAR consistent with chemical exchange and two that show a preference for TAR ES2 over the GS.

Together, this work explores multiple novel strategies for structure-specific RNA targeting.

An abstract of a thesis devoted to using helix-coil models to study unfolded states.\\

Research on polypeptide unfolded states has received much more attention in the last decade or so than it has in the past. Unfolded states are thought to be implicated in various

misfolding diseases and likely play crucial roles in protein folding equilibria and folding rates. Structural characterization of unfolded states has proven to be

much more difficult than the now well established practice of determining the structures of folded proteins. This is largely because many core assumptions underlying

folded structure determination methods are invalid for unfolded states. This has led to a dearth of knowledge concerning the nature of unfolded state conformational

distributions. While many aspects of unfolded state structure are not well known, there does exist a significant body of work stretching back half a century that

has been focused on structural characterization of marginally stable polypeptide systems. This body of work represents an extensive collection of experimental

data and biophysical models associated with describing helix-coil equilibria in polypeptide systems. Much of the work on unfolded states in the last decade has not been devoted

specifically to the improvement of our understanding of helix-coil equilibria, which arguably is the most well characterized of the various conformational equilibria

that likely contribute to unfolded state conformational distributions. This thesis seeks to provide a deeper investigation of helix-coil equilibria using modern

statistical data analysis and biophysical modeling techniques. The studies contained within seek to provide deeper insights and new perspectives on what we presumably

know very well about protein unfolded states. \\

Chapter 1 gives an overview of recent and historical work on studying protein unfolded states. The study of helix-coil equilibria is placed in the context

of the general field of unfolded state research and the basics of helix-coil models are introduced.\\

Chapter 2 introduces the newest incarnation of a sophisticated helix-coil model. State of the art modern statistical techniques are employed to estimate the energies

of various physical interactions that serve to influence helix-coil equilibria. A new Bayesian model selection approach is utilized to test many long-standing

hypotheses concerning the physical nature of the helix-coil transition. Some assumptions made in previous models are shown to be invalid and the new model

exhibits greatly improved predictive performance relative to its predecessor. \\

Chapter 3 introduces a new statistical model that can be used to interpret amide exchange measurements. As amide exchange can serve as a probe for residue-specific

properties of helix-coil ensembles, the new model provides a novel and robust method to use these types of measurements to characterize helix-coil ensembles experimentally

and test the position-specific predictions of helix-coil models. The statistical model is shown to perform exceedingly better than the most commonly used

method for interpreting amide exchange data. The estimates of the model obtained from amide exchange measurements on an example helical peptide

also show a remarkable consistency with the predictions of the helix-coil model. \\

Chapter 4 involves a study of helix-coil ensembles through the enumeration of helix-coil configurations. Aside from providing new insights into helix-coil ensembles,

this chapter also introduces a new method by which helix-coil models can be extended to calculate new types of observables. Future work on this approach could potentially

allow helix-coil models to move into use domains that were previously inaccessible and reserved for other types of unfolded state models that were introduced in chapter 1.