Sleep Health and Wearable Technology: Algorithmic Development towards Field-based Sleep Monitoring

Abstract

Wearable devices have rapidly become essential tools for tracking sleep in natural, non-clinical settings. Despite their widespread adoption, consumer-grade wearables, such as smartwatches and fitness trackers, exhibit significant limitations in their ability to accurately track wake epochs after sleep onset and classify sleep stages, particularly in individuals with sleep disorders. Recent independent validation studies reported frequent misclassifications, especially distinguishing Rapid Eye Movement (REM) sleep and non-REM (NREM) N3 from other sleep stages. This inaccuracy is exacerbated in clinical populations like those with obstructive sleep apnea (OSA), where sleep is fragmented and physiological signals and sleep patterns deviate from those seen in healthy individuals. Furthermore, it is practically impossible for independent academic researchers to develop and evaluate sleep staging algorithms from consumer devices due to their proprietary nature and a lack of publicly available datasets for wearable sleep staging. The need for precise, reliable, and scalable sleep tracking methods in wearable devices is crucial as wearables become more integrated into both personal health management and clinical applications.To address these challenges, I collected and published the DREAMT (Dataset for Real-time sleep stage EstimAtion using Multisensor wearable Technology) dataset,1 a unique collection of multimodal physiological data recorded from 100 participants diagnosed with varying severities of OSA at the Duke Sleep Disorders Center. The DREAMT dataset includes synchronized recordings of both wearable device data, obtained using Empatica E4 smartwatches, and sleep stage annotations and sleep apnea events annotated by certified sleep technicians based on clinical polysomnography (PSG), the gold standard for clinical sleep studies and sleep staging. This dataset is the first and only high-resolution wearable smartwatch dataset with reliable sleep stages made public. It is an indispensable resource for advancing the development and validation of sleep staging algorithms capable of accurately detecting sleep patterns using wearable smartwatches by serving as a benchmark for the research community to develop and compare new algorithmic development. It represents an important step towards establishing an open science framework for wearable-based sleep research. Leveraging this dataset, I proposed a model approach to predict sleep vs wake, combining feature engineering, Light Gradient Boosting Machines (LightGBM) with Gaussian Process-based mixed effects modeling (GPBoost) for epoch-by-epoch sleep/wake prediction, and a Long Short-Term Memory (LSTM) network for post-processing. The LSTM module is an innovative approach to improve sleep/wake detection by capturing the temporal dependencies within these physiological signals. This feature engineering process significantly enhanced the model’s ability to detect transitions between wakefulness and sleep, especially in cases of individuals with sleep disorders by recognizing that individuals of varying degrees of sleep disorder severity are very likely to exhibit different sleep patterns and behaviors. This ensemble model established a baseline and also provided a foundation for exploring more sophisticated deep learning architectures tailored to wearable sleep data. Building on this foundation and to utilize the existing large external PSG datasets, I designed WatchSleepNet to predict wake vs NREM vs REM, a deep learning model specifically developed to tackle the inherent challenges of wearable-based sleep staging. WatchSleepNet integrates Convolutional Neural Networks (CNNs), Temporal Convolutional Networks (TCNs), and bidirectional LSTM networks with multi-head attention mechanisms to process Inter-beat Interval (IBI) signals for sequence-to-sequence classification. The model is trained to recognize both spatial and temporal dependencies in the physiological data, enabling it to accurately classify sleep stages in terms of wake, NREM, and REM. One of the unique strengths of WatchSleepNet is its ability to leverage IBI values calculated from both ECG and PPG signals available in large external PSG datasets, including the Sleep Heart Health Study (SHHS) and the Multi-Ethnic Study of Atherosclerosis (MESA). This pretraining step allowed the model to learn foundational patterns in sleep physiology across diverse populations and levels of sleep disorder severity. Following pretraining, the model was fine-tuned using the DREAMT dataset, ensuring that it could adapt to the unique characteristics of wrist-based PPG data collected in real-world settings. WatchSleepNet demonstrated superior performance compared to state-of-the-art models like SleepConvNet and InsightSleepNet, achieving significant improvements in REM sleep classification, an area where consumer-grade wearables typically perform poorly. The model achieved a REM F1-score of 0.649 and AUROC of 0.938, significantly higher than the results from the benchmark algorithms, highlighting its potential to bridge the gap between consumer and gold-standard in-clinic sleep tracking. Beyond model development, this dissertation contributes to the broader field of digital health, particularly in promoting open science and the standardization of wearable sensor data for sleep research. By publishing the DREAMT dataset and developing reproducible methodologies for digital sleep biomarkers, this work sets the stage for more transparent, collaborative research in wearable-based sleep tracking. Additionally, I explored the clinical utility of wearable sleep monitoring in detecting circadian rhythm disruptions and their impacts on mental health in adolescents. Circadian misalignments, common among shift workers and individuals with sleep disorders, are linked to an increased risk of mood disorders, anxiety, and metabolic dysfunction. By advancing the accuracy and reliability of wearable devices in tracking these disruptions, this research opens new avenues for early detection, intervention, and management of these conditions. Overall, this dissertation presents key innovations in wearable-based sleep monitoring through the presentation of the DREAMT dataset, development of the WatchSleepNet deep learning model, expanding the clinical applicability of wearable sleep tracking, and promoting open science and standardization in the field of sleep digital health research. Together, they represent a significant advancement in the ability to perform accurate, scalable, and clinically applicable sleep staging using wearable devices. These contributions not only enhance the field of sleep medicine but also offer a foundation for future research in sleep digital health, focusing on the integration of wearable technologies for improving sleep health, monitoring circadian rhythms, and supporting mental health interventions.