Browsing by Author "Xu, Pan"

Now showing 1 - 4 of 4

Open Access
Challenges of COVID-19 Case Forecasting in the US, 2020-2021.
(PLoS computational biology, 2024-05) Lopez, Velma K; Cramer, Estee Y; Pagano, Robert; Drake, John M; O'Dea, Eamon B; Adee, Madeline; Ayer, Turgay; Chhatwal, Jagpreet; Dalgic, Ozden O; Ladd, Mary A; Linas, Benjamin P; Mueller, Peter P; Xiao, Jade; Bracher, Johannes; Castro Rivadeneira, Alvaro J; Gerding, Aaron; Gneiting, Tilmann; Huang, Yuxin; Jayawardena, Dasuni; Kanji, Abdul H; Le, Khoa; Mühlemann, Anja; Niemi, Jarad; Ray, Evan L; Stark, Ariane; Wang, Yijin; Wattanachit, Nutcha; Zorn, Martha W; Pei, Sen; Shaman, Jeffrey; Yamana, Teresa K; Tarasewicz, Samuel R; Wilson, Daniel J; Baccam, Sid; Gurung, Heidi; Stage, Steve; Suchoski, Brad; Gao, Lei; Gu, Zhiling; Kim, Myungjin; Li, Xinyi; Wang, Guannan; Wang, Lily; Wang, Yueying; Yu, Shan; Gardner, Lauren; Jindal, Sonia; Marshall, Maximilian; Nixon, Kristen; Dent, Juan; Hill, Alison L; Kaminsky, Joshua; Lee, Elizabeth C; Lemaitre, Joseph C; Lessler, Justin; Smith, Claire P; Truelove, Shaun; Kinsey, Matt; Mullany, Luke C; Rainwater-Lovett, Kaitlin; Shin, Lauren; Tallaksen, Katharine; Wilson, Shelby; Karlen, Dean; Castro, Lauren; Fairchild, Geoffrey; Michaud, Isaac; Osthus, Dave; Bian, Jiang; Cao, Wei; Gao, Zhifeng; Lavista Ferres, Juan; Li, Chaozhuo; Liu, Tie-Yan; Xie, Xing; Zhang, Shun; Zheng, Shun; Chinazzi, Matteo; Davis, Jessica T; Mu, Kunpeng; Pastore Y Piontti, Ana; Vespignani, Alessandro; Xiong, Xinyue; Walraven, Robert; Chen, Jinghui; Gu, Quanquan; Wang, Lingxiao; Xu, Pan; Zhang, Weitong; Zou, Difan; Gibson, Graham Casey; Sheldon, Daniel; Srivastava, Ajitesh; Adiga, Aniruddha; Hurt, Benjamin; Kaur, Gursharn; Lewis, Bryan; Marathe, Madhav; Peddireddy, Akhil Sai; Porebski, Przemyslaw; Venkatramanan, Srinivasan; Wang, Lijing; Prasad, Pragati V; Walker, Jo W; Webber, Alexander E; Slayton, Rachel B; Biggerstaff, Matthew; Reich, Nicholas G; Johansson, Michael A
During the COVID-19 pandemic, forecasting COVID-19 trends to support planning and response was a priority for scientists and decision makers alike. In the United States, COVID-19 forecasting was coordinated by a large group of universities, companies, and government entities led by the Centers for Disease Control and Prevention and the US COVID-19 Forecast Hub (https://covid19forecasthub.org). We evaluated approximately 9.7 million forecasts of weekly state-level COVID-19 cases for predictions 1-4 weeks into the future submitted by 24 teams from August 2020 to December 2021. We assessed coverage of central prediction intervals and weighted interval scores (WIS), adjusting for missing forecasts relative to a baseline forecast, and used a Gaussian generalized estimating equation (GEE) model to evaluate differences in skill across epidemic phases that were defined by the effective reproduction number. Overall, we found high variation in skill across individual models, with ensemble-based forecasts outperforming other approaches. Forecast skill relative to the baseline was generally higher for larger jurisdictions (e.g., states compared to counties). Over time, forecasts generally performed worst in periods of rapid changes in reported cases (either in increasing or decreasing epidemic phases) with 95% prediction interval coverage dropping below 50% during the growth phases of the winter 2020, Delta, and Omicron waves. Ideally, case forecasts could serve as a leading indicator of changes in transmission dynamics. However, while most COVID-19 case forecasts outperformed a naïve baseline model, even the most accurate case forecasts were unreliable in key phases. Further research could improve forecasts of leading indicators, like COVID-19 cases, by leveraging additional real-time data, addressing performance across phases, improving the characterization of forecast confidence, and ensuring that forecasts were coherent across spatial scales. In the meantime, it is critical for forecast users to appreciate current limitations and use a broad set of indicators to inform pandemic-related decision making.
Open Access
Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.
(Proceedings of the National Academy of Sciences of the United States of America, 2022-04) Cramer, Estee Y; Ray, Evan L; Lopez, Velma K; Bracher, Johannes; Brennen, Andrea; Castro Rivadeneira, Alvaro J; Gerding, Aaron; Gneiting, Tilmann; House, Katie H; Huang, Yuxin; Jayawardena, Dasuni; Kanji, Abdul H; Khandelwal, Ayush; Le, Khoa; Mühlemann, Anja; Niemi, Jarad; Shah, Apurv; Stark, Ariane; Wang, Yijin; Wattanachit, Nutcha; Zorn, Martha W; Gu, Youyang; Jain, Sansiddh; Bannur, Nayana; Deva, Ayush; Kulkarni, Mihir; Merugu, Srujana; Raval, Alpan; Shingi, Siddhant; Tiwari, Avtansh; White, Jerome; Abernethy, Neil F; Woody, Spencer; Dahan, Maytal; Fox, Spencer; Gaither, Kelly; Lachmann, Michael; Meyers, Lauren Ancel; Scott, James G; Tec, Mauricio; Srivastava, Ajitesh; George, Glover E; Cegan, Jeffrey C; Dettwiller, Ian D; England, William P; Farthing, Matthew W; Hunter, Robert H; Lafferty, Brandon; Linkov, Igor; Mayo, Michael L; Parno, Matthew D; Rowland, Michael A; Trump, Benjamin D; Zhang-James, Yanli; Chen, Samuel; Faraone, Stephen V; Hess, Jonathan; Morley, Christopher P; Salekin, Asif; Wang, Dongliang; Corsetti, Sabrina M; Baer, Thomas M; Eisenberg, Marisa C; Falb, Karl; Huang, Yitao; Martin, Emily T; McCauley, Ella; Myers, Robert L; Schwarz, Tom; Sheldon, Daniel; Gibson, Graham Casey; Yu, Rose; Gao, Liyao; Ma, Yian; Wu, Dongxia; Yan, Xifeng; Jin, Xiaoyong; Wang, Yu-Xiang; Chen, YangQuan; Guo, Lihong; Zhao, Yanting; Gu, Quanquan; Chen, Jinghui; Wang, Lingxiao; Xu, Pan; Zhang, Weitong; Zou, Difan; Biegel, Hannah; Lega, Joceline; McConnell, Steve; Nagraj, VP; Guertin, Stephanie L; Hulme-Lowe, Christopher; Turner, Stephen D; Shi, Yunfeng; Ban, Xuegang; Walraven, Robert; Hong, Qi-Jun; Kong, Stanley; van de Walle, Axel; Turtle, James A; Ben-Nun, Michal; Riley, Steven; Riley, Pete; Koyluoglu, Ugur; DesRoches, David; Forli, Pedro; Hamory, Bruce; Kyriakides, Christina; Leis, Helen; Milliken, John; Moloney, Michael; Morgan, James; Nirgudkar, Ninad; Ozcan, Gokce; Piwonka, Noah; Ravi, Matt; Schrader, Chris; Shakhnovich, Elizabeth; Siegel, Daniel; Spatz, Ryan; Stiefeling, Chris; Wilkinson, Barrie; Wong, Alexander; Cavany, Sean; España, Guido; Moore, Sean; Oidtman, Rachel; Perkins, Alex; Kraus, David; Kraus, Andrea; Gao, Zhifeng; Bian, Jiang; Cao, Wei; Lavista Ferres, Juan; Li, Chaozhuo; Liu, Tie-Yan; Xie, Xing; Zhang, Shun; Zheng, Shun; Vespignani, Alessandro; Chinazzi, Matteo; Davis, Jessica T; Mu, Kunpeng; Pastore Y Piontti, Ana; Xiong, Xinyue; Zheng, Andrew; Baek, Jackie; Farias, Vivek; Georgescu, Andreea; Levi, Retsef; Sinha, Deeksha; Wilde, Joshua; Perakis, Georgia; Bennouna, Mohammed Amine; Nze-Ndong, David; Singhvi, Divya; Spantidakis, Ioannis; Thayaparan, Leann; Tsiourvas, Asterios; Sarker, Arnab; Jadbabaie, Ali; Shah, Devavrat; Della Penna, Nicolas; Celi, Leo A; Sundar, Saketh; Wolfinger, Russ; Osthus, Dave; Castro, Lauren; Fairchild, Geoffrey; Michaud, Isaac; Karlen, Dean; Kinsey, Matt; Mullany, Luke C; Rainwater-Lovett, Kaitlin; Shin, Lauren; Tallaksen, Katharine; Wilson, Shelby; Lee, Elizabeth C; Dent, Juan; Grantz, Kyra H; Hill, Alison L; Kaminsky, Joshua; Kaminsky, Kathryn; Keegan, Lindsay T; Lauer, Stephen A; Lemaitre, Joseph C; Lessler, Justin; Meredith, Hannah R; Perez-Saez, Javier; Shah, Sam; Smith, Claire P; Truelove, Shaun A; Wills, Josh; Marshall, Maximilian; Gardner, Lauren; Nixon, Kristen; Burant, John C; Wang, Lily; Gao, Lei; Gu, Zhiling; Kim, Myungjin; Li, Xinyi; Wang, Guannan; Wang, Yueying; Yu, Shan; Reiner, Robert C; Barber, Ryan; Gakidou, Emmanuela; Hay, Simon I; Lim, Steve; Murray, Chris; Pigott, David; Gurung, Heidi L; Baccam, Prasith; Stage, Steven A; Suchoski, Bradley T; Prakash, B Aditya; Adhikari, Bijaya; Cui, Jiaming; Rodríguez, Alexander; Tabassum, Anika; Xie, Jiajia; Keskinocak, Pinar; Asplund, John; Baxter, Arden; Oruc, Buse Eylul; Serban, Nicoleta; Arik, Sercan O; Dusenberry, Mike; Epshteyn, Arkady; Kanal, Elli; Le, Long T; Li, Chun-Liang; Pfister, Tomas; Sava, Dario; Sinha, Rajarishi; Tsai, Thomas; Yoder, Nate; Yoon, Jinsung; Zhang, Leyou; Abbott, Sam; Bosse, Nikos I; Funk, Sebastian; Hellewell, Joel; Meakin, Sophie R; Sherratt, Katharine; Zhou, Mingyuan; Kalantari, Rahi; Yamana, Teresa K; Pei, Sen; Shaman, Jeffrey; Li, Michael L; Bertsimas, Dimitris; Skali Lami, Omar; Soni, Saksham; Tazi Bouardi, Hamza; Ayer, Turgay; Adee, Madeline; Chhatwal, Jagpreet; Dalgic, Ozden O; Ladd, Mary A; Linas, Benjamin P; Mueller, Peter; Xiao, Jade; Wang, Yuanjia; Wang, Qinxia; Xie, Shanghong; Zeng, Donglin; Green, Alden; Bien, Jacob; Brooks, Logan; Hu, Addison J; Jahja, Maria; McDonald, Daniel; Narasimhan, Balasubramanian; Politsch, Collin; Rajanala, Samyak; Rumack, Aaron; Simon, Noah; Tibshirani, Ryan J; Tibshirani, Rob; Ventura, Valerie; Wasserman, Larry; O'Dea, Eamon B; Drake, John M; Pagano, Robert; Tran, Quoc T; Ho, Lam Si Tung; Huynh, Huong; Walker, Jo W; Slayton, Rachel B; Johansson, Michael A; Biggerstaff, Matthew; Reich, Nicholas G
Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks.
Open Access
Finite-Time Frequentist Regret Bounds of Multi-Agent Thompson Sampling on Sparse Hypergraphs
(2023-12-24) Jin, Tianyuan; Hsu, Hao-Lun; Chang, William; Xu, Pan
Open Access
Multiple models for outbreak decision support in the face of uncertainty.
(Proceedings of the National Academy of Sciences of the United States of America, 2023-05) Shea, Katriona; Borchering, Rebecca K; Probert, William JM; Howerton, Emily; Bogich, Tiffany L; Li, Shou-Li; van Panhuis, Willem G; Viboud, Cecile; Aguás, Ricardo; Belov, Artur A; Bhargava, Sanjana H; Cavany, Sean M; Chang, Joshua C; Chen, Cynthia; Chen, Jinghui; Chen, Shi; Chen, YangQuan; Childs, Lauren M; Chow, Carson C; Crooker, Isabel; Del Valle, Sara Y; España, Guido; Fairchild, Geoffrey; Gerkin, Richard C; Germann, Timothy C; Gu, Quanquan; Guan, Xiangyang; Guo, Lihong; Hart, Gregory R; Hladish, Thomas J; Hupert, Nathaniel; Janies, Daniel; Kerr, Cliff C; Klein, Daniel J; Klein, Eili Y; Lin, Gary; Manore, Carrie; Meyers, Lauren Ancel; Mittler, John E; Mu, Kunpeng; Núñez, Rafael C; Oidtman, Rachel J; Pasco, Remy; Pastore Y Piontti, Ana; Paul, Rajib; Pearson, Carl AB; Perdomo, Dianela R; Perkins, T Alex; Pierce, Kelly; Pillai, Alexander N; Rael, Rosalyn Cherie; Rosenfeld, Katherine; Ross, Chrysm Watson; Spencer, Julie A; Stoltzfus, Arlin B; Toh, Kok Ben; Vattikuti, Shashaank; Vespignani, Alessandro; Wang, Lingxiao; White, Lisa J; Xu, Pan; Yang, Yupeng; Yogurtcu, Osman N; Zhang, Weitong; Zhao, Yanting; Zou, Difan; Ferrari, Matthew J; Pannell, David; Tildesley, Michael J; Seifarth, Jack; Johnson, Elyse; Biggerstaff, Matthew; Johansson, Michael A; Slayton, Rachel B; Levander, John D; Stazer, Jeff; Kerr, Jessica; Runge, Michael C
Policymakers must make management decisions despite incomplete knowledge and conflicting model projections. Little guidance exists for the rapid, representative, and unbiased collection of policy-relevant scientific input from independent modeling teams. Integrating approaches from decision analysis, expert judgment, and model aggregation, we convened multiple modeling teams to evaluate COVID-19 reopening strategies for a mid-sized United States county early in the pandemic. Projections from seventeen distinct models were inconsistent in magnitude but highly consistent in ranking interventions. The 6-mo-ahead aggregate projections were well in line with observed outbreaks in mid-sized US counties. The aggregate results showed that up to half the population could be infected with full workplace reopening, while workplace restrictions reduced median cumulative infections by 82%. Rankings of interventions were consistent across public health objectives, but there was a strong trade-off between public health outcomes and duration of workplace closures, and no win-win intermediate reopening strategies were identified. Between-model variation was high; the aggregate results thus provide valuable risk quantification for decision making. This approach can be applied to the evaluation of management interventions in any setting where models are used to inform decision making. This case study demonstrated the utility of our approach and was one of several multimodel efforts that laid the groundwork for the COVID-19 Scenario Modeling Hub, which has provided multiple rounds of real-time scenario projections for situational awareness and decision making to the Centers for Disease Control and Prevention since December 2020.