Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.

Cramer, Estee Y

Ray, Evan L

Lopez, Velma K

Bracher, Johannes

Brennen, Andrea

Castro Rivadeneira, Alvaro J

Gerding, Aaron

Gneiting, Tilmann

House, Katie H

Huang, Yuxin

Jayawardena, Dasuni

Kanji, Abdul H

Khandelwal, Ayush

Le, Khoa

Mühlemann, Anja

Niemi, Jarad

Shah, Apurv

Stark, Ariane

Wang, Yijin

Wattanachit, Nutcha

Zorn, Martha W

Gu, Youyang

Jain, Sansiddh

Bannur, Nayana

Deva, Ayush

Kulkarni, Mihir

Merugu, Srujana

Raval, Alpan

Shingi, Siddhant

Tiwari, Avtansh

White, Jerome

Abernethy, Neil F

Woody, Spencer

Dahan, Maytal

Fox, Spencer

Gaither, Kelly

Lachmann, Michael

Meyers, Lauren Ancel

Scott, James G

Tec, Mauricio

Srivastava, Ajitesh

George, Glover E

Cegan, Jeffrey C

Dettwiller, Ian D

England, William P

Farthing, Matthew W

Hunter, Robert H

Lafferty, Brandon

Linkov, Igor

Mayo, Michael L

Parno, Matthew D

Rowland, Michael A

Trump, Benjamin D

Zhang-James, Yanli

Chen, Samuel

Faraone, Stephen V

Hess, Jonathan

Morley, Christopher P

Salekin, Asif

Wang, Dongliang

Corsetti, Sabrina M

Baer, Thomas M

Eisenberg, Marisa C

Falb, Karl

Huang, Yitao

Martin, Emily T

McCauley, Ella

Myers, Robert L

Schwarz, Tom

Sheldon, Daniel

Gibson, Graham Casey

Yu, Rose

Gao, Liyao

Ma, Yian

Wu, Dongxia

Yan, Xifeng

Jin, Xiaoyong

Wang, Yu-Xiang

Chen, YangQuan

Guo, Lihong

Zhao, Yanting

Gu, Quanquan

Chen, Jinghui

Wang, Lingxiao

Xu, Pan

Zhang, Weitong

Zou, Difan

Biegel, Hannah

Lega, Joceline

McConnell, Steve

Nagraj, VP

Guertin, Stephanie L

Hulme-Lowe, Christopher

Turner, Stephen D

Shi, Yunfeng

Ban, Xuegang

Walraven, Robert

Hong, Qi-Jun

Kong, Stanley

van de Walle, Axel

Turtle, James A

Ben-Nun, Michal

Riley, Steven

Riley, Pete

Koyluoglu, Ugur

DesRoches, David

Forli, Pedro

Hamory, Bruce

Kyriakides, Christina

Leis, Helen

Milliken, John

Moloney, Michael

Morgan, James

Nirgudkar, Ninad

Ozcan, Gokce

Piwonka, Noah

Ravi, Matt

Schrader, Chris

Shakhnovich, Elizabeth

Siegel, Daniel

Spatz, Ryan

Stiefeling, Chris

Wilkinson, Barrie

Wong, Alexander

Cavany, Sean

España, Guido

Moore, Sean

Oidtman, Rachel

Perkins, Alex

Kraus, David

Kraus, Andrea

Gao, Zhifeng

Bian, Jiang

Cao, Wei

Lavista Ferres, Juan

Li, Chaozhuo

Liu, Tie-Yan

Xie, Xing

Zhang, Shun

Zheng, Shun

Vespignani, Alessandro

Chinazzi, Matteo

Davis, Jessica T

Mu, Kunpeng

Pastore Y Piontti, Ana

Xiong, Xinyue

Zheng, Andrew

Baek, Jackie

Farias, Vivek

Georgescu, Andreea

Levi, Retsef

Sinha, Deeksha

Wilde, Joshua

Perakis, Georgia

Bennouna, Mohammed Amine

Nze-Ndong, David

Singhvi, Divya

Spantidakis, Ioannis

Thayaparan, Leann

Tsiourvas, Asterios

Sarker, Arnab

Jadbabaie, Ali

Shah, Devavrat

Della Penna, Nicolas

Celi, Leo A

Sundar, Saketh

Wolfinger, Russ

Osthus, Dave

Castro, Lauren

Fairchild, Geoffrey

Michaud, Isaac

Karlen, Dean

Kinsey, Matt

Mullany, Luke C

Rainwater-Lovett, Kaitlin

Shin, Lauren

Tallaksen, Katharine

Wilson, Shelby

Lee, Elizabeth C

Dent, Juan

Grantz, Kyra H

Hill, Alison L

Kaminsky, Joshua

Kaminsky, Kathryn

Keegan, Lindsay T

Lauer, Stephen A

Lemaitre, Joseph C

Lessler, Justin

Meredith, Hannah R

Perez-Saez, Javier

Shah, Sam

Smith, Claire P

Truelove, Shaun A

Wills, Josh

Marshall, Maximilian

Gardner, Lauren

Nixon, Kristen

Burant, John C

Wang, Lily

Gao, Lei

Gu, Zhiling

Kim, Myungjin

Li, Xinyi

Wang, Guannan

Wang, Yueying

Yu, Shan

Reiner, Robert C

Barber, Ryan

Gakidou, Emmanuela

Hay, Simon I

Lim, Steve

Murray, Chris

Pigott, David

Gurung, Heidi L

Baccam, Prasith

Stage, Steven A

Suchoski, Bradley T

Prakash, B Aditya

Adhikari, Bijaya

Cui, Jiaming

Rodríguez, Alexander

Tabassum, Anika

Xie, Jiajia

Keskinocak, Pinar

Asplund, John

Baxter, Arden

Oruc, Buse Eylul

Serban, Nicoleta

Arik, Sercan O

Dusenberry, Mike

Epshteyn, Arkady

Kanal, Elli

Le, Long T

Li, Chun-Liang

Pfister, Tomas

Sava, Dario

Sinha, Rajarishi

Tsai, Thomas

Yoder, Nate

Yoon, Jinsung

Zhang, Leyou

Abbott, Sam

Bosse, Nikos I

Funk, Sebastian

Hellewell, Joel

Meakin, Sophie R

Sherratt, Katharine

Zhou, Mingyuan

Kalantari, Rahi

Yamana, Teresa K

Pei, Sen

Shaman, Jeffrey

Li, Michael L

Bertsimas, Dimitris

Skali Lami, Omar

Soni, Saksham

Tazi Bouardi, Hamza

Ayer, Turgay

Adee, Madeline

Chhatwal, Jagpreet

Dalgic, Ozden O

Ladd, Mary A

Linas, Benjamin P

Mueller, Peter

Xiao, Jade

Wang, Yuanjia

Wang, Qinxia

Xie, Shanghong

Zeng, Donglin

Green, Alden

Bien, Jacob

Brooks, Logan

Hu, Addison J

Jahja, Maria

McDonald, Daniel

Narasimhan, Balasubramanian

Politsch, Collin

Rajanala, Samyak

Rumack, Aaron

Simon, Noah

Tibshirani, Ryan J

Tibshirani, Rob

Ventura, Valerie

Wasserman, Larry

O'Dea, Eamon B

Drake, John M

Pagano, Robert

Tran, Quoc T

Ho, Lam Si Tung

Huynh, Huong

Walker, Jo W

Slayton, Rachel B

Johansson, Michael A

Biggerstaff, Matthew

Reich, Nicholas G

2023-01-23T20:50:36Z

2023-01-23T20:50:36Z

2022-04

2023-01-23T20:50:30Z

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks.

0027-8424

1091-6490

https://hdl.handle.net/10161/26494

eng

Proceedings of the National Academy of Sciences

Proceedings of the National Academy of Sciences of the United States of America

10.1073/pnas.2113561119

Humans

Probability

Public Health

Forecasting

United States

Pandemics

Data Accuracy

COVID-19

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.

Journal article

Xu, Pan|0000-0002-2559-8622

e2113561119

15

Duke

School of Medicine

Basic Science Departments

Biostatistics & Bioinformatics

Published

119