Methodology

The composite Pandemic Danger Index (cPDI) is designed to estimate the danger of pandemic spread in a particular country or region, taking into account not only the current situation but future prospects as well. The Index enables an apples-to-apples comparison of the situation in different countries or regions, as well as at different points in the timeline of the pandemic.

The index uses two main input series – the numbers of new cases and new deaths  – and one auxillary input series, which is the number of new tests (when available). These inputs, together with current population data, are used to calculate three danger indices, which are then combined through a weighted average to create the composite index (cPDI).

Following is the full methodology used in the calculation. For simplicity, it is broken down into steps.

Step 1: Seven-day averages

In many countries, testing and reporting follows a weekly cycle, with some countries not reporting results on weekends at all and other showing a marked drop in both new cases and new deaths reported over the weekend. In order to avoid the ill effects of this cycle, all three data series are averaged over the last week, and these weekly averages are used in all calculations.

Step 2: Incidence Rate

In order to compare larger and smaller countries, we calculate a 7-day incidence rate, dividing the number of new cases in the past week by the country’s population (in units of 100,000) [1]. The 7-day incidence rate is displayed in the upper right corner of the dashboard, together with a column graph of new cases in the past month.

Step 3: Expected Mortality 

Expected Mortality is the most important metric which goes into the cPDI, as it embodies the risk for loss of life and the collapse of the health system. The use of expected mortality rather than the more naive incidence rate enables a meaningful comparison between countries where the Infection Fatality Rate (IFR) is different because of the median age or the status of the health system, as well as correcting for regions and periods in which the Case Fatality Rate (CFR) is or was much higher than the IFR because of insufficient testing [2]. The result not only predicts the actual mortality in the near future but can also serve as an estimate for the pressure on the country’s health care system, as in most cases, the number of patients in serious and critical condition correlates well with the number of deaths.

The first step in estimating expected mortality is calculating the current Mortality Rate (i. E. the CFR). This is done by dividing the weekly average of new deaths by the weekly average of new cases ten days earlier. The 10-day delay is inferred from current estimates of average time from infection to death (20-22 days), as well as the observed delay between infection and the inclusion of a case in the national statistics (10-12 days, including the 5-day incubation period, testing times and times for information transfer). [3]

Using the calculated mortality rate, we estimate the expected mortality in the next ten days by multiplying the (weekly-averaged) number of new cases in the past 10 days by the mortality rate. This raw expected mortality is shown in the second row of the right panel, along with the expected mortality per million (which is obtained by dividing the raw number by the population in millions) and the mortality rate. Note that these figures are for ten days, so in order to get the expected daily mortality you need to divide them by ten.

It is important to note that we have calculated the expected mortality values in selected countries for dates starting in March, and these correlate well with the true mortality which occurred in the next 10-day time period.

Step 4: The Expected Mortality Index (EMI)

In order to turn the “expected mortality per million”, which is an open-ended value, into an index with values  from 0 to 100, we used an asymptotic exponential function, according to the following formula:

EMI = 100(1-e^{M/\alpha })

Where M is the “expected mortality per million” and 𝛂 is a calibration parameter defining the expected mortality for which the index would produce the value of 63 (100-100/e). All current model parameters are listed in Appendix 1.

Step 5: The Reproduction Rate (R)

The reproduction rate, a cruciall value in epidemiological dynamics. represents the average number of people infected by each carrier of the virus during his/her infectious period. The basic reproduction number (R0) denotes the reproduction rate in an unprotected, unaware population (for COVID-19 it was estimated to be in the range of 2.8-3.8), while the reproduction rate is a dynamic value influenced by the makeup of the population, its level of connectivity and any measures taken to reduce contact and transmission. 

An exact calculation of R requires a full SEIR model and is beyond the scope of this work; however, because most infections take place in the 2-4 days around the onset of symptoms, we have found that a good estimate of R can be inferred by dividing the 7-day average of new cases by the one calculated one incubation period before (5 days in the case of COVID-19). In order to smooth the data and avoid major changes to R as a result of minor perturbations in case numbers, we have decided to calculate R over two incubation periods, by dividing the 7-day average of new cases by the one calculated ten days (two incubation periods) before and taking the square root of the result.

Step 6: The Reproduction Rate Index (RRI)

In order to turn the Reproduction Rate, which is an open-ended value, into an index with values from 0 to 100, as well as take into account the big difference between a reproduction rate which is less than 1, around 1 or more than 1, we used a different asymptotic exponential function, which uses the following formula:

RRI=100(1-Ke^{-\beta R})

Where R is the reproduction rate, and K and are calibration parameters chosen to make the index go to zero when R is less than 0.7 (i. E. the pandemic is under control), respond aggressively when R is close to 1 and flatten when R nears the value of R0 for COVID-19. All current model parameters are listed in Appendix 1.

Step 7 (added on Jan. 1st, 2021):  Calculating the RRI weight adjustment

While the reproduction rate is a valuable metric for determining the pace of pandemic spread, it tends to be erratic and inaccurate when the number of new cases (i. e.  the incidence rate) is relatively low. Moreover, when incidence is low, a high reproduction rate may be the result of a local outbreak, which does not really pose danger to the country as a whole.

In order to balance this tendency and avoid major shifts in the cPDI as a result of local outbreaks during relatively calm periods, we adjust the weight give to the RRI in the calculation of the main index using the following formula:

A_{adj}=A_0+(1-A_0)(1-e^{-IR/δ})

Where IR is the incidence rate, A0 is the minimum weight afforded to this index and δ is a calibration parameter which defines the incidence rate for which the parameter would be A0+0.63*(1-A0). This weight adjustment is then applied when weighing the RRI in step 9 (see below). All current model parameters are listed in Appendix 1.

Step 8 (optional):  The Test Positivity Index (TPI)

Test Positivity (the percentage of tests which are positive for COVID-19) has long been used as a metric to estimate the effect of unknown infections, which have ‘slipped through the radar’ of the testing system. We calculate the test positivity rate by dividing the 7-day average for new cases by the one for new tests, if this data is available.

In order to turn the test positivity rate into an index with values from 0 to 100 and decrease the effect of extreme values (such as ones observed in Latin America) we again used an asymptotic exponential function:

TPI=100(1-e^{-P / \gamma })

Where P is the positivity rate and is a calibration parameter which defines the expected mortality for which the index would produce the value of 63 (100-100/e). All current model parameters are listed in Appendix 1. 

Step 9: Putting it all together

Last but not least, we combine the three indices into the composite Pandemic Dange Index (cPDI) by using a weighted average:

cPDI = \frac{A_1\sdot EMI + A_2\sdot A_{adj}\sdot RRI + A_3\sdot TPI}{\ A_1 +A_2\sdot A_{adj} + A_3}

The weights have been chosen to reflect the relative importance of the three indices and calibrated by using data from various countries since March. 

When no testing information is available, and TPI is not calculated, The weight of TPI is distributed between the two other indices in a roughly 80-20 split, with more weight given to the EMI. This was done because the EMI encapsulates the measure of inadequate testing (see step 3 above) and thus is better suited to serve as an alternative to TPI when it is not available. The exact weight was chosen by comparing the index calculated with and without testing data for selected countries where it is available, and choosing the weight to produce the best fit. All current model parameters are listed in Appendix 1. 

Step 10: Figuring out the trend

The dashboard uses a relatively simple formula to decide how, if at all, the index has changed recently. We compare the index value to the one from three days ago, and check whether it had increased by 2 units or more, decreased by 2 units or changed by less than 2 units over this period.  The result is marked by a colored arrow or icon next to the index value.

Notes

[1] This specific measure of incidence is used because it is the metric utilized by Germany’s Robert Koch Institute, as well as many other European bodies, to define risk areas in Europe and elsewhere (a value above 50 denotes a risk area)

[2] for a more detailed explanation of IFR and CFR, see Estimating mortality from COVID-19, a scientific brief from the WHO.

[3] This delay period may vary between countries and periods, but the model is not very sensitive to slight changes in it.

Appendix 1: Current model and dashboard parameters

Fatality delay: 10 days

α (EMI parameter): 40 (as of 10-Nov-202. see change log below)

β (RRI parameter): 1.68

K (RRI parameter): 3.34

γ (TPI parameter): 8% (0.08)

δ (RRI weight adjustment): 20

A₀ (Minimum RRI weight adjustment): 0.3

A₁ (EMI weight): 1.8

A₂ (RRI weight): 1

A₃ (TPI weight): 1.2

A_1a (EMI weight when no TPI exists): 2.8

A_2a (RRI weight when no TPI exists): 1.2

Appendix 2: Sources

Main data source: https://ourworldindata.org/

Stringency Index based on https://www.bsg.ox.ac.uk/research/research-projects/coronavirus-government-response-tracker

Appendix 3: Change Log

Nov. 10th, 2020: α parameter was raised from 30 to 40 in order to differentiate better between states with very high expected mortality, such as had been observed in late October 2020 in Belgium and the Czech Republic.  As a result, most states lost a few points on the cPDI (Germany, for instance, goes from 54 to 48).

Jan. 1st, 2021: Added the RRI weight adjustment (see above for details)

Disclaimer

We are a team of mathematicians and data analysts, none of whom is an infectious disease expert or epidemiologist. The data is sourced from the website Our World in Data (see Data Sources below), and the cases represented are only confirmed cases, which may be limited by test availability. This dashboard should not be used for medical decision making; you can find updated guidelines and recommendations in websites and documents published by the Department of Health or other health authorities in your place of residence.

Team

Ronen Altman Kaydar

Modeling and Math Framework

Ronen completed a B.Sc. in Mathematics and Physics and an M,A. in Philosophy and History of Science, both from Tel Aviv University. He’s developed mathematical algorithms for the Israeli Armament Development Authority and today is a freelance writer, editor and tour guide based in Berlin, Germany

Website: www.yourberlintour.com.
Email: ronenakaydar@gmail.com

Evgeni Hasin

Development & Data Integration

Evgeni is a professional Business Intelligence expert, with over 15 years experience with Data Visualization & Analytics. Evgeni has a BA in Business Administration, and lives in Berlin, Germany.

Website: https://www.linkedin.com/in/evgenihasin.
Email: ehasin@idearig.com

Roie Shalom

Design

Roie is a Senior Creative UX/UI Designer with first-day-at-the-job passion, inspite years of experience. Roie is highly proficient with Web and Mobile research-based experience design, specializing in development of user-focused digital design products

Website: https://www.linkedin.com/in/roiesh.
Email: roiesh@gmail.com

Visit Ronen’s  Germany-Israel Corona Updates Facebook page (HEB), which has become a valuable source for the community.