Johns Hopkins University Study Found Lockdowns Did Little to Reduce COVID Mortality

Co-published on Publish0x.

Introduction

In early 2020, many nations around the world implemented lockdowns and shelters-in-place in response to the rising cases of SARS-COV-2. Originally portrayed as a short term measure (i.e. "2 weeks to flatten the curve"), the lockdowns continues for several months.

The effects of the lockdowns were pretty severe. As millions of people were isolated in their homes, they could not work or interact with others in person. Small businesses, particularly those deemed "non-essential", were forced to layoff their employees or worse, shut down indefinitely due to the lack of foot traffic and customers.

Throughout the lockdown period, negative health outcomes rose significantly. For example, from May 2020 to April 2021, drug overdose deaths rose by 28.5% compared to the previous 12-month period in the US according to the CDC. A report from the Horace Mann revealed that in 2020, 53% of American educators said their students suffered from "significant loss of [academic] learning" with another 44% of educators stating their students suffered from "some loss of [academic] learning". Nearly the exact same percentages of teachers said the same thing regarding social-emotion learning.

These outcomes among others have sparked debate over whether the lockdowns should end. Some states wanted to open everything up again in order to restore economic growth and return to normalcy. Other states disagreed as they feared doing so would spike COVID cases and deaths. The debate became highly political. For example, in March 2021, Texas was criticized for being "anti-science" for lifting its mask mandate and capacity restrictions, though cases and deaths actually fell.

It begs the question, do lockdowns actually work at reducing COVID mortality? If so, to what degree? Just last month, the The Johns Hopkins Institute for Applied Economics, Global Health, and the Study of Business Enterprise published a meta-analysis study that observed the overall effect of lockdowns, shelter-in-place orders (SIPOs) and non-pharmaceutical interventions (NPIs).

What's a Meta-Analysis?

Quite simply, a meta-analysis combines the results of many smaller studies into one big study for the purpose of increasing statistical power. The amount of time and resources it would take to perform one large study tend to be beyond what researchers can handle hence why meta-analyses come in handy.

There will inherently be heterogeneity between studies for a variety of reasons. Factors such as different sampling methods, study designs, statistical methods, authors' interpretation of results, or even random chance will introduce heterogeneity.

As a result, the study collection process needs to be well thought-out. The search technique should minimize publication bias; the selected studies should use a valid design (e.g. RCT) and proper sampling techniques (e.g. blinding); and reviewers should review each study blindly to avoid potential bias.

Once all the studies are gathered, you combine the results. Most commonly, researchers use the random effects model to weight each study differently. They usually do this by giving each study a validity score which depends on factors such as blinding, degree of bias, quality of measures, and reputation of researchers/journals. This is important as this addresses the heterogeneity I explained earlier.

In the Johns Hopkins study, the authors used four quality dimensions to partially determine the weight of each study: peer-reviewed vs. working papers, long vs. short study period, no early effect on mortality, and social sciences vs. other sciences (p. 24-25). A few of the quality dimensions are self-explanatory. Peer-reviewed papers are more valuable than working papers since the former has already gone through the review process. Studies with longer periods are less prone to high variability than those with short period. Studies that show no early effect on mortality are weighed more because "on average, it takes approximately three weeks from infection to death" (p. 25).

The paper's reasoning for the last quality dimension is interesting, stating that "while it is true that epidemiologists and researchers in natural sciences should, in principle, know much more about COVID-19 and how it spreads than social scientists, social scientists are, in principle, experts in evaluating the effect of various policy interventions" (p. 25).

Quite simply, the more quality dimensions a study meets, the more weight it will have on the meta-analysis.

The Johns Hopkins Study

The authors Jonas Herby, Lars Jonung, and Steve Hanke analyzed the effects of lockdowns, shelter-in-place orders, and non-pharmaceutical interventions on COVID mortality. They did not look at the effects of the three independent variables on aspects such as hospitalization rates or cases per capita.

One of the main motivations for the authors to publish this study was when they analyzed the COVID mortality rates in the US and Europe against the stringency index (i.e. the higher the number, the stricter the lockdown) using Our World in Data data. In both regions, there was an extremely slight positive correlation. In other words, the stricter the lockdown, the marginally higher the mortality rate.

For the lockdown meta-analysis, the authors narrowed down to seven studies. Five of the seven showed less than a 5% reduction in deaths whereas one concluded a 9% reduction and another showing a large 35.3% reduction. The studies that showed the largest reductions also weighed the least because they only met 2 of the 4 quality dimensions. In contrast, a study like Chisadza et al. (2021) which found a 0.1% increase in mortality had more weight because it met all criteria.

Overall, the precision-weighted average came out at a 0.2% reduction in mortality.

In their shelter-in-place order analysis, the authors found similar results whether they looked at SIPOs alone or SIPOs in conjunction with other measures. For the former, across six studies, all but one showed a reduction in mortality. Half of them yielded over a 30% reduction in mortality, one study showed a 6% reduction, and the rest showed about a 1% effect in either direction. Taking into account the quality dimensions and weighting, the precision-weighted average was a 5.1% reduction in mortality.

For the latter category, the authors looked at 7 studies. Two of them concluded a 0% effect though they did not provide a standard error and hence, they were not weighed. Four studies found an increase in mortality to varying degrees and one found a decrease in mortality. As a result, the precision-weighted average for SIPOS in conjunction with other interventions came out as a 2.8% increase. Combining all SIPO studies, the precision-weighted average equaled to a 2.9% reduction.

When looking at non-pharmaceutical interventions, most of the results were unremarkable. The overall effects of lockdowns (complete/partial), border closures, school closures, and limiting gatherings were negligible. Facemasks seemingly led to a large reduction in mortality (-21.2%), but the meta-analysis only looked at two studies, one of which implemented an employee facemask mandate. The authors also cited Liu et al. (2021) who said that “fourteen of sixteen identified randomized controlled trials comparing face masks to no mask controls failed to find statistically significant benefit in the intent-to-treat populations".

Closing restaurants yielded a 10.6% reduction in mortality. However, the authors warned against establishing a causal relationship as they argued that " even if lockdowns are successful in initially reducing the spread of COVID-19, the behavioral response may counteract the effect completely, as people respond to the lower risk by changing behavior" and "if closing bars and restaurants causes the prevalence of the disease to fall toward zero, the demand for costly disease prevention efforts like social distancing and increased focus on hygiene also falls towards zero, and the disease will return".

In conclusion, the authors' meta-analysis found that lockdowns had little to no effect on public health outcomes whereas they imposed substantial economic and social costs (though they did not analyze these aspects in their study).

My Closing Thoughts

Hindsight is 20/20, but it is important that we look back at these events, and evaluate what worked and what did not. While the lockdowns might have reduced COVID mortality rates, the overall effect was negligible. During that period, negative outcomes such as lost learning, drug overdose deaths, and suicide rates increased at a higher magnitude.

What we need to ask ourselves is if another pandemic hits the world, what can we do differently that may lead to better outcomes? While I and everyone else do not have the answers, we do have some hints. For instance, it turned out it was not a great idea to expose high risk populations to patients with COVID. This was the case with the nursing home scandals in states such as New York and Michigan.

There may also be a strong correlation between nutrition and COVID health outcomes. The CDC cites obesity as a big risk factor for COVID hospitalization and death. I wrote an article on how over-consumption of sugar can increase inflammation. In addition, there are meta-analyses that suggest nutrients such as vitamin A, vitamin D, and zinc provide better outcomes from SARS-COV-2. Could having a more balanced diet with fresh fruits and vegetables (especially leafy greens) mitigate symptoms? Maybe.

While I obviously do not have the answers, one thing is for sure is that the definition of insanity is to doing the same thing over and over and expecting different results.