To the editor

In a recent and interesting study, Pietrzak et al1 have concluded that the results of their research confirm high effectiveness of the BNT162b2 vaccine in preventing COVID-19–associated deaths in Poland in a real-world setting. This result is seemingly in accordance with a widespread belief that vaccination is a major contributor to the reduction of SARS-CoV-2–related mortality. However, while reading this nonrandomized study assessing COVID-19 vaccine effectiveness, one should consider multiple factors that may flaw the estimates due to several types of bias.2 Therefore, I would like the authors to provide more information about their results and ask them to present additional analyses.

It was not clearly stated in the paper which definition of COVID-19–associated death was used. If COVID-19 deaths were defined as occurring when the presence of SARS-CoV-2 infection was confirmed with the real-time polymerase chain reaction (PCR) assay, then there was a risk of a bias arising from misclassification of COVID-19 cases due to the patient vaccination status. Vaccinated patients in Poland had a lower chance of having their death attributed to COVID-19 than to other causes, as the COVID-19 vaccines were perceived as very effective. In contrast, unvaccinated people may have had their death attributed to COVID-19 more easily, especially because the unvaccinated group was more frequently tested. During the pandemic, the unvaccinated patients frequently needed a negative COVID-19 PCR test prior to any planned procedure or emergency intervention.3 Hence, I would like to ask the authors to present the results on the BNT162b2 vaccine efficiency separately with regard to in-hospital and out-of-hospital deaths.

Another type of bias possibly affecting this study is immortal time bias. In observational studies, immortal time refers to the period between the time point when patients enter the study cohort (in this study, upon administration of the first vaccine dose) and the point when they are considered to be exposed to the examined treatment (in this study, upon becoming fully vaccinated). The patients were considered fully vaccinated 2 weeks after administration of the second dose of a 2-dose vaccine or 2 weeks after administration of a single-dose vaccine. During the period between the first dose and achieving the fully vaccinated status, death could not occur in the treatment group, as those patients had to, by design, survive long enough to receive the treatment. Thus, the patients who survive up to 2 weeks after the second dose of a 2-dose vaccine (or 2 weeks after administration of a single-dose vaccine) are considered “immortal.”4 Any death that occurs within this immortal time can only be attributed to the unvaccinated group. This partially vaccinated group was excluded by the authors. For this reason, I would like the authors to provide the results of an additional risk comparison between the individuals at least 2 weeks postvaccination with the BNT162b2 vaccine and those who received the first dose of the BNT162b2 vaccine but were not fully vaccinated.

The rationale for this request is that immortal time bias can be avoided by ensuring that all time intervals during which the study participants may experience the outcome of interest are captured and accounted for in the analysis, including the time before achieving the fully vaccinated status.5 Some people skip the second dose of the vaccine due to unwanted side effects or simply die after the first dose and before they are considered fully vaccinated. As in general healthier people participate in vaccination programs, this type of comparison would also help to reduce healthy vaccinee bias due to confounding. The people vaccinated with 1 dose were probably more similar to those who received 2 doses than to the unvaccinated individuals, so this comparison could potentially minimize confounding due to behavioral and socioeconomic differences between the vaccinated and unvaccinated people.

Randomized placebo-controlled trials are considered the only method for providing reliable evidence on the efficacy of vaccines at the time of their approval. None of the trials on COVID-19 vaccines were designed to detect a reduction in any serious outcome, such as hospitalizations or death from COVID-19, because the sample sizes were too small. Important remaining questions about the effectiveness of vaccines regarding mortality can only be answered using data from observational studies.6 Confounding and other types of bias remain the major limitations of observational studies. It is critical that evaluations of the COVID-19 vaccines account for confounding factors as much as possible.