The Vaccine Adverse Event Reporting System (VAERS): An In-Depth Analysis of its Role, Mechanics, Limitations, and Methodological Interpretation in Post-Market Vaccine Safety Surveillance

Many thanks to our sponsor Esdebe who helped us prepare this research report.

Abstract

The Vaccine Adverse Event Reporting System (VAERS) stands as a foundational pillar within the United States’ comprehensive vaccine safety surveillance infrastructure. Jointly administered by the Centers for Disease Control and Prevention (CDC) and the Food and Drug Administration (FDA), VAERS was established in 1990 as a critical passive surveillance mechanism. Its explicit mandate is to systematically collect and analyze reports of adverse health events that occur temporally after vaccination. This exhaustive report provides a granular examination of VAERS, dissecting its historical genesis, explicit purpose, intricate operational mechanics, inherent methodological limitations, and the rigorous frameworks employed for data analysis and interpretation. A paramount emphasis is placed on articulating the fundamental distinction between raw, unverified reported data and scientifically validated conclusions derived from comprehensive epidemiological investigation. This distinction is vital for fostering accurate understanding, informing evidence-based public health discourse, and upholding public confidence in immunization programs, thereby mitigating the pervasive risks of misinterpretation in vaccine safety discussions.

Many thanks to our sponsor Esdebe who helped us prepare this research report.

1. Introduction

Vaccination represents one of humanity’s most profound and successful public health achievements, having demonstrably eradicated smallpox, brought poliomyelitis to the brink of global elimination, and dramatically reduced the incidence and severity of numerous other infectious diseases, thereby averting countless cases of morbidity and mortality worldwide (Andre et al., 2008). The sustained success of these vital public health interventions hinges critically upon public trust, which is inextricably linked to the demonstrable safety of vaccines. Consequently, robust and transparent systems for monitoring vaccine safety are not merely beneficial but absolutely indispensable to maintaining immunization coverage and societal well-being. The Vaccine Adverse Event Reporting System (VAERS) occupies a central, albeit frequently misunderstood, position within this intricate ecosystem of post-market vaccine safety monitoring in the United States. Its primary function is to serve as an early warning system, meticulously tracking and evaluating adverse health events that manifest following vaccination.

However, the utility and interpretability of VAERS data are often subject to profound misconstrual, giving rise to pervasive misinterpretations that can inadvertently erode public confidence, fuel misinformation, and exert undue influence on public perception and policy formulation. Such misinterpretations frequently stem from a lack of understanding regarding the system’s design, its inherent operational parameters, and the rigorous scientific methodologies required to transform raw data into actionable, evidence-based conclusions. The historical imperative for such a system emerged from a confluence of factors, including escalating litigation against vaccine manufacturers in the 1970s and 1980s, which threatened vaccine supply and public health (Evans, 1992). This report endeavors to furnish a comprehensive and nuanced analysis of VAERS, systematically elucidating its foundational purpose, detailing its intricate operational framework, critically examining its inherent methodological limitations, and delineating the scientifically rigorous approaches that are absolutely essential for its appropriate data interpretation. By providing this detailed exposition, the aim is to empower stakeholders—from healthcare professionals and policymakers to the general public—with a deeper, more accurate understanding of VAERS, thereby facilitating informed decision-making and fostering a more constructive discourse surrounding vaccine safety.

Many thanks to our sponsor Esdebe who helped us prepare this research report.

2. Purpose of VAERS

VAERS was not conceived in a vacuum; its establishment was a direct legislative mandate stemming from the landmark National Childhood Vaccine Injury Act (NCVIA) of 1986. This pivotal legislation was enacted in response to a growing public health crisis where increasing numbers of lawsuits against vaccine manufacturers for alleged vaccine injuries threatened to destabilize vaccine supply and escalate costs, thereby jeopardizing established immunization programs (National Vaccine Program Office, 2000). The NCVIA sought to address this crisis by creating a federal no-fault compensation program for individuals potentially injured by vaccines, known as the National Vaccine Injury Compensation Program (VICP), and concurrently by mandating the creation of a national vaccine safety surveillance system to systematically monitor and evaluate adverse events occurring post-vaccination. This dual approach aimed to ensure both vaccine availability and public confidence through comprehensive safety monitoring and a fair compensation mechanism.

The primary objectives of VAERS are multifaceted and interconnected, designed to provide a broad safety net for vaccines once they are introduced into widespread public use:

• Detection of Potential Adverse Events: At its core, VAERS functions as a highly sensitive system for identifying the broadest possible spectrum of potential adverse events that occur after vaccination. This includes not only commonly recognized reactions but crucially, also rare or entirely unexpected events that might not have been apparent during pre-licensure clinical trials due to sample size limitations. The passive nature of VAERS allows it to capture a wide array of health conditions reported across diverse populations, serving as an initial alert mechanism for any health issue temporally associated with vaccination (Shimabukuro et al., 2015). This broad net is crucial for catching signals that might otherwise be missed by more targeted, active surveillance systems.

• Signal Detection: Beyond merely identifying individual events, a critical objective of VAERS is to recognize patterns or ‘safety signals’ that may indicate a potential causal relationship between a vaccine and a particular adverse event. A safety signal is defined as reported information on a possible causal relationship between an adverse event and a vaccine, the relationship being unknown or incompletely documented previously (WHO, 2002). This involves identifying disproportionate reporting of specific adverse events following the administration of a particular vaccine, compared to other vaccines or compared to the expected background rate of such events in the general population. The early identification of such signals triggers further, more rigorous scientific investigation.

• Public Health Surveillance: VAERS plays an indispensable role in the ongoing post-market surveillance of vaccine safety in the general population. While pre-licensure clinical trials provide robust data on efficacy and common adverse reactions, they are typically conducted on selected populations and with sample sizes that may be insufficient to detect very rare adverse events (occurring in fewer than 1 in 10,000 or 1 in 100,000 vaccinations). VAERS complements these trials by monitoring vaccines in real-world settings across millions of individuals, including diverse age groups, those with underlying health conditions, and individuals receiving multiple vaccines concurrently. This continuous monitoring informs public health decisions and recommendations, ensuring that the benefit-risk profile of vaccines remains favorable over time (CDC, 2021a).

• Regulatory Oversight and Action: The data collected by VAERS is an essential input for regulatory agencies, primarily the FDA and CDC. This information is meticulously reviewed to assess the evolving risk-benefit ratio of licensed vaccines. VAERS data, especially when corroborated by other safety monitoring systems, can inform a range of regulatory actions. These may include updating vaccine package inserts with new warnings or precautions, issuing public health advisories, modifying vaccine recommendations (e.g., changes to age groups, contraindications), or, in extremely rare instances, even initiating the process for vaccine withdrawal from the market if a serious and unequivocally proven risk is found to outweigh the benefits (FDA, 2021a). This oversight ensures that vaccines remain safe and effective throughout their lifecycle.

• Hypothesis Generation for Further Research: VAERS data, while not conclusive on its own, serves as a rich source for generating hypotheses regarding potential vaccine-adverse event associations. When a signal is detected, it does not confirm causality but rather flags an area for focused scientific inquiry. Researchers can then design and execute targeted epidemiological studies (e.g., cohort studies, case-control studies) using more robust data sources to investigate these hypotheses thoroughly. This iterative process of signal generation and rigorous investigation is fundamental to advancing vaccine safety science.

Many thanks to our sponsor Esdebe who helped us prepare this research report.

3. Operational Mechanics of VAERS

VAERS functions as a cornerstone of the United States’ vaccine safety infrastructure, operating primarily as a passive surveillance system. This fundamental characteristic implies that the system relies on the voluntary submission of reports by various stakeholders rather than actively seeking out adverse events through systematic screening or direct patient follow-up. While this passive approach offers unparalleled breadth in data collection, it also introduces specific operational dynamics and inherent limitations that are crucial to comprehend (Shimabukuro et al., 2015). The operational process can be segmented into distinct, yet interconnected, phases:

3.1. Report Submission

The initiation of the VAERS process resides with the act of reporting. The system is designed to be accessible to a wide array of individuals, fostering comprehensive data collection. Anyone can submit a report to VAERS, facilitating a broad capture of potential events:

• Who Can Report:

  • Healthcare Providers: Physicians, nurses, pharmacists, and other medical professionals are strongly encouraged to report any clinically significant adverse event that occurs after vaccination, even if they are unsure whether the vaccine caused the event. For certain serious events, or those listed in the vaccine manufacturer’s package insert as a contraindication or warning, reporting by healthcare providers may be mandated by law or regulation. For example, the NCVIA specifically mandates healthcare providers to report specific adverse events following certain childhood vaccinations (NCVIA, 1986).
  • Vaccine Manufacturers: Manufacturers of vaccines licensed in the United States are legally required to report all adverse events that come to their attention to VAERS. This includes events from clinical trials, post-market surveillance, and even reports received directly from patients or healthcare providers. This mandatory reporting by manufacturers ensures that a significant volume of data, often with detailed clinical information, is captured.
  • Patients, Parents, or Caregivers: Members of the general public can submit reports. This direct reporting pathway is crucial for capturing patient perspectives and events that might not be reported by healthcare providers, particularly less severe or delayed reactions, or instances where a patient might have sought care outside their primary medical system.

• How to Report: Reports can be submitted through several channels to maximize accessibility:

  • Online Portal: The primary and most common method is via the secure VAERS website (VAERS.HHS.gov), which provides an intuitive online form for detailed information submission.
  • Mail or Fax: Printable forms are available on the VAERS website and can be completed and sent via postal mail or fax.
  • Phone: While less common for initial detailed submissions, the VAERS hotline can be used for inquiries and guidance on reporting.

• Information Requested in a VAERS Report: A comprehensive report aims to gather crucial details, typically including:

  • Patient Demographics: Age, sex, race/ethnicity, date of vaccination, medical history (e.g., allergies, chronic conditions, current medications), and other relevant patient characteristics.
  • Vaccine Information: Product name, manufacturer, lot number, administration site, route, dose, and any co-administered vaccines.
  • Adverse Event Details: Description of the event(s), onset date and time, duration, clinical course, treatments received, outcome (e.g., recovered, recovering, hospitalization, life-threatening, death), and any relevant laboratory or diagnostic findings.
  • Reporter Information: Type of reporter (e.g., healthcare professional, consumer, manufacturer), contact details (for follow-up), and their assessment of causality (though this is not considered definitive by VAERS).

The completeness and accuracy of this submitted information are paramount for subsequent analysis. Reports lacking essential details often necessitate follow-up by VAERS staff to gather missing clinical or vaccine-specific data.

3.2. Data Collection and Processing

Upon submission, reports enter a centralized data processing pipeline jointly managed by the CDC and FDA. This phase involves several critical steps to prepare the data for analysis:

• Initial Review and Triage: Each incoming report undergoes an initial review for completeness and clarity. Reports describing serious adverse events (e.g., hospitalization, life-threatening illness, permanent disability, death) are triaged for expedited review and potential follow-up. VAERS staff may contact the reporter or healthcare providers to obtain additional medical records or clarification.
• Data Entry and Coding: Information from reports is meticulously entered into the VAERS database. Medical events described in free-text fields are then coded using a standardized medical terminology system, primarily the Medical Dictionary for Regulatory Activities (MedDRA). MedDRA provides a comprehensive hierarchy of terms for classifying adverse drug and vaccine reactions, ensuring consistency in data categorization across different reports and facilitating systematic analysis (MedDRA, 2021). This coding process is vital for aggregating similar events and identifying potential patterns.
• Duplicate Checking: Rigorous checks are performed to identify and merge duplicate reports. It is not uncommon for multiple individuals (e.g., patient, physician, manufacturer) to report the same event, and VAERS employs sophisticated algorithms and manual review to ensure that each unique adverse event is represented only once in the primary analysis dataset.

3.3. Data Analysis

The collected and processed data subsequently undergoes preliminary analysis, designed primarily to detect potential safety signals or trends rather than to establish causality:

• Preliminary Screening: Automated tools and statistical algorithms continuously screen the incoming data for unusual patterns. This often involves comparing the observed frequency of specific adverse events after a particular vaccine to the expected background rate of those events in the unvaccinated population or to the frequency after other vaccines (see Section 5 for detailed methodologies). This initial screening is highly sensitive, aiming to detect any unusual increases or clusters of events.
• Clinical Review of Serious Reports: All reports describing serious outcomes (e.g., death, life-threatening events, hospitalizations) are individually reviewed by a team of medical officers, epidemiologists, and clinical specialists from the FDA and CDC. This in-depth clinical review assesses the medical plausibility of the reported event, considers the patient’s medical history, and evaluates whether additional information or follow-up is warranted. For fatalities, medical records (e.g., autopsy reports, hospital discharge summaries) are often actively sought and reviewed to determine the most likely cause of death.
• Descriptive Epidemiology: Basic descriptive analyses are routinely performed, including calculating frequencies of reported events by vaccine, age group, sex, and time since vaccination. While these descriptive statistics cannot yield incidence rates due to the lack of denominator data, they can highlight temporal associations and demographic concentrations.

3.4. Signal Evaluation and Validation

The identification of a ‘safety signal’ in VAERS data is merely the starting point for a more extensive and rigorous investigation. A signal, by definition, suggests a potential association, but it does not equate to confirmed causality. The subsequent phase involves a multi-pronged approach to evaluate and, if necessary, validate the detected signal:

• Further Investigation: Once a signal is identified, the FDA and CDC initiate further investigation. This can involve reviewing all available clinical documentation associated with the reported events, consulting external medical experts, and conducting targeted literature reviews to assess existing scientific knowledge about the suspected association.

• Epidemiological Studies: The most critical step in evaluating a signal involves designing and conducting additional, more robust epidemiological studies. These studies leverage larger, more controlled datasets and methodologies that can assess the strength, consistency, and causality of any observed association. Key examples include:

  • Vaccine Safety Datalink (VSD): A collaborative project between the CDC and nine integrated healthcare organizations, the VSD monitors vaccine safety in populations of approximately 3% of the U.S. population (about 24 million people). The VSD provides access to comprehensive medical records, including vaccination dates, diagnoses, and demographic data, enabling rapid and robust epidemiological studies, such as cohort and case-control studies, with reliable denominator data (McNeil et al., 2014).
  • Clinical Immunization Safety Assessment (CISA) Project: This CDC-funded network of vaccine safety experts at medical research centers conducts clinical research, evaluates complex cases of adverse events, and provides expert consultation to clinicians and public health officials on vaccine safety issues. CISA focuses on detailed clinical assessments of individuals who have experienced unusual or severe adverse events following vaccination (CISA Network, 2021).
  • Case-Control Studies and Cohort Studies: These traditional epidemiological designs are employed to compare the rates of adverse events in vaccinated versus unvaccinated (or differently vaccinated) populations, controlling for potential confounding factors (see Section 5 for further details).

• Causality Assessment: The ultimate goal of signal evaluation is to move towards a causality assessment. This is a complex scientific endeavor that considers multiple lines of evidence, including the temporality of the event, biological plausibility, consistency with other studies, and the exclusion of alternative causes. International consensus on causality assessment, such as the Brighton Collaboration criteria, may be utilized for specific adverse events (Brighton Collaboration, 2017).

3.5. Public Communication

Transparency and clear communication of findings are paramount to maintaining public trust in vaccination programs:

• Dissemination of Information: Findings from VAERS analyses, particularly those that lead to regulatory actions or changes in vaccine recommendations, are communicated to healthcare providers through official channels (e.g., ‘Dear Healthcare Provider’ letters, medical journal publications) and to the public via official websites (CDC, FDA), press releases, and public meetings. The VAERS database itself is publicly accessible for raw data download, underscoring a commitment to transparency (VAERS, 2021).
• Educational Initiatives: Both the CDC and FDA actively engage in educational initiatives aimed at explaining the purpose, limitations, and correct interpretation of VAERS data to the public and media, specifically to counter misinformation and promote evidence-based understanding of vaccine safety (CDC, 2021b).

This meticulous operational framework, while extensive, is constantly refined and augmented by advancements in data science and epidemiological methodologies. It underscores that VAERS is not an endpoint for safety assessment but rather a vital initial step within a layered and rigorous vaccine safety ecosystem.

Many thanks to our sponsor Esdebe who helped us prepare this research report.

4. Inherent Limitations of VAERS

While VAERS stands as an indispensable tool for the post-market surveillance of vaccine safety, its utility is inextricably linked to a thorough understanding of its inherent methodological and operational limitations. These limitations are not indicative of a flawed system but rather reflect the nature of passive surveillance and the complexities of real-world data collection. Ignoring these caveats can lead to profound misinterpretations, particularly when drawing conclusions about causality or incidence rates (Shimabukuro et al., 2015).

4.1. Underreporting

One of the most significant and widely acknowledged limitations of VAERS is the phenomenon of underreporting. Not every adverse event that occurs following vaccination is reported to the system. The exact magnitude of underreporting is difficult to quantify precisely but is generally accepted to be substantial, particularly for less severe or common adverse reactions (Rosenthal & Chen, 1995). Several factors contribute to this phenomenon:

• Lack of Awareness: Healthcare providers or individuals may simply be unaware of VAERS or the process for reporting.
• Time Constraints: Busy healthcare professionals may not have the time or resources to complete and submit reports for all relevant events.
• Perception of Event Triviality: Mild reactions (e.g., sore arm, low-grade fever) are often not reported, as they are considered expected and non-serious.
• Uncertainty of Causality: Healthcare providers may hesitate to report an event if they are not confident that the vaccine caused it, despite VAERS’s instruction to report even when causality is uncertain.
• Delayed Onset: Events with a delayed onset may be less likely to be attributed to a vaccine and therefore less likely to be reported.

The impact of underreporting is multifaceted. It leads to an underestimation of the true incidence of adverse events, particularly the more common and less serious ones. While severe and unusual events are more likely to be reported, underreporting can still obscure real signals if the reporting rate is not consistent across different event types or vaccines.

4.2. Incomplete or Inaccurate Data

Reports submitted to VAERS can vary significantly in their quality and completeness. While VAERS staff strive to obtain additional information for serious or incomplete reports, many submissions may lack essential details, contain inaccuracies, or be based on anecdotal accounts rather than verifiable medical records.

• Variability in Reporting Quality: Reports from healthcare professionals often contain more clinical detail and are supported by medical records, whereas consumer reports might be less medically precise. Manufacturer reports are often highly detailed but can be biased by the manufacturer’s own internal review processes.
• Missing Information: Critical data points, such as vaccine lot numbers, specific dates of onset, medical history, or laboratory results, may be absent. This hinders accurate assessment and linkage to other data sources.
• Recall Bias: Individuals reporting events long after they occur might have inaccurate recollections of timings or symptoms.

Incomplete or inaccurate data significantly impacts the quality and reliability of the overall dataset. It can complicate the coding process, make it challenging to identify true patterns, and necessitate resource-intensive follow-up, which is not always feasible for every report.

4.3. Lack of Causality Assessment at Submission

Perhaps the most crucial limitation often misunderstood by the public is that VAERS accepts all reports of adverse events following vaccination without attempting to determine causality at the time of submission. This means that a report simply signifies that an adverse event occurred after a vaccine was administered; it does not confirm or imply that the vaccine caused the event (CDC, 2021c).

• Maximizing Sensitivity: This design choice is deliberate. VAERS is primarily a signal detection system, aiming for maximum sensitivity to capture any potential safety concerns. Requiring causality assessment at the reporting stage would likely lead to severe underreporting, as individuals and even healthcare providers are often not in a position to definitively determine causality.
• The Coincidence Factor: In a population where millions of vaccine doses are administered annually, it is statistically inevitable that various health issues—some serious, some not—will occur purely by coincidence in the days or weeks following vaccination. These events represent the ‘background rate’ of illness in the general population (Stratton et al., 2005). VAERS captures these coincidental events alongside any potentially vaccine-related ones. Without rigorous follow-up epidemiological studies, it is impossible to differentiate correlation from causation.

Misinterpreting raw VAERS data as definitive evidence of causation is a common fallacy (post hoc ergo propter hoc – ‘after this, therefore because of this’) and is a leading source of misinformation regarding vaccine safety.

4.4. Biases in Reporting

The passive nature of VAERS makes it susceptible to various reporting biases that can skew the data and create spurious safety signals:

• Stimulated Reporting Bias: This occurs when increased public or media attention surrounding a specific vaccine or an alleged adverse event leads to a surge in reports for that event. For example, extensive media coverage of a particular perceived vaccine risk can lead to more individuals reporting similar events, regardless of actual causation. This creates an artificial ‘signal’ in the data that does not reflect a true change in event incidence.
• Recall Bias: Individuals who experience a severe or negative health outcome may be more likely to recall receiving a vaccine and attribute their condition to it, especially if there is public discussion about vaccine risks. This can lead to a disproportionate number of reports for certain severe outcomes, even if the vaccine is not causally linked.
• Healthcare Provider Bias: Healthcare providers might be more inclined to report certain types of events (e.g., those they are legally mandated to report, or those that are particularly unusual) compared to others.

These biases can distort the observed frequencies of adverse events, making it difficult to distinguish between a true increase in adverse events and an increase solely due to heightened reporting activity.

4.5. No Denominator Data

A critical structural limitation of VAERS is the absence of information on the total number of individuals vaccinated for specific vaccines within a given timeframe. VAERS collects numerator data (the number of reported adverse events) but lacks denominator data (the number of vaccine doses administered or individuals vaccinated in the population from which the reports originated) (FDA, 2021a).

• Inability to Calculate Incidence Rates: Without denominator data, it is impossible to directly calculate the incidence rate (the number of events per 100,000 doses administered or per 100,000 vaccinated individuals) of adverse events using VAERS data alone. This severely limits the ability to determine if an observed number of reports represents a higher-than-expected occurrence of an event compared to the general population or an unvaccinated cohort.
• Ecological Fallacy Risk: Attempting to infer population-level risks or rates from VAERS data without proper denominator information can lead to the ecological fallacy, where conclusions drawn about groups are incorrectly applied to individuals or where relationships observed at an aggregate level do not hold true at the individual level.

To overcome this limitation, VAERS data must be linked with other data sources that provide denominator information, such as immunization registries or large integrated healthcare systems like the Vaccine Safety Datalink (VSD) (see Section 5).

4.6. Confounding Factors

Distinguishing a vaccine-related adverse event from other concurrent health issues or background illnesses is a persistent challenge. Individuals receiving vaccines may have pre-existing medical conditions, be taking other medications, or develop unrelated illnesses shortly after vaccination. These are known as confounding factors, and they can obscure or mimic a vaccine-related event.

• Background Rates of Illness: Many health conditions occur naturally in the population at various rates. For instance, heart attacks, strokes, and autoimmune diseases occur regularly. When millions are vaccinated, some individuals will, by chance, experience these background health events shortly after vaccination. Disentangling these coincidental occurrences from true vaccine-induced events requires careful epidemiological analysis that accounts for baseline incidence rates.
• Polypharmacy and Comorbidities: Patients, particularly older adults, often have multiple underlying health conditions and take several medications. It can be challenging to determine if an adverse event is attributable to the vaccine, a pre-existing condition, an interaction with another medication, or a newly developed, unrelated illness.

These inherent limitations underscore that VAERS is an initial screening tool. Its data provides valuable signals for further investigation but cannot, by itself, serve as a definitive source for establishing causality or precise incidence rates of vaccine adverse events. A rigorous, multi-faceted approach to data analysis and interpretation is therefore essential.

Many thanks to our sponsor Esdebe who helped us prepare this research report.

5. Data Analysis and Interpretation

Given the inherent limitations of VAERS, particularly its passive nature, susceptibility to reporting biases, and lack of denominator data, the process of data analysis and interpretation demands an exceedingly cautious, methodologically rigorous, and multi-layered approach. The primary objective is to move beyond simple correlation to establish robust, scientifically verified conclusions about vaccine safety (FDA, 2021b).

5.1. Signal Detection Methodologies

The initial phase of VAERS data analysis focuses on ‘signal detection,’ which involves identifying patterns of adverse events that are unusual or unexpected. These methodologies serve as screening tools, not definitive causality assessments.

• Descriptive Statistics: The most basic form of analysis involves examining the frequency and characteristics of reported events. This includes tabulating the number of reports for specific vaccines, detailing the age and sex distribution of reporters, and observing the temporal relationship between vaccination and event onset. While these statistics can highlight areas of interest, they cannot establish causation or incidence rates due to the lack of denominator data.

• Disproportionality Analysis: This is a cornerstone of signal detection in passive surveillance systems. It involves comparing the proportion of a specific adverse event reported for a particular vaccine to the proportion of that same event reported for all other vaccines or drugs in the database. Common techniques include:

  • Reporting Odds Ratio (ROR): The ROR compares the odds of an adverse event being reported with a specific vaccine versus the odds of the same event being reported with all other vaccines in the database. An ROR significantly greater than 1 suggests a disproportionate reporting of that event for the specific vaccine, indicating a potential safety signal (Rothman et al., 2008).

  • Proportional Reporting Ratio (PRR): Similar to ROR, PRR compares the proportion of reports for a specific event with a particular vaccine to the proportion of reports for that event with all other vaccines. A PRR significantly above 1 suggests a disproportionality (Evans et al., 2001).

  • Caveats: Disproportionality analyses are sensitive screening tools designed to identify potential signals. They do not account for all reporting biases (e.g., stimulated reporting) or confounding factors. A high ROR or PRR only indicates a statistical association in reporting; it does not establish causality. Further epidemiological investigation is always required to validate these signals.

• Data Mining Algorithms: More advanced statistical and computational techniques are increasingly employed to detect subtle patterns in large datasets. These algorithms can identify clusters of related events, detect shifts in event frequency over time, or uncover associations that might not be apparent through manual review or simpler statistical methods. Machine learning approaches are also being explored to enhance signal detection capabilities, though their application requires careful validation to avoid generating false positives (Sasaki et al., 2017).

5.2. Epidemiological Studies for Causality Assessment

Once a potential safety signal is detected in VAERS, the critical next step involves rigorous epidemiological studies designed to assess the strength, consistency, and most importantly, the causality of the observed association. These studies utilize more robust methodologies and often draw upon linked healthcare databases that provide essential denominator data, allowing for the calculation of incidence rates and control for confounding factors.

• Vaccine Safety Datalink (VSD): As previously mentioned, the VSD is a vital resource. It provides access to comprehensive, de-identified medical data for millions of individuals across multiple healthcare organizations. This enables researchers to conduct a variety of powerful epidemiological studies:

  • Self-Controlled Case Series (SCCS): This design is particularly valuable for studying acute outcomes after vaccination. Each vaccinated individual serves as their own control, comparing the rate of an adverse event during a ‘risk period’ immediately following vaccination to the rate during ‘control periods’ before or after vaccination. This effectively controls for stable, individual-level confounding factors (e.g., genetic predispositions, chronic conditions) that do not change over time (Farrington et al., 1995).
  • Case-Control Studies: These studies compare individuals who experienced a specific adverse event (cases) with individuals who did not (controls) but are otherwise similar. Researchers then look retrospectively at their vaccination history to determine if cases were more likely to have received a particular vaccine. They are efficient for studying rare adverse events but can be susceptible to recall bias.
  • Cohort Studies: In a cohort study, groups of vaccinated and unvaccinated (or differently vaccinated) individuals are followed prospectively over time to observe the incidence of specific adverse events. This design is robust for calculating incidence rates and identifying risk factors but requires large populations and long follow-up periods, making them costly and time-consuming.

• Clinical Immunization Safety Assessment (CISA) Project: The CISA Network complements large-scale epidemiological studies by providing in-depth clinical evaluation of individual, complex adverse events. CISA experts perform detailed medical record reviews, conduct specialized diagnostic tests, and offer clinical consultations to ascertain the most likely cause of a challenging adverse event, contributing to a deeper understanding of specific mechanisms (CISA Network, 2021).

• Other Data Sources: Beyond VSD and CISA, the FDA and CDC utilize data from other sources, including the Medicare and Medicaid databases, private insurance claims data, and other national and international vaccine safety surveillance systems (e.g., EudraVigilance in Europe, Canada’s Public Health Agency). Consistency of a signal across multiple, independent data sources significantly strengthens the evidence for a causal association.

• Bradford Hill Criteria for Causality: While not a rigid checklist, the Bradford Hill criteria provide a widely accepted framework for assessing causality in epidemiological investigations. These criteria include temporality (exposure must precede outcome), strength of association, consistency across studies, biological plausibility, dose-response relationship, reversibility, specificity, coherence, and analogy (Hill, 1965). Epidemiologists and regulatory bodies consider these criteria when evaluating the totality of evidence to determine if a causal link can be established between a vaccine and an adverse event.

5.3. Risk-Benefit Assessment and Public Health Decisions

The ultimate goal of comprehensive vaccine safety monitoring is to continuously assess the overall risk-benefit profile of vaccines. This assessment informs crucial public health decisions and recommendations:

• Balancing Benefits and Risks: Public health agencies like the CDC (via its Advisory Committee on Immunization Practices, ACIP) and the FDA must weigh the enormous benefits of preventing infectious diseases (e.g., reduced morbidity, mortality, healthcare burden) against the potential for rare, sometimes serious, vaccine-related adverse events. This involves quantitative analysis of disease burden reduction versus the incidence of confirmed adverse events.
• Informing Policy Changes: Robust safety data can lead to various policy adjustments, such as changes to vaccine schedules, recommendations for specific populations (e.g., immunocompromised individuals), updates to vaccine labeling (e.g., warnings, contraindications), or even, in exceedingly rare cases, the withdrawal of a vaccine if its risks demonstrably outweigh its benefits. A historical example of a vaccine withdrawal due to safety concerns was the rotavirus vaccine (RotaShield) in 1999, following a confirmed association with intussusception identified through VAERS and subsequent epidemiological studies (CDC, 1999).
• Ethical Considerations: These decisions involve significant ethical considerations, balancing individual risks with population-level health benefits and ensuring equitable access to safe and effective vaccines.

5.4. Communication of Findings

Transparent, timely, and accurate communication of vaccine safety findings is critical for maintaining public trust and informed decision-making. This includes:

• Clear and Nuanced Messaging: Public health authorities are responsible for communicating complex scientific information in an understandable manner, explicitly distinguishing between reported events and confirmed causal links. The communication must acknowledge uncertainties where they exist but firmly ground conclusions in the best available scientific evidence.
• Addressing Misinformation: In an era of rapid information dissemination, agencies must proactively address misinformation and disinformation that often arises from misinterpretations of VAERS data. This requires consistent messaging, providing accessible explanations of VAERS limitations, and directing the public to reliable, evidence-based resources.
• Targeted Communication: Findings are disseminated through various channels tailored to specific audiences: scientific publications for researchers, clinical advisories for healthcare providers, and accessible summaries for the general public via official websites and media outlets.

The entire process, from initial signal detection in VAERS to comprehensive epidemiological studies and subsequent policy decisions, underscores a commitment to continuous vigilance and scientific rigor in ensuring the safety of vaccines.

Many thanks to our sponsor Esdebe who helped us prepare this research report.

6. Distinction Between Raw Data and Scientifically Verified Conclusions

One of the most persistent challenges in public discourse surrounding vaccine safety is the fundamental misunderstanding and conflation of raw data from passive surveillance systems like VAERS with scientifically verified conclusions derived from rigorous epidemiological investigation. This distinction is not merely semantic; it is foundational to sound scientific interpretation and essential for fostering informed public understanding and trust in immunization programs.

6.1. Raw VAERS Data: Unverified Reports and Coincidental Events

Raw VAERS data consists of a compilation of unverified reports, each representing an individual’s observation or suspicion of an adverse health event occurring after vaccination. These reports are inherently anecdotal in nature and carry several critical characteristics that prevent them from being considered definitive evidence of causation:

• Anecdotal and Unverified Nature: A VAERS report is typically a firsthand account or a summary of observations submitted by a reporter. While serious reports are often followed up to gather more medical detail, the initial submission is not systematically verified against comprehensive medical records or independent clinical assessment. This means a report can contain inaccuracies, subjective interpretations, or incomplete clinical information. It is crucial to remember that anyone can submit a report, and while the vast majority are submitted in good faith, the system’s open nature means that the information within a report is not vetted for scientific accuracy or clinical confirmation at the point of submission.

• The Inevitability of Coincidental Events: In any large population, a wide array of health events—from minor ailments to severe diseases, hospitalizations, and deaths—occur daily, regardless of vaccination status. These represent the ‘background rate’ of illness. When millions of vaccine doses are administered annually, it is a statistical certainty that some individuals will, purely by chance, experience these pre-existing health issues or develop new, unrelated conditions shortly after receiving a vaccine. For example, in the United States, approximately 8,000 people die each day (CDC, 2023). If 1 million people receive a vaccine on a given day, it is statistically predictable that roughly 20-30 of those vaccinated individuals will die within 24 hours from causes entirely unrelated to the vaccine, simply by natural chance. VAERS collects these coincidental events alongside any potentially vaccine-related ones. Without detailed epidemiological studies to compare vaccinated populations with unvaccinated ones, and to account for background rates and confounding factors, it is impossible to distinguish between a coincidental occurrence and a causal link.

• Reporting Bias and Confounding Factors: As detailed in Section 4, raw VAERS data is susceptible to various biases, including stimulated reporting, recall bias, and the influence of confounding factors. These biases can create spurious signals, where an event appears disproportionately linked to a vaccine merely because it is being reported more frequently, not because it is actually occurring more frequently or due to the vaccine. For instance, heightened media attention on a particular alleged vaccine-related outcome can lead to a surge in reports for that outcome, irrespective of any true causal link.

Therefore, raw VAERS data serves as a repository of observations and hypotheses. It cannot, by itself, establish a causal relationship between a vaccine and an adverse event. Treating individual VAERS reports as definitive proof of causation is a fundamental misapplication of the system’s design and intent.

6.2. Scientifically Verified Conclusions: Rigorous Inquiry and Causal Inference

In stark contrast to raw VAERS data, scientifically verified conclusions regarding vaccine safety are the product of rigorous, systematic investigation, utilizing established epidemiological principles and methodologies. This process extends far beyond mere signal detection:

• The ‘Body of Evidence’ Approach: Establishing a causal relationship requires considering a comprehensive ‘body of evidence’ from multiple, diverse sources. No single study or data source is usually sufficient to definitively prove causation. This body of evidence includes:

  • Data from Pre-licensure Clinical Trials: These trials provide initial safety profiles, identifying common and some less common adverse reactions.
  • VAERS and other Passive Surveillance Systems: Providing early signals and hypothesis generation.
  • Active Surveillance Systems (e.g., VSD, CISA): These systems offer high-quality, linked medical data with denominator information, enabling robust epidemiological studies (cohort, case-control, SCCS) to calculate incidence rates and assess risk in vaccinated versus unvaccinated populations.
  • Specialized Research and Laboratory Studies: Investigating biological mechanisms (biological plausibility) that could explain an observed association.
  • International Surveillance Data: Comparing findings across different countries and populations to assess consistency.
  • Peer-Reviewed Literature: Independent scientific scrutiny and replication of findings.

• Application of Causality Criteria: As discussed, frameworks like the Bradford Hill criteria are implicitly or explicitly applied when evaluating the totality of evidence to determine causality. These criteria guide scientists in assessing the strength, consistency, temporality, and biological plausibility of an observed association.

• Controlling for Confounding Factors and Bias: Rigorous epidemiological studies are meticulously designed to minimize bias and account for confounding factors. This involves statistical adjustments for age, sex, underlying health conditions, medications, and other variables that could independently influence the occurrence of an adverse event. Sophisticated study designs like the SCCS are specifically employed to control for stable individual characteristics.

• Consensus and Peer Review: Scientific conclusions are not typically formed by individual researchers working in isolation. They emerge through a process of peer review, replication of findings, and ultimately, scientific consensus within the broader expert community. Independent advisory committees (e.g., ACIP for the CDC, VRBPAC for the FDA) review all available data and provide recommendations based on the highest standards of scientific evidence.

• The Danger of Premature Conclusions and Misinformation: Misinterpreting raw VAERS data as definitive evidence of causation can have severe public health consequences. Historically, such misinterpretations have fueled baseless vaccine scares (e.g., the debunked link between the MMR vaccine and autism), leading to vaccine hesitancy, declines in immunization rates, and subsequent resurgence of vaccine-preventable diseases (Godlee et al., 2011). This underscores the imperative for public health communicators, media, and individuals to rely on scientifically verified conclusions rather than unanalyzed raw data.

In essence, VAERS functions as a crucial early warning system—a highly sensitive radar for detecting potential issues. However, the signals it generates must be subjected to a rigorous scientific validation process, utilizing a diverse array of epidemiological tools and expert review, before definitive conclusions about vaccine safety can be drawn. This two-tiered process ensures that public health decisions are grounded in robust evidence, not anecdotal observations or statistical anomalies.

Many thanks to our sponsor Esdebe who helped us prepare this research report.

7. Conclusion

The Vaccine Adverse Event Reporting System (VAERS) is an indispensable and integral component of the multifaceted vaccine safety monitoring ecosystem within the United States. Established under the explicit mandate of the National Childhood Vaccine Injury Act of 1986, VAERS serves as a critical passive surveillance mechanism for the initial detection of potential adverse events following vaccination, thereby providing invaluable information for the continuous assessment of vaccine safety in a dynamic real-world environment. Its primary strength lies in its broad reach, enabling the capture of a wide spectrum of reported health issues across diverse populations, including those that might be rare or unexpected, thus complementing the controlled environment of pre-licensure clinical trials.

However, the utility and interpretation of VAERS data are predicated upon a profound understanding of its inherent methodological and operational limitations. These include pervasive issues such as underreporting, the potential for incomplete or inaccurate data, the absence of initial causality assessment at the point of submission, susceptibility to various reporting biases (e.g., stimulated reporting), and critically, the lack of robust denominator data required for the direct calculation of incidence rates. These limitations collectively necessitate an exceptionally cautious, rigorous, and scientifically methodical approach to data analysis and interpretation.

The process of interpreting VAERS data progresses from initial signal detection, often employing disproportionality analyses, to sophisticated epidemiological investigations utilizing robust active surveillance systems such as the Vaccine Safety Datalink (VSD) and the Clinical Immunization Safety Assessment (CISA) Project. These follow-up studies, which include self-controlled case series, case-control studies, and cohort studies, are meticulously designed to assess the strength, consistency, and biological plausibility of any observed associations, rigorously controlling for confounding factors and biases. The ultimate goal is to generate scientifically verified conclusions that inform comprehensive risk-benefit assessments and guide evidence-based public health decisions.

Crucially, a clear and unequivocal distinction must always be maintained between raw, unverified VAERS data and scientifically verified conclusions. Raw reports represent observations and hypotheses; they are not, by themselves, definitive proof of causation. Definitive conclusions on vaccine safety are only established through a robust body of evidence, derived from multiple lines of inquiry and subjected to rigorous scientific scrutiny, including peer review and expert consensus. Misinterpreting raw VAERS data as causal evidence not only distorts public perception but also significantly undermines public confidence in the scientific process and, by extension, in vital immunization programs.

Looking forward, continuous advancements in data science, including sophisticated data mining and machine learning algorithms, hold promise for enhancing VAERS’s signal detection capabilities. Furthermore, strengthened linkages between passive and active surveillance systems will undoubtedly improve the efficiency and accuracy of causality assessments. By fully comprehending the purpose, operational mechanics, and inherent limitations of VAERS, and by consistently upholding the critical distinction between reported events and scientifically validated causal relationships, all stakeholders can more effectively navigate complex vaccine safety discussions, foster a culture of evidence-based decision-making, and collectively contribute to the sustained success of global immunization efforts.

Many thanks to our sponsor Esdebe who helped us prepare this research report.

References

• Andre, F. E., Booy, J. L., et al. (2008). ‘Vaccination greatly reduces disease, disability, death and inequity worldwide’. Bulletin of the World Health Organization, 86(2), 140–146.
• Brighton Collaboration. (2017). Causality Assessment of Adverse Events Following Immunization. Retrieved from https://brightoncollaboration.us/about-causality-assessment/
• Centers for Disease Control and Prevention. (1999). ‘Withdrawal of Rotavirus Vaccine Recommendation’. Morbidity and Mortality Weekly Report (MMWR), 48(43), 1007. Retrieved from https://www.cdc.gov/mmwr/preview/mmwrhtml/mm4843a4.htm
• Centers for Disease Control and Prevention. (2021a). Vaccine Adverse Event Reporting System (VAERS). Retrieved from https://www.cdc.gov/vaccinesafety/ensuringsafety/monitoring/vaers/index.html
• Centers for Disease Control and Prevention. (2021b). Guide to Interpreting VAERS Data. Retrieved from https://vaers.hhs.gov/data/dataguide.html
• Centers for Disease Control and Prevention. (2021c). VAERS: What is it, and what does it tell us? Retrieved from https://www.cdc.gov/vaccinesafety/ensuringsafety/monitoring/vaers/VAERS-COV-VAX-INFO.html
• Centers for Disease Control and Prevention. (2023). FastStats: Deaths and Mortality. Retrieved from https://www.cdc.gov/nchs/faststats/deaths.htm
• CISA Network. (2021). Clinical Immunization Safety Assessment (CISA) Project. Retrieved from https://www.cdc.gov/vaccinesafety/research/cisa/index.html
• Evans, G. (1992). ‘The National Childhood Vaccine Injury Act’. Pediatrics, 89(1), 163–166.
• Evans, S. J., Waller, P. C., & Davis, S. (2001). ‘Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports’. Pharmacoepidemiology and Drug Safety, 10(6), 483–486.
• Farrington, C. P., Nashef, L., & Miller, E. (1995). ‘Case series analysis of adverse reactions to vaccines: a comparative evaluation’. American Journal of Epidemiology, 141(3), 228–236.
• Food and Drug Administration. (2021a). Vaccine Adverse Event Reporting System (VAERS) Overview. Retrieved from https://www.fda.gov/vaccines-blood-biologics/vaccine-adverse-events/vaers-overview
• Food and Drug Administration. (2021b). Vaccine Adverse Event Reporting System (VAERS) Questions and Answers. Retrieved from https://www.fda.gov/vaccines-blood-biologics/report-problem-center-biologics-evaluation-research/vaccine-adverse-events
• Godlee, F., Smith, J., & Marcovitch, H. (2011). ‘Wakefield’s article linking MMR vaccine and autism was fraudulent’. BMJ, 342, c7452.
• Hill, A. B. (1965). ‘The Environment and Disease: Association or Causation?’. Proceedings of the Royal Society of Medicine, 58(5), 295–300.
• MedDRA. (2021). What is MedDRA? Retrieved from https://www.meddra.org/how-to-use/basics/what-is-meddra
• McNeil, M. M., Gee, J., et al. (2014). ‘The Vaccine Safety Datalink: a collaborative project to monitor vaccine safety’. Pharmacoepidemiology and Drug Safety, 23(3), 324–333.
• National Childhood Vaccine Injury Act of 1986, Public Law 99–660, 42 U.S.C. § 300aa-1 et seq. (1986).
• National Vaccine Program Office. (2000). The National Vaccine Injury Compensation Program: A Review of the First 10 Years. U.S. Department of Health and Human Services.
• Rosenthal, S. & Chen, R. (1995). ‘The reporting of adverse events following immunization: estimates of reporting completeness’. Vaccine, 13(1), 89–94.
• Rothman, K. J., Greenland, S., & Lash, T. L. (2008). Modern Epidemiology (3rd ed.). Lippincott Williams & Wilkins.
• Sasaki, T., Arai, W., et al. (2017). ‘Exploring adverse drug event signals using a novel data mining technique with a large health claims database’. Pharmacoepidemiology and Drug Safety, 26(2), 273–279.
• Shimabukuro, T. T., Nguyen, M., Martin, D., & DeStefano, F. (2015). ‘Safety monitoring in the Vaccine Adverse Event Reporting System (VAERS)’. Vaccine, 33(36), 4398–4405.
• Stratton, K. R., Ford, A., & Rusch, J. R. (2005). ‘Adverse events of medical products in the context of vaccination: the need for careful attribution’. Vaccine, 23(17-18), 2209–2214.
• VAERS. (2021). VAERS Data Search. Retrieved from https://vaers.hhs.gov/data.html
• World Health Organization. (2002). The Importance of Pharmacovigilance: Safety Monitoring of Medicinal Products. WHO Press.