Use of TreeScan by Non-Sentinel Investigators

Many non-Sentinel investigators have used TreeScan software. This includes academia, industry, and other regulators. 

The Sentinel System's analytic tools support dataset creation for signal identification analyses, which you can execute with TreeScan software. Additionally, the Sentinel Initiative has supported the addition of new analytic models within the TreeScan software, e.g., Bernoulli Scan Statistics and Tree-Temporal Scan Statistics.

Below is a list of publications of how others in the scientific community have used TreeScan. These publications showcase how the community has further advanced these methods or applied them in novel ways beyond medical product safety.

Summary of the Use of Tree Scan Statistics for Drug and Vaccine Safety Monitoring

Sun Yixin, Wang Miao, Yang Mingfang, Zhan Siyan
November 3, 2021

The purpose of this review is to summarize the development and application of tree-based scan statistic (TreeScan), explain the methodology and provide a reference for future use of this method by reviewing the original pharmacoepidemiological and vaccine studies using the TreeScan. Medline, Embase and Web of Science databases were used for the retrieval of eligible studies using keywords related to TreeScan. A total of 15 eligible studies were included, in which 9 studies explored the adverse events of drugs and 6 studies focused on the safety of vaccines. Three types of models (Poisson probability model, Bernoulli probability model and tree-temporal scan statistic model) of TreeScan were used. The major differences among the three models were 1) whether predefined control was used according to research question, 2) whether the time from exposure to onset of adverse events was considered. Several studies explored its ability by comparing with other methods for adverse event detection or by using known adverse events. This review shows that TreeScan is an effective method for the safety signal detection of drugs or vaccines, which develops rapidly and globally. It is very necessary to promote its use in drug safety monitoring and other related fields in China.

Translated from Chinese.

View the article (Chinese).

A Novel Data Mining Application to Detect Safety Signals for Newly Approved Medications in Routine Care of Patients With Diabetes

Michael Fralick, Martin Kulldorff, Donald Redelmeier, Shirley V. Wang, Seanna Vine, Sebastian Schneeweiss, Elisabetta Patorno
April 6, 2021

Clinical trials are often underpowered to detect serious but rare adverse events of a new medication. We applied a novel data mining tool to detect potential adverse events of canagliflozin, the first sodium glucose co-transporter 2 (SGLT2 inhibitor) in the United States, using real-world data from shortly after its market entry and before public awareness of its potential safety concerns. In a U. S. commercial claims dataset (29 March 2013-30 Sept 2015), two pairwise cohorts of patients over 18 years of age with type 2 diabetes (T2D) who were newly dispensed canagliflozin or an active comparator, that is a dipeptidyl peptidase 4 inhibitor (DPP4) or a glucagon-like peptide 1 receptor agonist (GLP1), were identified and propensity score-matched. We used variable ratio matching with up to four people receiving a DPP4 or GLP1 for each person receiving canagliflozin. We identified potential safety signals using a hierarchical tree-based scan statistic data mining method with the hierarchical outcome tree constructed based on international classification of disease coding. We screened for incident adverse events where there were more outcomes observed among canagliflozin vs. comparator initiators than expected by chance, after adjusting for multiple testing.

View the article.

A Broad Safety Assessment of the 9-Valent Human Papillomavirus Vaccine

W. Katherine Yih, Martin Kulldorff, Inna Dashevsky, Judith C. Maro
February 9, 2021

Surveys of parents indicate safety is their top concern about human papillomavirus (HPV) vaccination. A data-mining method not requiring pre-specification of health outcome(s) of interest or post-exposure period(s) of potentially increased risk can check for associations between an exposure and any of thousands of medically attended health outcomes. The method was applied to the 9-valent HPV vaccine (HPV9) to detect potential safety problems. Data on 9-26-year-olds who had received HPV9 vaccine between November 4, 2016 and August 5, 2018, inclusive, were extracted from Marketscan and analyzed for statistically significant clustering of incident diagnoses within the hierarchy of ICD-10-CM coded diagnoses and temporally within the 1 year after vaccination, using the self-controlled tree-temporal scan statistic and TreeScan software. Only 56 days of post-vaccination enrollment was required; subsequent follow-up was censored at disenrollment. Multiple testing was adjusted for. The analysis included 493,089 doses of HPV9. Almost all signals resulted from temporal confounding, not unexpected with a 1-year follow-up period. The only plausible signals were for non-specific adverse events (e.g., injection-site reactions and headache) on Days 1-2 after vaccination, with attributable risks as low as 1 per 100,000 vaccinees. Considering the broad scope of the evaluation and the high statistical power, the findings of no specific serious adverse events should provide reassurance about this vaccine's safety.

View the article.

Active Surveillance of the Safety of Medications Used in Pregnancy

Krista F. Huybrechts, Martin Kulldorff, Sonia Hernández-Díaz, Brian T. Bateman, Yanmin Zhu, Helen Mogun, Shirley V. Wang
January 11, 2021

We rely on post-marketing approaches to define the risk of medications in pregnancy because information at the time of drug approval is limited. Most studies in pregnancy focus on a single or selected outcomes. However, women must balance the benefit of treatment against all possible adverse effects. Our objective was to apply and evaluate a tree-based scan statistic data mining method (TreeScan) as a safety surveillance approach that allows for simultaneous evaluation of a comprehensive range of adverse pregnancy outcomes, while preserving the overall false positive rate. We evaluated TreeScan with a cohort design and adjustment via propensity score techniques using two test cases: (1) opioids and neonatal opioid withdrawal syndrome, and (2) valproate and congenital malformations, implemented in pregnancy cohorts nested in the Medicaid Analytic eXtract (1/1/2000 - 12/31/2014) and IBM MarketScan Research Database (1/1/2003 - 9/30/2015). In both cases, we identified known safety concerns, with only one previously unreported alert at the preset statistical alerting threshold. This evaluation shows the promise of TreeScan-based approaches for systematic drug safety monitoring in pregnancy. A targeted screening approach followed by deeper investigation to refine understanding of potential signals will ensure pregnant women and their physicians have access to the best available evidence to inform treatment decisions.

View the article.

Data Mining for Adverse Events of Tumor Necrosis Factor-Alpha Inhibitors in Pediatric Patients: Tree-Based Scan Statistic Analyses of Danish Nationwide Health Data

Wintzell V, Svanström H, Melbye M, Ludvigsson JF, Pasternak B, Kulldorff M
October 26, 2020

Tumor necrosis factor-alpha (TNF-α) inhibitors are efficacious and considered generally safe in adults. However, pediatric-specific safety evidence is scarce. The aim of this study was to screen for signals of previously unknown adverse events of TNF-α inhibitors in pediatric patients. We conducted a data-mining study based on routinely collected, nationwide Danish healthcare data for 2004-2016. Using tree-based scan statistics to identify events with unexpectedly high incidence during TNF-α inhibitor use among patients with inflammatory bowel disease or juvenile idiopathic arthritis, two analyses were performed: comparison with episodes of no use and with other time periods from the same patient. Based on incident physician-assigned diagnosis codes from outpatient and inpatient visits in specialist care, we screened thousands of potential adverse events while adjusting for multiple testing. We identified 1310 episodes of new TNF-α inhibitor use that met the eligibility criteria. Two signals of adverse events of TNF-α inhibitors, as compared with no use, were detected. First, there were excess events of dermatologic complications (ICD-10: L00-L99, 87 vs. 44 events, risk difference [RD] 3.3%), which have been described previously in adults and children. Second, there were excess events of psychiatric diagnosis adjustment disorders (ICD-10: F432, 33 vs. 7 events, RD 2.0%), which was likely associated with the underlying disease and its severity, rather than with the treatment. The self-controlled analysis generated no signal. No signals of previously unknown adverse events of TNF-α inhibitors in pediatric patients were detected. The study showed that real-world data and newly developed methods for adverse events data mining can play a particularly important role in pediatrics where pre-approval drug safety data are scarce.

View the article

Safety Surveillance of Pneumococcal Vaccine Using Three Algorithms: Disproportionality Methods, Empirical Bayes Geometric Mean, and Tree-Based Scan Statistic

Lee H, Kim JH, Choe YJ, Shin JY
May 22, 2020

Diverse algorithms for signal detection exist. However, inconsistent results are often encountered among the algorithms due to different levels of specificity used in defining the adverse events (AEs) and signal threshold. We aimed to explore potential safety signals for two pneumococcal vaccines in a spontaneous reporting database and compare the results and performances among the algorithms. Safety surveillance was conducted using the Korea national spontaneous reporting database from 1988 to 2017. Safety signals for pneumococcal vaccine and its subtypes were detected using the following the algorithms: disproportionality methods comprising of proportional reporting ratio (PRR), reporting odds ratio (ROR), and information component (IC); empirical Bayes geometric mean (EBGM); and tree-based scan statistics (TSS). Moreover, the performances of these algorithms were measured by comparing detected signals with the known AEs or pneumococcal vaccines (reference standard). Among 10,380 vaccine-related AEs, 1135 reports and 101 AE terms were reported following pneumococcal vaccine. IC generated the most safety signals for pneumococcal vaccine (40/101), followed by PRR and ROR (19/101 each), TSS (15/101), and EBGM (1/101). Similar results were observed for its subtypes. Cellulitis was the only AE detected by all algorithms for pneumococcal vaccine. TSS showed the best balance in the performance: the highest in accuracy, negative predictive value, and area under the curve (70.3%, 67.4%, and 64.2%). Discrepancy in the number of detected signals was observed between algorithms. EBGM and TSS calibrated noise better than disproportionality methods, and TSS showed balanced performance. Nonetheless, these results should be interpreted with caution due to a lack of a gold standard for signal detection.

View the article

Using the Self-Controlled Tree-Temporal Scan Statistic to Assess the Safety of Live Attenuated Herpes Zoster Vaccine

Yih WK, Kulldorff M, Dashevsky I, Maro J
May 7, 2019

The self-controlled tree-temporal scan statistic allows detection of potential vaccine- or drug-associated adverse events without pre-specifying the specific events or post-exposure risk intervals of concern. It thus opens a promising new avenue for safety studies. The method has been successfully used to evaluate the safety of two vaccines for adolescents and young adults, but its suitability to study vaccines for older adults had not been established. The current study applied the method to assess the safety of live attenuated herpes zoster vaccination during 2011-2017 in U.S. adults ≥ 60 years old, using claims data from Truven Health MarketScan® Research Databases. Counts of International Classification of Diseases diagnosis codes recorded in emergency department or hospital settings were scanned for any statistically unusual clustering within a hierarchical tree structure of diagnoses and within 42 days after vaccination. Among 1.24 million vaccinations, four clusters were found: cellulitis on Days 1-3, non-specific erythematous condition on Days 2-4, "other complications…" on Days 1-3, and non-specific allergy on Days 1-6. These results are consistent with local injection-site reactions and other known, generally mild vaccine-associated adverse events and a favorable safety profile. This method may be useful for assessing the safety of other vaccines for older adults. 

View the article

Bacillus Calmette-Guérin (BCG) Vaccine Safety Surveillance in the Korea Adverse Event Reporting System Using the Tree-Based Scan Statistic and Conventional Disproportionality-Based Algorithms

Kim JH, Lee H, Shin JY
May 6, 2020

Substantial variations in the safety profiles of different formulations of the bacillus Calmette-Guérin (BCG) vaccine exist. Therefore, we aimed to detect safety signals of BCG vaccine for intradermal injection (BCG-ID) and percutaneous injection (BCG-PC) in the Korea Adverse Event Reporting System (KAERS). We conducted a vaccine safety surveillance study from the adverse events (AEs) reported following BCG vaccine in the Korea Institute of Drug Safety and Risk Management KAERS Database (KIDS-KD) between 2005 and 2017. We used the tree-based scan statistic (TSS) and four disproportionality-based algorithms for signal detection: empirical Bayesian geometric mean; proportional reporting ratio; reporting odds ratio; and information component. The detected signals from each algorithm was compared with the known AEs of BCG vaccine (reference standard) to present positive predictive value (PPV) and area under the receiver operating curve (AUC).

View the article

An Implementation and Visualization of the Tree-Based Scan Statistic for Safety Event Monitoring in Longitudinal Electronic Health Data

Schachterle SE, Hurley S, Liu Q, Petronis KR, Bate A
January 8, 2019

Longitudinal electronic healthcare data hold great potential for drug safety surveillance. The tree-based scan statistic (TBSS), as implemented by the TreeScan® software, allows for hypothesis-free signal detection in longitudinal data by grouping safety events according to branching, hierarchical data coding systems, and then identifying signals of disproportionate recording (SDRs) among the singular events or event groups. The objective of this analysis was to identify and visualize SDRs with the TBSS in historical data from patients using two antifungal drugs, itraconazole or terbinafine. By examining patients who used either itraconazole or terbinafine, we provide a conceptual replication of a previous TBSS analyses by varying methodological choices and using a data source that had not been previously used with the TBSS, i.e., the Optum Clinformatics™ claims database. With this analysis, we aimed to test a parsimonious design that could be the basis of a broadly applicable method for multiple drug and safety event pairs. 

View the article

Tree-Based Scan Statistic - Application in Manufacturing-Related Safety Signal Detection

Mahaux O, Bauchau V, Zeinoun Z, Van Holle L
January 3, 2019

Over the last decades, medicinal regulations have been put into place and have considerably improved manufacturing practices. Nevertheless, safety issues may still arise. Using the simulation described in this manuscript, our aim is to develop adequate detection methods for manufacturing-related safety signals, especially in the context of biological products. Pharmaceutical companies record the entire batch genealogies, from seed batches over intermediates to final product (FP) batches. We constructed a hierarchical tree based on this genealogy information and linked it to the spontaneous safety data available for the FP batch numbers. The tree-based scan statistic (TBSS) was used on simulated data as a proof of concept to locate the source that may have subsequently generated an excess of specific adverse events (AEs) within the manufacturing steps, and to evaluate the method's adjustment for multiple testing.

View the article

Meningococcal Conjugate Vaccine Safety Surveillance in the Vaccine Safety Datalink Using a Tree-Temporal Scan Data Mining Method

Li R, Weintraub E, McNeil MM, Kulldorff M, Lewis EM, Nelson J, Xu S, Qian L, Klein NP, Destefano F
February 18, 2018

The objective of this study was to conduct a data mining analysis to identify potential adverse events (AEs) following MENACWY-D using the tree-temporal scan statistic in the Vaccine Safety Datalink population and demonstrate the feasibility of this method in a large distributed safety data setting. Traditional pharmacovigilance techniques used in vaccine safety are generally geared to detecting AEs based on pre-defined sets of conditions or diagnoses. Using a newly developed tree-temporal scan statistic data mining method, a pilot study was performed to evaluate the safety profile of the meningococcal conjugate vaccine Menactra® (MenACWY-D), screening thousands of potential AE diagnoses and diagnosis groupings. The study cohort included enrolled participants in the Vaccine Safety Datalink aged 11 to 18 years who had received MenACWY-D vaccination(s) between 2005 and 2014. The tree-temporal scan statistic was employed to identify statistical associations (signals) of AEs following MENACWY-D at a 0.05 level of significance, adjusted for multiple testing. 

View the article


If you have used Sentinel's tools or TreeScan to support signal identification, contact us to add your publication to this page.