Evaluation of Propensity Score Based Methods in Sentinel Study Settings Using Simulation Experiments

The Mini-Sentinel pilot developed several pre-programmed analytic modules for medical product safety assessments including a propensity score matching (PSM) module that has been converted to a routine querying tool for the FDA’s active postmarket risk identification and analysis (ARIA) system.

This project will inform appropriate use of the PSM tool within the ARIA system for a one-time (non-sequential) data analysis via rigorous demonstration of relative performance under numerous simulated circumstances similar to real-world safety analyses. With the introduction of the ARIA framework, FDA re-assessed the relevance and potential utility of the four existing analytic modules and related methods, and re-prioritized methods evaluation work. As a result, this workgroup focused on the evaluation of PS-based methods only, which includes regression and weighting methods but not the self-controlled risk interval module.

The project also assesses other related PS methods that could expand the functionality of the existing PSM tool by adding PS stratification, regression, and inverse-probability weighting methods. The goal of this project is to describe the statistical and epidemiological performance of these PS-based methods under conditions likely to be encountered by the ARIA system so that end-users will be able to optimize the tools’ use. The findings from this effort will also guide future tool development and enhancement work. The infrastructure for simulation experiments built by this workgroup is re-usable and can be used in future assessments of the performance of other modules and methods.