Quantifying How Small Variations in Design Elements Affect Risk in an Incident Cohort Study in Claims

Epidemiological study reporting is improving but is not transparent enough for easy evaluation or replication. One barrier is insufficient details about design elements in published studies. Using a previously conducted drug safety evaluation in claims as a test case, we investigated the impact of small changes in five key design elements on risk estimation. These elements are index day of incident exposure's determination of look-back or follow-up periods, exposure duration algorithms, heparin exposure exclusion, propensity score model variables, and Cox proportional hazard model stratification. We covaried these elements using a fractional factorial design, resulting in 24 risk estimates for one outcome. We repeated eight of these combinations for two additional outcomes. We measured design effects on cohort sizes, follow-up time, and risk estimates.