Faculty Web Pages
DoPE News
- DEcIDE Methods Center CER Scan (February 2012)
- Aaron Kesselheim Examines Path to More Affordable and Effective Drugs
- Steven Brunelli on Iodide and Thyroid Disease
- DEcIDE Methods Center CER Scan (January 2012)
- Steven Brunelli Receives Mentor Award
- Aaron Kesselheim Profiled in the December Issue of Health Affairs
DoPE Tweets
- Division Chief discusses Lipitor going off-patent on PBS @NewsHour http://t.co/ddDSB3GX https://twitter.com/#!/bwhdope 2011/12/01
- RT @NewsHour: Lipitor, the most profitable prescription drug in history, is going generic http://t.co/5bNwGjkI https://twitter.com/#!/bwhdope 2011/12/01
DEcIDE Methods Center Monthly Literature Scan
AHRQ has funded the Brigham DEcIDE Center for Comparative Effectiveness Research to lead the DEcIDE Methods Center (DMC). One of the primary aims of the DMC is to develop a multifaceted Methods Learning Network for comparative effectiveness research. As part of the Learning Network, monthly literature reviews are conducted to alert the DEcIDE network to articles on methodological innovations or reviews of analytic approaches that may help improve the validity of original comparative effectiveness research. Highly specific topics are weeded out and there is a focus on approaches relevant to a larger number of applied investigators in CER.
For some months, a cluster of methods references on a specific theme thought to be of interest is also provided.
Current and previous scans may be found below:
February 2012
January 2012
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011
February 2012
CER Scan [Epub ahead of print]
Bias in Observational Studies of Prevalent Users: Lessons for Comparative Effectiveness Research From a Meta-Analysis of Statins. Danaei G, Tavakkoli M, Hernán MA.
Randomized clinical trials (RCTs) are usually the preferred strategy with which to generate evidence of comparative effectiveness, but conducting an RCT is not always feasible. Though observational studies and RCTs often provide comparable estimates, the questioning of observational analyses has recently intensified because of randomized-observational discrepancies regarding the effect of postmenopausal hormone replacement therapy on coronary heart disease. Reanalyses of observational data that excluded prevalent users of hormone replacement therapy led to attenuated discrepancies, which begs the question of whether exclusion of prevalent users should be generally recommended. In the current study, the authors evaluated the effect of excluding prevalent users of statins in a meta-analysis of observational studies of persons with cardiovascular disease. The pooled, multivariate-adjusted mortality hazard ratio for statin use was 0.77 (95% confidence interval (CI): 0.65, 0.91) in 4 studies that compared incident users with nonusers, 0.70 (95% CI: 0.64, 0.78) in 13 studies that compared a combination of prevalent and incident users with nonusers, and 0.54 (95% CI: 0.45, 0.66) in 13 studies that compared prevalent users with nonusers. The corresponding hazard ratio from 18 RCTs was 0.84 (95% CI: 0.77, 0.91). It appears that the greater the proportion of prevalent statin users in observational studies, the larger the discrepancy between observational and randomized estimates.
PMID:22223710
CER Scan [published within the last 30 days]
The "best balance" allocation led to optimal balance in cluster-controlled trials. de Hoop E, Teerenstra S, van Gaal BG, Moerbeek M, Borm GF. Department of Epidemiology, Biostatistics and HTA, 133, Radboud University Nijmegen Medical Centre, PO Box 9101, 6500 HB Nijmegen, The Netherlands.
OBJECTIVE: Balance of prognostic factors between treatment groups is desirable because it improves the accuracy, precision, and credibility of the results. In cluster-controlled trials, imbalance can easily occur by chance when the number of cluster is small. If all clusters are known at the start of the study, the "best balance" allocation method (BB) can be used to obtain optimal balance. This method will be compared with other allocation methods.
STUDY DESIGN AND SETTING: We carried out a simulation study to compare the balance obtained with BB, minimization, unrestricted randomization, and matching for four to 20 clusters and one to five categorical prognostic factors at cluster level.
RESULTS: BB resulted in a better balance than randomization in 13-100% of the situations, in 0-61% for minimization, and in 0-88% for matching. The superior performance of BB increased as the number of clusters and/or the number of factors increased.
CONCLUSION: BB results in a better balance of prognostic factors than randomization, minimization, stratification, and matching in most situations. Furthermore, BB cannot result in a worse balance of prognostic factors than the other methods. Copyright © 2012 Elsevier Inc. All rights reserved.
PMID: 21840173
Challenges in designing comparative-effectiveness trials for antidepressants. Leon AC. Departments of Psychiatry and Public Health, Weill Cornell Medical College, New York, New York, USA.
Comparative-effectiveness antidepressant trials offer promise to provide empirical evidence for clinicians choosing among interventions. Whether such trials posit superiority or noninferiority (NI) hypotheses, they pose formidable challenges. For instance, if meaningful antidepressant differences are seen in comparative-superiority trials, they will be small. NI hypothesis testing, on the other hand, requires an a priori NI margin and evidence of trial assay sensitivity. Either design demands unusually large samples, which could render such trials infeasible.
PMID: 22261683 [PubMed - in process]
FEBRUARY THEME: Selected Methods Manuscripts from the Pharmacoepidemiology and Drug Safety Mini-Sentinel Supplement
The Mini-Sentinel is a pilot program that is developing methods, tools, resources, policies, and procedures to facilitate the use of routinely collected electronic healthcare data to perform active surveillance of the safety of marketed medical products, including drugs, biologics, and medical devices. The U.S. Food and Drug Administration (FDA) initiated the program in 2009 as part of its Sentinel Initiative, in response to a Congressional mandate in the FDA Amendments Act of 2007. After two years, Mini-Sentinel includes 31 academic and private organizations. It has developed policies, procedures, and technical specifications for developing and operating a secure distributed data system comprised of separate data sets that conform to a common data model covering enrollment, demographics, encounters, diagnoses, procedures, and ambulatory dispensing of prescription drugs. The distributed data sets currently include administrative and claims data from 2000 to 2011 for over 300 million person-years, 2.4 billion encounters, 38 million inpatient hospitalizations, and 2.9 billion dispensings. Selected laboratory results and vital signs data recorded after 2005 are also available. There is an active data quality assessment and characterization program, and eligibility for medical care and pharmacy benefits is known. Systematic reviews of the literature have assessed the ability of administrative data to identify health outcomes of interest, and procedures have been developed and tested to obtain, abstract, and adjudicate full-text medical records to validate coded diagnoses. Mini-Sentinel has also created a taxonomy of study designs and analytical approaches for many commonly occurring situations, and it is developing new statistical and epidemiologic methods to address certain gaps in analytic capabilities. Assessments are performed by distributing computer programs that are executed locally by each data partner. The system is in active use by FDA, with the majority of assessments performed using customizable, reusable queries (programs). Prospective and retrospective assessments that use customized protocols are conducted as well. To date, several hundred unique programs have been distributed and executed. Current activities include active surveillance of several drugs and vaccines, expansion of the population, enhancement of the common data model to include additional types of data from electronic health records and registries, development of new methodologic capabilities, and assessment of methods to identify and validate additional health outcomes of interest. Copyright © 2012 John Wiley & Sons, Ltd.
Link to Free PDF: http://onlinelibrary.wiley.com/doi/10.1002/pds.2343/pdf
Successful implementation of a program of active safety surveillance of drugs and medical products depends on public trust. This article summarizes how the initial pilot phase of the FDA’s Sentinel Initiative, Mini-Sentinel, is being conducted in compliance with applicable federal and state laws. The article also sets forth the attributes of Mini-Sentinel that enhance privacy and public trust, including the use of a distributed data system (where identifiable information remains at the data partners) and the adoption by participants of additional mandatory policies and procedures implementing fair information practices. The authors conclude by discussing the implications of this model for other types of secondary health data uses. Copyright © 2012 John Wiley & Sons, Ltd.
Link to Free PDF: http://onlinelibrary.wiley.com/doi/10.1002/pds.2319/pdf
Purpose: We describe the design, implementation, and use of a large, multiorganizational distributed database developed to support the Mini-Sentinel Pilot Program of the US Food and Drug Administration (FDA). As envisioned by the US FDA, this implementation will inform and facilitate the development of an active surveillance system for monitoring the safety of medical products (drugs, biologics, and devices) in the USA.
Methods: A common data model was designed to address the priorities of the Mini-Sentinel Pilot and to leverage the experience and data of participating organizations and data partners. A review of existing common data models informed the process. Each participating organization designed a process to extract, transform, and load its source data, applying the common data model to create the Mini-Sentinel Distributed Database. Transformed data were characterized and evaluated using a series of programs developed centrally and executed locally by participating organizations. A secure communications portal was designed to facilitate queries of the Mini-Sentinel Distributed Database and transfer of confidential data, analytic tools were developed to facilitate rapid response to common questions, and distributed querying software was implemented to facilitate rapid querying of summary data.
Results: As of July 2011, information on 99 260 976 health plan members was included in the Mini-Sentinel Distributed Database. The database includes 316 009 067 person-years of observation time, with members contributing, on average, 27.0 months of observation time. All data partners have successfully executed distributed code and returned findings to the Mini-Sentinel Operations Center.
Conclusion: This work demonstrates the feasibility of building a large, multiorganizational distributed data system in which organizations retain possession of their data that are used in an active surveillance system. Copyright © 2012 John Wiley & Sons, Ltd.
Link to Free PDF: http://onlinelibrary.wiley.com/doi/10.1002/pds.2336/pdf
Distributed medical product safety monitoring systems such as the Sentinel System, to be developed as a part of Food and Drug Administration’s Sentinel Initiative, will require automation of large parts of the safety evaluation process to achieve the necessary speed and scale at reasonable cost without sacrificing validity. Although certain functions will require investigator intervention, confounding control is one area that can largely be automated. The high-dimensional propensity score (hd-PS) algorithm is one option for automated confounding control in longitudinal healthcare databases. In this article, we discuss the use of hd-PS for automating confounding control in sequential database cohort studies, as applied to safety monitoring systems. In particular, we discuss the robustness of the covariate selection process, the potential for over- or under-selection of variables including the possibilities of M-bias and Z-bias, the computation requirements, the practical considerations in a federated database network, and the cases where automated confounding adjustment may not function optimally. We also outline recent improvements to the algorithm and show how the algorithm has performed in several published studies. We conclude that despite certain limitations, hd-PS offers substantial advantages over non-automated alternatives in active product safety monitoring systems. Copyright © 2012 John Wiley & Sons, Ltd.
Link to Free PDF: http://onlinelibrary.wiley.com/doi/10.1002/pds.2328/pdf
Purpose: To assess case-only designs for surveillance with administrative databases.
Methods: We reviewed literature on two designs that are observational analogs to crossover experiments: the self-controlled case series (SCCS) and the case-crossover (CCO) design.
Results: SCCS views the ‘experiment’ prospectively, comparing outcome risks in windows with different exposures. CCO retrospectively compares exposure frequencies in case and control windows. The main strength of case-only designs is they entail self-controlled analyses that eliminate confounding and selection bias by time-invariant characteristics not recorded in healthcare databases. They also protect privacy and are computationally efficient, as they require fewer subjects and variables. They are better than cohort designs for investigating transient effects of accurately recorded preventive agents, for example, vaccines. They are problematic if timing of self-administration is sporadic and dissociated from dispensing times, for example, analgesics. They tend to have less exposure misclassification bias and time-varying confounding if exposures are brief. Standard SCCS designs are bidirectional (using time both before and after the first exposure event), so they are more susceptible than CCOs to reverse-causality bias, including immortal-time bias. This is true also for sequence symmetry analysis, a simplified SCCS. Unidirectional CCOs use only time before the outcome, so they are less affected by reverse causality but susceptible to exposure-trend bias. Modifications of SCCS and CCO partially deal with these biases. The head-to-head comparison of multiple products helps to control residual biases.
Conclusion: The case-only analyses of intermittent users complement the cohort analyses of prolonged users because their different biases compensate for one another. Copyright © 2012 John Wiley & Sons, Ltd.
Link to Free PDF: http://onlinelibrary.wiley.com/doi/10.1002/pds.2330/pdf
Purpose: Many challenges arise when conducting a sequentially monitored medical product safety surveillance evaluation using observational electronic data captured during routine care. We review existing sequential approaches for potential use in this setting, including a continuous sequential testing method that has been utilized within the Vaccine Safety Datalink (VSD) and group sequential methods, which are used widely in randomized clinical trials.
Methods: Using both simulated data and preliminary data from an ongoing VSD evaluation, we discuss key sequential design considerations, including sample size and duration of surveillance, shape of the signaling threshold, and frequency of interim testing.
Results and Conclusions: All designs control the overall Type 1 error rate across all tests performed, but each yields different tradeoffs between the probability and timing of true and false positive signals. Designs tailored to monitor efficacy outcomes in clinical trials have been well studied, but less consideration has been given to optimizing design choices for observational safety settings, where the hypotheses, population, prevalence and severity of the outcomes, implications of signaling, and costs of false positive and negative findings are very different. Analytic challenges include confounding, missing and partially accrued data, high misclassification rates for outcomes presumptively defined using diagnostic codes, and unpredictable changes in dynamically accessed data over time (e.g., differential product uptake). Many of these factors influence the variability of the adverse events under evaluation and, in turn, the probability of committing a Type 1 error. Thus, to ensure proper Type 1 error control, planned sequential thresholds should be adjusted over time to account for these issues. Copyright © 2012 John Wiley & Sons, Ltd.
Link to Free PDF: http://onlinelibrary.wiley.com/doi/10.1002/pds.2324/pdf
Purpose: This manuscript describes the current statistical methodology available for active postmarket surveillance of pre-specified safety outcomes using a prospective incident user concurrent control cohort design with existing electronic healthcare data.
Methods: Motivation of the active postmarket surveillance setting is provided using the Food and Drug Administration’s Mini-Sentinel Pilot as an example. Four sequential monitoring statistical methods are presented including the Lan–Demets error spending approach, a matched likelihood ratio test statistic approach with the binomial MaxSPRT as a special case, the conditional sequential sampling procedure with stratification, and a generalized estimating equation regression approach using permutation. Information on the assumptions, limitations, and advantages of each approach is provided, including how each method defines sequential monitoring boundaries, what test statistic is used, and how robust it is to settings of rare events or frequent testing.
Results: A hypothetical example of how the approaches could be applied to data comparing a medical product of interest, drug A, to a concurrent control drug, drug B, is presented including providing the type of information one would have available for monitoring such drugs. Copyright © 2012 John Wiley & Sons, Ltd.
Link to Free PDF: http://onlinelibrary.wiley.com/doi/10.1002/pds.2320/pdf
Purpose: To describe a protocol for active surveillance of acute myocardial infarction (AMI) in users of a recently approved oral antidiabetic medication, saxagliptin, and to provide the rationale for decisions made in drafting the protocol.
Methods: A new-user cohort design is planned for evaluating data from at least four Mini-Sentinel data partners from 1 August 2009 (following US Food and Drug Administration’s approval of saxagliptin) through mid-2013. New users of saxagliptin will be compared in separate analyses with new users of sitagliptin, pioglitazone, long-acting insulins, and second-generation sulfonylureas. Two approaches to controlling for confounding will be evaluated: matching by exposure propensity score and stratification by AMI risk score. The primary analyses will use Cox regression models specified in a way that does not require pooling of patient-level data from the data partners. The Cox models are fit to summarized data on risk sets composed of saxagliptin users and similar comparator users at the time of an AMI. Secondary analyses will use alternative methods including Poisson regression and will explore whether further adjustment for covariates available only at some data partners (e.g., blood pressure) modifies results.
Results: The results of this study are pending.
Conclusions: The proposed protocol describes a design for surveillance to evaluate the safety of a newly marketed agent as postmarket experience accrues. It uses data from multiple partner organizations without requiring sharing of patient-level data and compares alternative approaches to controlling for confounding. It is hoped that this initial active surveillance project of the Mini-Sentinel will provide insights that inform future population-based surveillance of medical product safety. Copyright © 2012 John Wiley & Sons, Ltd.
Link to Free PDF: http://onlinelibrary.wiley.com/doi/10.1002/pds.2337/pdf
January 2012
CER Scan [Epub ahead of print]
Applying propensity scores estimated in a full cohort to adjust for confounding in subgroup analyses. Rassen JA, Glynn RJ, Rothman KJ, Setoguchi S, Schneeweiss S. Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School, Boston, MA, USA. jrassen@post.harvard.edu.
BACKGROUND: A correctly specified propensity score (PS) estimated in a cohort ("cohort PS") should, in expectation, remain valid in a subgroup population.
OBJECTIVE: We sought to determine whether using a cohort PS can be validly applied to subgroup analyses and, thus, add efficiency to studies with many subgroups or restricted data. METHODS: In each of three cohort studies, we estimated a cohort PS, defined five subgroups, and then estimated subgroup-specific PSs. We compared difference in treatment effect estimates for subgroup analyses adjusted by cohort PSs versus subgroup-specific PSs. Then, over 10 million times, we simulated a population with known characteristics of confounding, subgroup size, treatment interactions, and treatment effect and again assessed difference in point estimates. RESULTS: We observed that point estimates in most subgroups were substantially similar with the two methods of adjustment. In simulations, the effect estimates differed by a median of 3.4% (interquartile (IQ) range 1.3-10.0%). The IQ range exceeded 10% only in cases where the subgroup had < 1000 patients or few outcome events. CONCLUSIONS: Our empirical and simulation results indicated that using a cohort PS in subgroup analyses was a feasible approach, particularly in larger subgroups. Copyright © 2011 John Wiley & Sons, Ltd.
PMID: 22162077 [PubMed - as supplied by publisher]
Extension of the modified Poisson regression model to prospective studies with correlated binary data. Zou GY, Donner A. Department of Epidemiology & Biostatistics, and Robarts Clinical Trials of Robarts Research Institute, Schulich School of Medicine & Dentistry, Canada.
The Poisson regression model using a sandwich variance estimator has become a viable alternative to the logistic regression model for the analysis of prospective studies with independent binary outcomes. The primary advantage of this approach is that it readily provides covariate-adjusted risk ratios and associated standard errors. In this article, the model is extended to studies with correlated binary outcomes as arise in longitudinal or cluster randomization studies. The key step involves a cluster-level grouping strategy for the computation of the middle term in the sandwich estimator. For a single binary exposure variable without covariate adjustment, this approach results in risk ratio estimates and standard errors that are identical to those found in the survey sampling literature. Simulation results suggest that it is reliable for studies with correlated binary data, provided the total number of clusters is at least 50. Data from observational and cluster randomized studies are used to illustrate the methods.
PMID: 22072596 [PubMed - as supplied by publisher]
Treating Depression After Initial Treatment Failure: Directly Comparing Switch and Augmenting Strategies in STAR*D. Gaynes BN, Dusetzina SB, Ellis AR, Hansen RA, Farley JF, Miller WC, Stürmer T. Department of Psychiatry, School of Medicine, UNC at Chapel Hill, Chapel Hill, NC; Department of Health Care Policy, Harvard Medical School, Boston, MA; Harrison School of Pharmacy, Auburn University, Auburn, AL; Division of Pharmaceutical Outcomes and Policy, Eshelman School of Pharmacy, and Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC.
OBJECTIVE: Augmenting and switching antidepressant medications are the 2 most common next-step strategies for depressed patients failing initial medication treatment. These approaches have not been directly compared; thus, our objectives are to compare outcomes for medication augmentation versus switching for patients with major depressive disorder in the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) clinical trial. METHODS: We conducted a retrospective analysis of participants aged 18 to 75 years with DSM-IV nonpsychotic depression who failed to remit with initial treatment in the STAR*D clinical trial (N =1292). We compared depressive symptom remission, response, and quality of life among participants in each study arm using propensity score matching to minimize selection bias. RESULTS: The propensity-score-matched augment (N = 269) and switch (N = 269) groups were well balanced on measured characteristics. Neither the likelihood of remission (risk ratio, 1.14; 95% confidence level, 0.82-1.58) or response (risk ratio, 1.14; 95% confidence level, 0.82-1.58), nor the time to remission (log-rank test, P = 0.946) or response (log-rank test, P = 0.243) differed by treatment strategy. Similarly, quality of life did not differ. Post hoc analyses suggested that augmentation improved outcomes for patients tolerating 12 or more weeks of initial treatment and those with partial initial treatment response. CONCLUSIONS: For patients receiving and tolerating aggressive initial antidepressant trials, there is no clear preference for next-step augmentation versus switching. Findings tentatively suggest that those who complete an initial treatment of 12 weeks or more and have a partial response with residual mild depressive severity may benefit more from augmentation relative to switching.
PMID: 22198447 [PubMed - as supplied by publisher]
Variation in Antipsychotic Treatment Choice Across US Nursing Homes. Huybrechts KF, Rothman KJ, Brookhart MA, Silliman RA, Crystal S, Gerhard T, Schneeweiss S. Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School; Department of Epidemiology, Boston University School of Public Health, Boston, MA; RTI Health Solutions, Research Triangle Park; UNC, Gillings School of Global Public Health, Chapel Hill, NC; Department of Medicine, Boston University School of Medicine, Boston, MA; and Rutgers University, New Brunswick, NJ.
OBJECTIVE: Despite serious safety concerns, antipsychotic medications continue to be used widely in US nursing homes. The objective of this study was to quantify the variation in antipsychotic treatment choice across US nursing homes, and to characterize its correlates.
METHODS: Prescribing practices were assessed in a cohort of 65,618 patients 65 years or older in 45 states who initiated treatment with an antipsychotic medication after nursing home admission between 2001 and 2005, using merged Medicaid; Medicare; Minimum Data Set; and Online Survey, Certification, and Reporting data. We fit mixed-effects logistic regression models to examine how antipsychotic treatment choice at the patient-level depends on patient and nursing home fixed and random effects. RESULTS: Among antipsychotic medication users, 9% of patients initiated treatment with a conventional agent. After adjustment for case-mix and facility characteristics, 95% of nursing homes had a predicted conventional antipsychotic prescribing rate between 2% and 20%. Individually, patient characteristics accounted for 36% of the explained variation, facility characteristics for 23%, and nursing home prescribing tendency for 81%. Results were consistent in the subgroup of nursing home patients with a diagnosis of dementia. The prescribing physician was not considered as a determinant of treatment choice owing to data limitations.
CONCLUSION: These findings indicate that antipsychotic treatment choice is to some extent influenced by a nursing home’s underling prescribing "culture." This culture may reveal strategies for targeting quality improvement interventions. In addition, these findings suggest that a nursing home’s tendency for specific antipsychotics merits further exploration as an instrumental variable for improved confounding adjustment in comparative effectiveness studies.
PMID: 22198446 [PubMed - as supplied by publisher]
Diagnosing imputation models by applying target analyses to posterior replicates of completed data. He Y, Zaslavsky AM. Department of Health Care Policy, Harvard Medical School, Boston, MA, 02115, USA. he@hcp.med.harvard.edu.
Multiple imputation fills in missing data with posterior predictive draws from imputation models. To assess the adequacy of imputation models, we can compare completed data with their replicates simulated under the imputation model. We apply analyses of substantive interest to both datasets and use posterior predictive checks of the differences of these estimates to quantify the evidence of model inadequacy. We can further integrate out the imputed missing data and their replicates over the completed-data analyses to reduce variance in the comparison. In many cases, the checking procedure can be easily implemented using standard imputation software by treating re-imputations under the model as posterior predictive replicates. Thus, it can be applied for non-Bayesian imputation methods. We also sketch several strategies for applying the method in the context of practical imputation analyses. We illustrate the method using two real data applications and study its property using a simulation. Copyright © 2011 John Wiley & Sons, Ltd.
PMID: 22139814 [PubMed - as supplied by publisher]
CER Scan [published within the last 30 days]
Is probabilistic bias analysis approximately bayesian? Maclehose RF, Gustafson P. From the Divisions of Biostatistics, and Epidemiology and Community Health, University of Minnesota, Minneapolis, MN; and Department of Statistics, University of British Columbia, Vancouver, British Columbia, Canada.
Case-control studies are particularly susceptible to differential exposure misclassification when exposure status is determined following incident case status. Probabilistic bias analysis methods have been developed as ways to adjust standard effect estimates based on the sensitivity and specificity of exposure misclassification. The iterative sampling method advocated in probabilistic bias analysis bears a distinct resemblance to a Bayesian adjustment; however, it is not identical. Furthermore, without a formal theoretical framework (Bayesian or frequentist), the results of a probabilistic bias analysis remain somewhat difficult to interpret. We describe, both theoretically and empirically, the extent to which probabilistic bias analysis can be viewed as approximately Bayesian. Although the differences between probabilistic bias analysis and Bayesian approaches to misclassification can be substantial, these situations often involve unrealistic prior specifications and are relatively easy to detect. Outside of these special cases, probabilistic bias analysis and Bayesian approaches to exposure misclassification in case-control studies appear to perform equally well.
PMID: 22157311 [PubMed - in process]
Evaluation of an automated safety surveillance system using risk adjusted Sequential Probability Ratio Testing. Matheny ME, Normand SL, Gross TP, Marinac-Dabic D, Loyo-Berrios N, Vidi VD, Donnelly S, Resnic FS.
BACKGROUND: Automated adverse outcome surveillance tools and methods have potential utility in quality improvement and medical product surveillance activities. Their use for assessing hospital performance on the basis of patient outcomes has received little attention. We compared risk-adjusted sequential probability ratio testing (RA-SPRT) implemented in an automated tool to Massachusetts public reports of 30-day mortality after isolated coronary artery bypass graft surgery. METHODS: A total of 23,020 isolated adult coronary artery bypass surgery admissions performed in Massachusetts hospitals between January 1, 2002 and September 30, 2007 were retrospectively re-evaluated. The RA-SPRT method was implemented within an automated surveillance tool to identify hospital outliers in yearly increments. We used an overall type I error rate of 0.05, an overall type II error rate of 0.10, and a threshold that signaled if the odds of dying 30-days after surgery was at least twice than expected. Annual hospital outlier status, based on the state-reported classification, was considered the gold standard. An event was defined as at least one occurrence of a higher-than-expected hospital mortality rate during a given year. RESULTS: We examined a total of 83 hospital-year observations. The RA-SPRT method alerted 6 events among three hospitals for 30-day mortality compared with 5 events among two hospitals using the state public reports, yielding a sensitivity of 100% (5/5) and specificity of 98.8% (79/80). CONCLUSIONS: The automated RA-SPRT method performed well, detecting all of the true institutional outliers with a small false positive alerting rate. Such a system could provi