Trials in Implementation Science

I recently had the pleasure of attending the 39th Annual Meeting of the Society for Clinical Trials in Portland, Oregon. It was my first time attending, and I was quite impressed with the lineup of speakers and sessions. I was fortunate enough to present a working concept with my colleagues, Drs. Kirsty Loudon and Merrick Zwarenstein, with whom we’ve been developing an adaptation of the PRagmatic Explanatory Continuum Indicator Summary (PRECIS-2) to implementation trials. It also gave me time to reflect on trials in implementation science and ongoing NIH initiatives to enhance the stewardship of clinical trials.

Clinical trials—and randomized clinical trials in particular—have long been argued as having characteristics that seem at odds with the goals of implementation science. Traditional clinical trials are often defined as tightly-controlled studies conducted in artificial settings with limited generalizability to real-world situations, or so goes the standard argument. But is that true? Is it fair to make such a broad statement? And, perhaps more importantly, is it accurate? Should implementation science eschew clinical trials?

Over the past few years, I’ve been involved in several initiatives related to clinical trials here at the NCI, and am increasingly convinced of the important use of trials to advance the field of implementation science. I’m also a bit surprised (and concerned) that some members of the research community have such a negative perception of trials, but recognize that this may stem from how one defines and conceptualizes a clinical trial.  These definitions vary by scientific discipline and funding agency.

Per NIH (published in 2014 and still applicable today), a clinical trial is defined as follows: A research study in which one or more human subjects are prospectively assigned to one or more interventions (which may include placebo or other control) to evaluate the effects of those interventions on health-related biomedical or behavioral outcomes. Importantly, this definition does not limit clinical trials to one specific study design (i.e., only randomized controlled trials) or one type of intervention (i.e., only drugs), but encompasses a variety of experimental (e.g., cluster randomized) and quasi-experimental (e.g., interrupted time series, regression discontinuity) study designs and types of interventions (e.g., drugs, behavioral). Importantly, this conceptualization does not implicitly suggest that all clinical trials maximize internal validity to the detriment of external validity.

Clinical trials have an important role to play in advancing implementation science. If one uses the definition above, clinical trials are not limited to well-controlled, “artificial” settings. In fact, there are different types of clinical trials that can be designed to more accurately reflect ‘real-world’ settings, and specific criteria for doing so. In 1967, Schwartz and Lellouch published an article, Explanatory and pragmatic attitudes in therapeutical trials. This idea—of conceptualizing the purpose of trials along the explanatory--pragmatic continuum—gained significant traction in the 2000s, and has made its way across disciplines in health-focused research. The original PRECIS tool, and revised PRECIS-2 tool, have been instrumental in defining and operationalizing characteristics of trials across the explanatory--pragmatic continuum. Specifically, PRECIS-2 outlines 9 domains trialists should consider when planning their studies, and how scores on those domains reflect trials that take a more explanatory or a more pragmatic approach. Indeed, PRECIS-2 domains go well-beyond the (incorrect) interpretation that anything done in the ‘real-world,’ or any study that doesn’t involve randomization, is automatically a pragmatic study. Some of this misclassification was recently highlighted in an article by Dal-Ré and colleagues, to which our own Dr. Gila Neta—given her expertise in pragmatic trials and a 2016 co-authored article on applying PRECIS-2 criteria to NIH Collaboratory trials—published a correspondence with Dr. Karin Johnson, outlining the challenges and opportunities of using more systematic approaches (i.e., the PRECIS-2 tool) for assessing trials along the explanatory--pragmatic continuum.

Trials have a purpose in advancing the field of implementation science. They are uniquely positioned to answer research questions that other designs (i.e., observational) are ill-equipped to answer, for example studies that investigate the comparative impact of different strategies to implement effective interventions within diverse practice settings. Trials that are more pragmatic in attitude may be particularly appropriate for implementation science, although explanatory trials should not be shunned entirely, as they still have their place in helping to answer different types of research questions. Using PRECIS-2 as a guide for ensuring the pragmatic nature of a trial may be particularly applicable to hybrid designs, where studying the effectiveness (vs. efficacy) of an intervention is a primary or secondary objective.

Of note, we are currently working on several initiatives related to the explanatory-pragmatic continuum of trials and implementation science, building on previous work from members of our team (i.e., Gila Neta, David Chambers, Margaret Farrell, and our previous IS Director, Russ Glasgow). We are also working with colleagues from the Division of Cancer Control and Population Sciences and the Division of Cancer Prevention to better understand how the explanatory--pragmatic continuum may be applicable to trials across the cancer control continuum. Stay tuned for more information as these initiatives get underway.

In closing, we hope that some of our efforts to advance the concept of pragmatic trials in implementation science will be helpful to researchers, practitioners, and policymakers, and increase the likelihood that results from implementation trials are applicable to and of interest among end-users. As always, we look forward to your feedback on these issues and suggestions for next steps.