Healthcare Delivery Research Program

Research to Improve Cancer Care Outcomes and Enhance the Patient Experience

In 2015, NCI created the Healthcare Delivery Research Program (HDRP) within DCCPS to advance innovative research to improve the delivery of cancer-related care and achieve optimal health outcomes for individuals, families, and communities affected by cancer. HDRP is organized into three branches, which spearhead scientific initiatives at the patient, healthcare system, and community levels.

Approaches to Advancing the Science of Healthcare Delivery

HDRP advances healthcare delivery research by developing funding opportunities to stimulate research in the extramural community on key issues and priorities for the field and by providing programmatic oversite of grants and contracts. Collectively, HDRP fosters research across the cancer control continuum. Recent major scientific initiatives include research aimed at improving cancer screening (i.e., Population-based Research to Optimize the Screening PRocess [PROSPR], Accelerating Colorectal Cancer Screening and follow-up through Implementation Science [ACCSIS]), delivery of cancer treatment (Centers on Telehealth Research for Cancer-Related Care [RFA-CA-21-029]), and survivorship care (Improving the Management of symPtoms during And following Cancer Treatment [IMPACT]).

Major initiatives in HDRP also center around the development and support of research infrastructure, data resources, and measurement tools. For example, the NCI Community Oncology Research Program Cancer Care Delivery Research, a nation-wide network of over 1,000 community-based hospitals and practices, provides unique infrastructure to support research to improve care delivery in community hospitals and practices.

Data resources, such as linkages between Surveillance, Epidemiology, and End Results (SEER) Program data and CMS data, provide population-based information about the utilization and cost of care (SEER-Medicare), experience with care (SEER-Consumer Assessment of Healthcare Providers and Systems [CAHPS]), and health-related quality of life (SEER-Medical Health Outcomes Survey [MHOS]) among older adults with cancer as well as how enrollment in Medicaid impacts cancer outcomes (SEER-Medicaid). Supplements to nationally representative surveys — for example, the Cancer Control Supplement to the National Health Interview Survey (NHIS) and the Experiences with Cancer Survivorship Supplement to the Medical Expenditure Panel Survey (MEPS) — leverage existing data to support research on healthcare access, utilization, cost, and cancer control more broadly.

Measurement tools, such as the Patient Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE™) Measurement System, were developed to evaluate symptomatic toxicities by self-report in adults, adolescents, and children participating in cancer clinical trials, thereby complementing data captured through the CTCAE. Additionally, the HealthMeasures resource provides state-of-the-science assessment of physical, mental, and social health, as well as performance measures of cognition, motor, and sensory function, for use across a range of chronic diseases and the general population.

The future of healthcare delivery research depends on a pipeline of well-trained multidisciplinary investigators. Building the workforce of investigators who conduct healthcare delivery research is central to the mission of HDRP. To foster the development of mid-career investigators, HDRP partners with the NCI/AcademyHealth Healthcare Delivery Research Program to host the Visiting Scholars Program, whereby investigators are recruited to collaborate with HDRP staff and lead research to advance healthcare delivery. To support continuing education, HDRP also periodically develops and sponsors training opportunities to address emerging methodological topics. A recent example is the Multilevel Intervention Training Institute (MLTI), first offered in 2020.

Future Directions

As cancer treatment continues to evolve, so does the social, political, and economic landscape in which that treatment is delivered. Over the next two years, programmatic activities will be guided by a set of cross-cutting research priorities. The first priority is to develop and support efforts to improve the use of digital health data both as a source of information for care delivery research and as a tool for improving cancer-related care. Recognizing the rapid expansion in use of telehealth during the COVID-19 pandemic, HDRP also seeks to support research demonstrating how telehealth can be used to improve patient and care delivery outcomes. The second priority is to expand research evaluating the impact of legislation, regulation, and reimbursement on cancer-related care delivery and outcomes. Given the negative impact of rising costs at the individual level, HDRP also seeks to expand research to understand and address the impact of the ability to afford care on receipt of recommended care and patient and caregiver outcomes. Although HDRP has long advocated for research that addresses disparities in cancer care and the needs of vulnerable and underserved communities, events in 2020 emphasized the persistence of systemic racism and social disparities in American society. HDRP’s third cross-cutting priority is to promote research aimed at increasing knowledge of the role of systemic racism in cancer care delivery and promoting research that effectively addresses the impact of social disparities on delivery of high-quality cancer care.

Healthcare Assessment Research Branch: Data Resources for Healthcare Delivery and Health Economics Research

The Healthcare Assessment Research Branch (HARB) promotes population-based research on demographic, social, economic, provider, and health system factors as they relate to providing screening, treatment, and survivorship services for cancer. Key focuses of the branch are to understand patterns of cancer-related healthcare in community settings and to develop, improve, and disseminate data resources to support economic and health services research across the cancer continuum. Below are examples of resources that are currently or have been funded by DCCPS.

SEER-CMS Linkages

SEER cancer registry data and Medicare enrollment and claims data from CMS were first linked in 1991. In the ensuing three decades, SEER-Medicare has probably been the most influential resource available for cancer-related healthcare delivery research. Recent enhancements to this suite of databases include CAHPS, which measures Medicare patients’ care experiences, and MHOS, which provides health-related quality-of-life information about Medicare Advantage enrollees; SEER-CAHPS and SEER-MHOS are led by HDRP’s Outcomes Research Branch (ORB). Most recently, SEER-Medicaid has expanded our ability to study non-Medicare populations via a SEER linkage with national Medicaid enrollment data. Other enhancements to SEER-CMS linkages are highlighted in a May 2020 JNCI Monograph. In the future, we intend to expand available information regarding provider- and system-level characteristics, and to incorporate new sources of data regarding social determinants of health, economic position, and financial distress.

Data Collected as Part of Routine Clinical Care

While SEER-Medicare and other claims-based databases have been a mainstay of cancer-related healthcare delivery research, they cannot capture certain features of care including, for example, results of screening or laboratory tests or patient-reported symptoms. DCCPS has also supported consortia that capture more detailed data from healthcare as delivered in community settings. These include registries such as the Breast Cancer Surveillance Consortium, first developed in response to the 1992 Mammography Quality Standards Act, which mandated the development of a surveillance system to monitor breast cancer screening. Additionally, in 1998 NCI partnered with the HMO Research Network (now the Health Care Systems Research Network) to create the Cancer Research Network (CRN), a consortium of healthcare systems with defined patient populations that extract data from patient health records for research purposes. CRN research includes not only descriptive studies of patterns of cancer-related care, but also design and testing of interventions to improve care delivery. The CRN pioneered development and use of the Virtual Data Warehouse (VDW), a set of common data standards that has served as a model for numerous other distributed research networks, including PROSPR, an HDRP-funded network studying cervical, colorectal, and lung cancer screening in 10 different healthcare systems across the United States.

Patterns of Care

The Patterns of Care (POC) study was created in 1987 in response to a congressional mandate to evaluate the dissemination of state-of-the-art cancer therapy into community oncology practice, as well as to assess patient-, provider-, and system-level factors associated with receipt of specific therapies. For a subset of cancer cases in the SEER registries, more detailed treatment data are collected and verified. The study includes two to three different cancer sites annually; the current cycle focuses on kidney and urinary bladder cancers. More recently, we have broadened the types of data collected, including symptom assessment, patient-provider cost discussions, receipt of multidisciplinary care, and impacts of drug shortages on cancer treatment patterns, and have oversampled cases testing positive for specific tumor mutations. While POC data are not routinely available to researchers outside of NCI and SEER investigators, the recently developed POC*Explorer website allows for simple data queries to generate tabular and graphical summaries of POC data.

National Surveys

DCCPS has supported multiple surveys to obtain nationally representative estimates of the utilization of cancer-related healthcare services and prevention behaviors. In partnership with CDC, the Cancer Control Supplement to the NHIS has assessed cancer screening, diet and nutrition, physical activity, and tobacco use and control since 2000. Additionally, the Cancer Survivorship Supplement to MEPS, conducted in collaboration with AHRQ and other non-federal partners, addresses issues related to financial burden, access to care, and employment among cancer survivors.

For over 30 years, HARB and its predecessors have pursued the development of multiple types of new data sources. As availability of data to evaluate cancer-related healthcare delivery continues to expand rapidly, our goal is to enhance and improve resources for the extramural community.

Health Systems and Interventions Research Branch: Understanding Multilevel Contextual Factors Related to Processes and Outcomes of Care

The Health Systems and Interventions Research Branch (HSIRB) studies the context of healthcare delivery and its effect on people, from healthy people being vaccinated and screened, to clinicians, administrators, policymakers, cancer patients, family members, and cancer survivors. Our focus on these multiple levels is a deliberate departure from behavioral interventions focused on individuals and occasionally their clinicians (Meissner et al). Our shift to a multilevel focus can be traced to 2010, when a group of NCI scientists, led by Stephen Taplin, began planning to conceptualize, measure, and intervene on a range of new targets, including healthcare organizations and care teams (Edwards et al, Taplin et al).

From 2012–2014, this DCCPS-based group published funding opportunities calling for multilevel interventions, but few projects were funded. Challenges included inadequate investigator training, reviewer expectations, and lack of validated analytic methods and measures. Nevertheless, NCI and other funders, such as the Patient-Centered Outcomes Research Institute (PCORI), continued to see a need for intervening on more than an individual level.

HSIRB continued this focus on healthcare organizations and systems, interplay across levels, and multilevel interventions from its inception in early 2015. The branch began with expertise in behavioral interventions and epidemiology, and quickly added staff who study IT, team performance, and organizational measurement. At present, our expertise includes clinical practice, industrial/organizational psychology, public health behavioral and administrative sciences, social psychology, health services, and health disparities. Additional expertise may come from anthropology, sociology, and system science.

This deliberate combination of training and experience allows us to advance innovations across a range of healthcare processes. These processes include IT, where HSIRB has used IT both to extract data describing patterns in treatments (Chen et al) and as a tool to support delivery of care for cancer patients with depression (PA-18-492, PA-18-493), and follow-up to abnormal cancer screening tests (PA-17-495). The performance of clinical care teams is another area where HSIRB is innovating. Our staff have developed models to describe processes used by teams in cancer care (Verhoeven et al); identified essential features of training cancer care teams (Chollette et al); and brought together academic team scientists with clinicians and cancer care researchers to create unique research teams to promote better care teams (NCI-ASCO Teams). Each of these processes is studied in the context of a multilevel healthcare organization.

Directly addressing the challenges of funding multilevel interventions, HSIRB developed and has twice offered training in multilevel methods and intervention through its MLTI; to our knowledge, this was the first training of its kind. Nearly all the funding opportunities offered by HSIRB now require a multilevel perspective. These include announcements for research on the HPV vaccine (PAR-19-358, PAR-19-359, PAR-19-360), on multi-team clinician systems (NOT-CA-19-059), on follow-up to abnormal cancer screening tests (PA-17-495), on de-implementation of low-value care (NOT-CA-20-021, PA-18-005, PA-18-015) on implementing colorectal cancer screening and follow-up (ACCSIS), and on care coordination to improve care of cancer patients with depression (PA-18-493).

Focus on organizational characteristics and their effect on cancer prevention and control outcomes has been increasing in HSIRB. These characteristics go beyond aggregations of individuals, such as numbers of patients or specialists, and beyond administrative features, such as proportion of patients supported by Medicaid. Instead, in collaborations across NCI and with external investigators, HSIRB is leading work to identify influential, modifiable characteristics of healthcare organizations, such as clinical policies, physician autonomy, and staff turnover. Completed and ongoing work in this area includes the Organizational Research in Healthcare Workshop, development and implementation of organizational surveys (PROSPR II), and a Delphi survey, in development, to identify available, underused data in healthcare and organizational business for future cancer care delivery research.

Moving Forward: Flexibility to Address Rising Public Health Challenges

Our focus on context creates flexibility to study issues and outcomes across the cancer continuum and to address rising public health challenges. In 2021, we began to consider how every ongoing HSIRB project could consider systemic racism; the Delphi project underway will be the first to change midstream to increase that focus. Our systems lens allows us to move quickly toward measurement and intervention. Rural health issues and multi-morbidities are two other rising health problems that are best considered from a multilevel, system perspective, as care is inevitably distributed across providers and delivered simultaneously. In response, HSIRB supports funding opportunities in both areas (RFA-CA-18-026, RFA-CA-19-064, RFA-CA-20-004, RFA-CA-20-005).

HSIRB is committed to understanding the healthcare system in ways that promote innovative cancer prevention and control interventions. We approach this work with multidisciplinary expertise and the understanding that cancer care is delivered by interacting systems of policies, organizations, clinicians, and individuals that influence care quality, health outcomes, and our nation’s public health.

Outcomes Research Branch: Research to Facilitate Patient Engagement and Optimize Health and Well-Being

Outcomes research is a field of inquiry that studies the quality and effectiveness of healthcare and its impact on health outcomes. This research uses quantitative and qualitative methodology to examine specific components, including the organization, management, and financing of health services resultant outcomes such as the health status, functioning, and well-being of patients and populations.

In its inception, ORB focused on the development of methods and measurement to establish the best ways to capture multidimensional health outcomes that are meaningful to, and often reported by, patients. These included quality of life, functional status, and symptoms, as well as preference-based utility measures and economic costs of cancer care. ORB’s goal was to embed robust and feasible measures in clinical research. As the field of outcomes research evolved, ORB efforts became more applied and focused on care quality and outcomes as part of a larger cancer survivorship research umbrella, supporting research on patients and families from diagnosis onward. Though we continue our stewardship of methods and measurement, we also generate national and population-based outcome data, support the conduct of novel interventional research, and support the generation of an evidence base of best practices for incorporating patient-centered health outcomes in healthcare delivery.

ORB, along with many internal and external partners, has supported groundbreaking research on patient-reported care quality and health outcomes, symptom management and palliative care, and novel methodological patient-centered data capture. Examples of our efforts include data resources (SEER-MHOS, SEER-CAHPS) (Ambs et al; Mollica et al), measurement tools (PRO-CTCAE, HealthMeasures, Patient-Reported Outcomes Measurement Information System [PROMIS]) (Kluetz et al; Smith AW, Mitchell SA et al; Smith AW, Jensen RE), and initiatives focused on augmenting research on specific populations. These include pediatric and AYA survivors (RFA-CA-20-027; RFA-CA-20-028) (Adolescent & Young Adult Health Outcomes & Patient Experience Study [AYA HOPE]) (Smith AW, Keegan T et al) (Smith AW, Seibel NL et al); cancer survivors with multiple chronic conditions (Smith AW, Reeve BB et al) PAR-20-179; PAR-20-180; as well as informal cancer caregivers (Kent EE et al; PAR-19-352; PAR-19-355). We also invest in intervention research to improve healthcare delivery and outcomes across the life course, including interventions to improve outcomes in young survivors (RFA-CA-19-033) and in interventions that generate real-world data collection to expeditiously drive evidence into practice (IMPACT).

Our Future

Our evolving and planned future efforts recognize the importance of the changing healthcare environment, important shifts in use of telehealth, new healthcare payment models, health policy, and technological advancements that have made patient-centered outcomes more widely available and usable in clinical care by patients and care teams. We have continued our efforts related to novel methodological approaches, broadening this area to also consider areas of data science, such as natural language processing and machine learning. We are paying close attention to the growing use of clinical informatics and EHR-based research. We are broadening our reach to understudied populations, such as those with advanced and metastatic cancers (Mollica MA, Tesauro G et al); the aging cancer population; and historically marginalized individuals with cancer such as black, indigenous, and people of color, and LGBTQI communities; and those managing complex care due to multiple health conditions. We also continue to develop scientific goals to better understand social health and social determinants of health and their influence on patient care and health outcomes, including addressing health inequities and inadequate access to care.

Celebration of Team Science

We could not do what we do without tremendous collaboration, both within and across DCCPS and NCI, and with external partnerships. Our science, at its heart, is team-based because outcomes research is cross-cutting and interdisciplinary by design. In ORB, we have many collaborations with federal partners, national and international organizations, and professional societies. We work with scientists across disciplines, such as psychology, epidemiology, psychometrics, nursing science, medicine, and health services research.

We co-facilitate research funding and activities with colleagues across NIH institutes, centers, and offices. We are grateful for the many scientists, administrators, and fellows both within and outside of our branch, along with the survivors themselves, who have contributed their time and energy to the progress made in cancer outcomes research.

Footnotes

Meissner HI, et al. The future of research that promotes cancer screening. Cancer. 2004;101(5 SUPPL.):1251–1259.

Edwards HM, et al. Summary of the multilevel interventions in health care conference. J Natl Cancer Inst Monogr. 2012;2012(44):123–126.

Taplin SH, et al. Introduction: Understanding and influencing multilevel factors across the cancer care continuum. J Natl Cancer Inst Monogr. 2012;2012(44):2–10.

Chen R, et al. Treatment patterns for chronic comorbid conditions in patients with cancer using a large-scale observational data network. JCO Clin Cancer Informatics. 2020;4(4):171–183.

Verhoeven DC, et al. The anatomy and physiology of teaming in cancer care delivery: a conceptual framework. J Natl Cancer Inst. 2021;113(4):360–370.

Chollette V, et al. Identifying cancer care team competencies to improve care coordination in multiteam systems: a Modified Delphi Study. JCO Oncol Pract. 2020;16(11):e1324–e1331.

NCI-ASCO Teams. Special Series: NCI-ASCO Teams in Cancer Care. Journal of Oncology Practice. October 18, 2016.

Ambs A, et al. Overview of the SEER—Medicare Health Outcomes Survey linked dataset. Health Care Financ Rev. Summer 2008;29(4):5–21.

Mollica MA, et al. Patient experiences of cancer care: scoping review, future directions, and introduction of a new data resource: Surveillance Epidemiology and End Results-Consumer Assessment of Healthcare Providers and Systems (SEER-CAHPS). Patient Experience Journal. 2017;Vol. 4: Iss. 1, Article 12.

Kluetz PG, et al. Patient-reported outcomes in cancer clinical trials: measuring symptomatic adverse events with the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). American Society of Clinical Oncology Educational Book. American Society of Clinical Oncology. Annual Meeting vol. 35 (2016): 67–73.

Smith AW, et al. News from the NIH: person-centered outcomes measurement: NIH-supported measurement systems to evaluate self-assessed health, functional performance, and symptomatic toxicity. Transl Behav Med. 2016 Sep;6(3):470–4.

Smith AW, et al. Beyond methods to applied research: realizing the vision of PROMIS®. Health Psychology. 2019;38(5):347–350.

Smith AW, et al. Understanding care and outcomes in adolescents and young adult with cancer: a review of the AYA HOPE study. Pediatr Blood Cancer. 2019 Jan;66(1):e27486.

Smith AW, et al. Next steps for adolescent and young adult oncology workshop: an update on progress and recommendations for the future. Cancer. 2016;122(7):988–999.

Smith AW, et al. Cancer, comorbidities, and health-related quality of life of older adults. Health Care Financ Rev. Summer 2008;29(4):41–56.

Kent EE, et al. Caring for caregivers and patients: research and clinical priorities for informal cancer caregiving. Cancer. 2016;122:1987-1995.

Mollica MA, et al. Current state of funded National Institutes of Health grants focused on individuals living with advanced and metastatic cancers: a portfolio analysis. J Cancer Surviv. 2021 Jun;15(3):370–374.