Transdisciplinary Tobacco Use Research Centers (TTURC) Initiative
The Transdisciplinary Tobacco Use Research Centers (TTURC) initiative was funded from 1999-2009 through the Tobacco Control Research Branch. The initiative aimed to facilitate integrated transdisciplinary approaches to tobacco use research. It was the first transdisciplinary research center funded at NIH.
The SciTS Team was formed in 2001 to develop evaluation tools that could be used for future evaluations of team science grant initiatives.
TTURC Evaluation Logic Model
A logic model was created to help develop an evaluation plan and guide research questions. It was created using a concept mapping strategy and was led by William Trochim, a professor at Cornell University and an expert in program evaluation, who played a leadership role in the SciTS team at the time.
TTURC Evaluation System
A battery of 20 instruments was developed during a pilot program evaluation of the TTURC initiative. This evaluation utilized a mixed method approach and involved participation from a variety of stakeholders in an effort to explore potential methods and measures that could adequately assess the complexities of a large transdisciplinary research initiative. Details of the methods and results of this evaluation can be found in the following publication:
Trochim WM, Marcus SE, Masse LC, Moser RP, Weld PC. The evaluation of large research initiatives - A participatory integrative mixed-methods approach. Am J Eval. 2008;29(1):8-28. doi:10.1177/1098214007309280
.
TTURC Bibliometric Study
The SciTS team conducted a quasi-experimental longitudinal study comparing bibliometric indicators of productivity, collaboration, and impact of the TTURCs (as a representative of transdisciplinary center grant initiatives) and R01s (traditional investigator-initiated grants) in the same field.
Hall KL, Stokols D, Stipelman BA, et al. Assessing the value of team science: a study comparing center- and investigator-initiated grants. Am J Prev Med. 2012 Feb;42(2):157-163. doi:10.1016/j.amepre.2011.10.011.
TTURC Visualization and Global Mapping
The SciTS team worked to apply innovative visualization methodologies to the TTURC publication data. The introduction of these visualization techniques provides a richer understanding of the relationships identified through other methodological approaches (e.g., bibliometrics).
Transdisciplinary Research on Energetics and Cancer (TREC) Initiative
The Transdisciplinary Research on Energetics and Cancer (TREC) initiative (2005-2010; refunded 2011-2016) fosters transdisciplinary research in nutrition, physical activity, energy balance, obesity and cancer. The SciTS team has been involved in program evaluation efforts with the TREC initiative since it began in 2005.
TREC Evaluation Logic Model
The TREC logic model identifies the temporal relationships among key process and outcome variables related to team science including: collaborative readiness, collaborative capacity, and outcomes. The model was developed as the result of a systematic review of the literature from a variety of disciplines (e.g., team research, virtual teams, informatics, human and computer interaction) and served as the basis for a stage-dependent evaluation strategy of the TREC initiative.
TREC Baseline Researcher Survey
The TREC Baseline Researcher Survey was developed to gather baseline information on collaborative readiness and early evidence of cross-disciplinary collaboration. It included sections on respondents' demographic characteristics, history of collaboration, research orientation (i.e., degree of proclivity toward transdisciplinary collaborative research), collaborative resources, collaborative processes, collaboration activities, attitudes about other center participants, institutional transdisciplinary research culture, and training resources, activities, and opportunities.
This survey is adaptable for use in the evaluation of other large cross-disciplinary team science initiatives.
Hall KL, Stokols D, Moser RP, et al. Collaboration readiness of transdisciplinary research teams and centers: Findings from the National Cancer Institute's TREC year 1 evaluation study. Am J Prev Med. 2008;35(2S):S161-S172. doi:10.1016/j.amepre.2008.03.035.
TREC Follow-up Researcher Survey
The TREC Follow-up Researcher Survey gathered information about ongoing collaboration and training activities, processes surrounding collaboration, and the benefits and challenges of implementing and participating in TREC research and training activities. The survey included measures that focused on assessing TREC training activities and perceived benefits and challenges of collaboration and training within the TREC initiative.
Results were analyzed in conjunction with results of the baseline survey in order to assess the relationships among antecedent conditions and early collaborative processes measured in the baseline survey with collaborative outcomes and products measured in the follow-up survey.
Like the baseline survey, the follow-up survey is adaptable for use in the evaluation and monitoring of other large cross-disciplinary team science collaboration and training.
Stipelman BA, Feng A, Hall KL, et al. The relationship between collaborative readiness and scientific productivity in the Transdisciplinary Research on Energetics and Cancer (TREC) centers. Poster presented at: The 31st annual meeting of the Society of Behavioral Medicine; January, 2010; Seattle, WA.
Written Products Protocol
The Written Products Protocol was a rating instrument developed to assess the extent to which transdisciplinary integration occurred in TREC Developmental Pilot Projects. These pilot projects were developed by investigators beginning in the second year of the TREC initiative, to elaborate upon preexisting research activities within TREC, through innovative transdisciplinary approaches.
Reviewers used the instrument to rate research proposals on the following factors: (1) collaboration within or across TREC centers, (2) inclusion of multiple disciplinary perspectives, and (3) the scope of transdisciplinary integration with respect to research methods and analytical approaches.
Hall KL, Stokols D, Moser RP, et al. Collaboration readiness of transdisciplinary research teams and centers: Findings from the National Cancer Institute's TREC year 1 evaluation study. Am J Prev Med. 2008;35(2S):S161-S172. doi:10.1016/j.amepre.2008.03.035.
TREC Lessons Learned Study
As the TREC initiative neared the end of its first funding cycle (2005-2010), qualitative in-depth interviews were conducted to capitalize on knowledge gained and lessons learned by both grantees and NCI program staff about engaging in transdisciplinary center grant initiatives. The study goals were: (1) to record TREC grantees' strategies for success, challenges, and lessons learned related to engaging in a transdisciplinary center grant initiative; (2) to record TREC Coordinating Center and NCI TREC program staff members' success, challenges, and lessons learned related to supporting a transdisciplinary center grant initiative; and (3) to document broad impacts of TREC, including advances in the science of energetics and cancer; impacts on participating scientists, trainees, and academic institutions; and impacts on other NCI and NIH activities.
Interview guides were developed for different types of TREC participants, including center directors, senior and junior investigators, trainees, coordination center staff, and program staff. These guides were tailored to capture the unique perspectives of each interview participant, based on his or her unique role within TREC. Interview questions addressed diverse aspects of successful team science, including factors at the levels of the academic institution and center that facilitated team science; strategies used by investigators to facilitate team science; training benefits and impacts on professional development, and coordination center and NCI strategies to support effective transdisciplinary collaboration and cross-center collaboration.
Vogel AL, Stipelman BA, Feng A, Stokols D, Hall KL, Nebeling LC. Strategies for facilitating and supporting cross-disciplinary team science on cancer: Lessons from the National Cancer Institute's TREC initiative. Oral presentation at: 2011 American Public Health Association Conference; November, 2011; Washington DC, Washington DC.
Centers for Population Health and Health Disparities (CPHHD) Initiative
Social Network Survey
The CPHHD initiative (2003-2008; refunded 2010-2015) was initially established through a partnership of the NCI, the National Institute for Environmental Health Sciences, and the National Institute on Aging. It aims to accelerate the science on health disparities by supporting transdisciplinary approaches that could help advance our understanding of causal factors and effective interventions to address health disparities. CPHHD was renewed through a partnership between NCI, the National Heart, Lung and Blood Institute, and the Office of Behavioral and Social Sciences Research at the NIH.
During the second funding period for CPHHD, the SciTS team implemented a baseline social network and collaboration readiness survey in late 2010 and early 2010. Over 90% of the key investigators involved in CPHHD centers completed the survey. Data were collected on investigators' current and prior collaborations and on the type of each reported collaboration (e.g. co-authored publication, mentoring/trainee relationship, committee or work group participation, etc.).
A follow-up survey will be conducted toward the end of the five-year funding period in order to assess the impact of the initiative on transdisciplinary collaborations.
Okamoto J, Vogel AL, Hall KL, Srinivasan S, Shariff-Marco S. Assessing the impact of a transdisciplinary team science initiative on scientific collaboration networks. Poster presented at: The 2nd Annual International Science of Team Science (SciTS) Conference; April, 2011; Chicago, IL.
Review of Evaluations of Center and Network Initiatives at NIH
In an effort to better understand the scope of program evaluations for center and network initiatives supported by NIH, in 2010, the SciTS team contracted a review of published and unpublished evaluations of these initiatives produced over the last three decades (1978-2009). A total of 61 evaluation cases were identified, and reports and articles produced by these evaluations were analyzed. Study design characteristics, including variables used, research methods, and data sources were summarized. Implications for further program planning and evaluation design were discussed.