Skip to main content

A review of policy dissemination and implementation research funded by the National Institutes of Health, 2007–2014

Abstract

Background

Policy has a tremendous potential to improve population health when informed by research evidence. Such evidence, however, typically plays a suboptimal role in policymaking processes. The field of policy dissemination and implementation research (policy D&I) exists to address this challenge. The purpose of this study was to: (1) determine the extent to which policy D&I was funded by the National Institutes of Health (NIH), (2) identify trends in NIH-funded policy D&I, and (3) describe characteristics of NIH-funded policy D&I projects.

Methods

The NIH Research Portfolio Online Reporting Tool was used to identify all projects funded through D&I-focused funding announcements. We screened for policy D&I projects by searching project title, abstract, and term fields for mentions of “policy,” “policies,” “law,” “legal,” “legislation,” “ordinance,” “statute,” “regulation,” “regulatory,” “code,” or “rule.” A project was classified as policy D&I if it explicitly proposed to conduct research about the content of a policy, the process through which it was developed, or outcomes it produced. A coding guide was iteratively developed, and all projects were independently coded by two researchers. ClinicalTrials.gov and PubMed were used to obtain additional project information and validate coding decisions. Descriptive statistics—stratified by funding mechanism, Institute, and project characteristics—were produced.

Results

Between 2007 and 2014, 146 projects were funded through the D&I funding announcements, 12 (8.2 %) of which were policy D&I. Policy D&I funding totaled $16,177,250, equivalent to 10.5 % of all funding through the D&I funding announcements. The proportion of funding for policy D&I projects ranged from 14.6 % in 2007 to 8.0 % in 2012. Policy D&I projects were primarily focused on policy outcomes (66.7 %), implementation (41.7 %), state-level policies (41.7 %), and policies within the USA (83.3 %). Tobacco (33.3 %) and cancer (25.0 %) control were the primary topics of focus. Many projects combined survey (58.3 %) and interview (33.3 %) methods with analysis of archival data sources.

Conclusions

NIH has made an initial investment in policy D&I research, but the level of support has varied between Institutes. Policy D&I researchers have utilized a variety of designs, methods, and data sources to investigate the development processes, content, and outcomes of public and private policies.

Peer Review reports

Background

Policy has a tremendous potential to improve population health [1, 2]. The extent to which policy produces such outcomes, however, often depends upon the degree to which it is informed by scientific knowledge and aligned with evidence-based interventions [35]. Unfortunately, research typically plays a minor role in policymaking processes [6, 7]. For decades, scholars have documented myriad barriers to translating research into policy—including political influences, economic constraints, scientific uncertainty, and institutional culture [820].

The field of health policy dissemination and implementation research (policy D&I) exists to address these challenges. Policy D&I is focused on generating knowledge to effectively spread research evidence among policymakers and integrate evidence-based interventions into policy designs [5]. Policy D&I, known as policy “knowledge translation and exchange” outside of the USA, has evolved over the past 40 years. The 1970s and 1980s were marked by interest in how social science research was used in public policymaking and saw the development of key contributions to the field—such as Weiss’ typology of research evidence [7] and Caplan’s two communities theory of knowledge utilization [17]. In the 1990s and 2000s, frameworks and instruments for policy D&I emerged [2125] and the field was embraced within the discipline of public health [9, 26, 27]. Today, policy D&I is a priority of the World Health Organization [28, 29] and a major focus of journals such as Health Research Policy and Systems and Evidence & Policy.

The history of policy D&I has demonstrated that government funding is essential for a county to develop and maintain a robust policy D&I research infrastructure [22, 30]. Outside of the USA, governments have made sustained investments in policy D&I research and initiatives to promote the use of research evidence in health policymaking. Examples include the Canadian Institutes of Health Research’s integrated knowledge translation programs, the National Health and Medical Research Council of Australia’s Partnership Projects and Centres, and the UK National Institute for Health Research’s Collaboration for Leadership in Applied Health Research and Care. While national governments have invested in policy D&I abroad, policy D&I research has largely existed outside of the mainstream, government-funded health research enterprise in the USA [6]. This is potentially changing, however, as the National Institutes of Health (NIH)—the largest government funder of health research in the USA—has recently invested in the broader field of D&I research and identified policy D&I as a priority [31].

In 2002, the National Institute of Mental Health issued a Program Announcement (PAR) entitled “Dissemination and Implementation Research in Mental Health” that highlighted the importance of developing effective strategies to disseminate research findings to the policymakers [32]. In 2005, the PAR was re-issued under the broader name “Dissemination and Implementation Research in Health” (D&IRH) with more participating Institutes (e.g., the National Cancer Institute, National Institute on Drug Abuse) [33]. In 2007, the NIH Center for Scientific Review created a permanent D&IRH study section to evaluate D&I proposals, re-affirming NIH’s investment in the field [34]. In 2013, the “D&IRH” PAR was re-issued, this time with a greater emphasis on policy D&I research [35]. For example, the PAR encouraged applications that “address[ed] the complexity of bridging research, policy and practice.”

NIH has expressed interest in policy D&I research, but the extent to which it has funded research in this area is unclear. Two recent studies reviewed NIH D&I funding related to specific topics; one study focused on nursing [36] and the other focused on cancer control [37]. A similar review of policy D&I research is needed to assess growth of the field and identify funding gaps. Such a study would support growth in the field of policy D&I in the USA by providing investigators with a menu of approaches to policy D&I research that can be adopted and adapted when developing proposals. This article reports the results of a study that was conducted to address these knowledge gaps. The aims of the study were to: (1) determine the extent to which policy D&I research has been funded by NIH between 2007 and 2014, (2) identify trends in NIH-funded policy D&I research, and (3) describe the characteristics of NIH-funded policy D&I research projects.

Methods

We used the NIH Research Portfolio Online Reporting Tool [38] to identify all projects that received funding during any US fiscal year between 2007 and 2014 through the D&I funding opportunity announcements (FOAs) Dissemination and Implementation Research in Health (PARs: 06-039, 06-071, 06-072, 06-520, 06-521, 07-086, 10-038, 10-039, 10-040, 13-054, 13-055, 13-056) or Dissemination and Implementation Research in Mental Health (PA 02-131). We selected 2007 as the starting point because it was the first year that projects supported by the Dissemination and Implementation Research in Health FOAs received funding. For each project, we extracted the title, abstract, project terms, award amount per fiscal year, funding mechanism, and Institute.

Our review was guided by Bogenschneider’s definition of policy as: “the development, enactment, and implementation of a plan or course of action carried out through law, rule, code, or other mechanism in the public or private sector” [5]. Accordingly, we identified potential policy D&I projects by searching the project title, abstract, and term fields for mentions of “policy,” “policies,” “law,” “legal,” “legislation,” “ordinance,” “statute,” “regulation,” “regulatory,” “code,” or “rule.” Two coders then independently reviewed project abstracts and developed preliminary coding categories to capture project characteristics. These categories reflected themes in that data, categories used in previous NIH D&I funding reviews [36, 37], and policy D&I scholarship [36, 8, 9]. The coders then jointly developed a coding guide and independently re-reviewed and coded the projects. Because many projects were not exclusively focused on policy D&I, projects were coded according to their policy D&I features. Incongruent coding decisions were identified in <10 % of projects and resolved through discussions.

We classified a project as policy D&I if it explicitly proposed to conduct empirical research about the “content” of a policy (e.g., analysis of the text of clean indoor air laws), the “process” through which it was developed (e.g., assessment of how state legislators use research evidence when developing clean indoor air laws), or the “outcomes” it produced (e.g., evaluation of the impacts of clean indoor air laws on cardiovascular health outcomes). These inclusion criteria were informed by Bogenschneider’s definition of policy [6] and domains of health policy research proposed by Brownson and colleagues [35]. For projects classified as policy D&I, we searched for its NIH project number in ClinicalTrials.gov [39] and PubMed [40] to obtain additional information and validate coding decisions.

We calculated the total dollar amount awarded through all NIH research grants and through the D&I FOAs, stratified by funding mechanism and Institute. We also calculated the amount awarded for policy D&I projects, and the percentage of total D&I FOA funding they comprised, within strata. Data were managed and analyzed in Microsoft Excel.

Results and discussion

Between 2007 and 2014, 146 projects were funded through the D&I FOAs, 12 (8.2 %) of which were classified as policy D&I research (Fig. 1). A total of $16,177,250 was awarded for these projects, equivalent to 10.5 % of all funding through the D&I FOAs (Table 1). The NIH Office of the Director allocated 69.7 % of its D&I FOA funding to a policy D&I project, indicating agency-level support for the field. The National Cancer Institute (NCI) was the primary funder of policy D&I research, supporting six projects which comprised 13.0 % of all NCI funding through the D&I FOAs. Some Institutes that were engaged in D&I funding—such as the National Institute of Nursing Research which funded seven D&I projects comprising over 1.0 % of the Institute’s total research grant spending, a proportion larger than any other Institute—did not fund any policy D&I projects. Differences in policy D&I funding between Institutes could potentially be a reflection of varying levels of knowledge about policy D&I research among investigators within different health science disciplines or differences in the degree to which the disciplines are clinically, as opposed to policy, oriented.

Fig. 1
figure 1

Flow diagram summarizing the process used to identify policy D&I research projects funded by NIH, federal fiscal years 2007–2014. NIH National Institutes of Health, D&I dissemination and implementation, FOAs funding opportunity announcements

Table 1 NIH-funded policy D&I research projects by funding mechanism, Institute, and percentage of all research funded through D&I FOAs, federal fiscal years 2007–2014

NIH funding for policy D&I increased between 2007 and 2014 (Fig. 2). Annual policy D&I funding increased by 98.9 % within this period, from $1,584,327 in 2007 to $3,151,286 in 2014. The proportion of funding awarded through the D&I FOAs that was for policy D&I projects ranged from 14.6 % in 2007 to 8.0 % in 2011. Between 2012 and 2013, funding decreased by 6.3 % for all NIH research grants and increased by 7.4 % for projects funded through the D&I FOAs and by 51.8 % for policy D&I projects.

Fig. 2
figure 2

Annual trends in NIH funding for D&I research, federal fiscal years 2007–2014. “Policy D&I funding” and “all D&I FOA funding” in millions, “all NIH research grant funding” in billions, US dollars. NIH National Institutes of Health, D&I dissemination and implementation, FOAs funding opportunity announcements

NIH-funded policy D&I projects utilized a range of methodologies and data sources across different domains of policy research (Tables 2, 3, Additional file 1). These projects were primarily focused on policy outcomes (66.7 %), implementation research (41.7 %), state-level policies (41.7 %), and the USA (83.3 %). Tobacco (33.3 %) and cancer control (25.0 %), for which evidence-based policy strategies exist [41, 42], were the most common topics of focus. Many projects combined survey (58.3 %) and interview (33.3 %) methods with analysis of archival data sources, such as medical records (25.0 %) and policy documents (25.0 %). It was unclear; however, the extent to which projects used true mixed-method designs (e.g., the results of one method informed data collection of the other, or the results of both methods were presented together).

Table 2 NIH-funded policy D&I research projects by study characteristics, federal fiscal years 2007–2014
Table 3 NIH-funded policy D&I research projects by individual project details, federal fiscal years 2007–2014 (N = 12)

Limitations

We limited our review to projects funded through FOAs explicitly focused on D&I and may not have identified all NIH-funded policy D&I projects. Our review also did not capture policy D&I projects funded by other US government agencies or philanthropies. We did not attempt to obtain the full text of proposals and thus did not have complete information on project characteristics, such as information on how the projects planned to disseminate their findings. We also did not have data on the characteristics of policy D&I research proposals that were unsuccessful in obtaining funding through NIH. Thus, we were unable to identify specific characteristics that increased the likelihood of a policy D&I project being funded. We dichotomously categorized projects as policy D&I (yes/no) and did not differentiate between projects in which policy D&I was the primary focus or subcomponent. Because only 12 policy D&I projects were identified, subgroup differences and comparisons should be interpreted with caution.

Conclusions

NIH has made an initial investment in policy D&I research, signaling that the field might be entering the mainstream US health research enterprise. The level of support for policy D&I research, however, has varied between Institutes and is probably not commensurate with the potential of evidence-based policy to positively impact human health. Policy D&I researchers have utilized a variety of designs, methods, and data sources to investigate the development processes, content, and outcomes of public and private policies. By mapping the characteristics of NIH funded policy D&I research projects, the current study provides investigators with guidance on approaches to policy D&I research that they can consider when conceptualizing research ideas and developing NIH proposals in this area.

References

  1. Institute of Medicine. For the public’s health: revitalizing law and policy to meet new challenges. Washington, DC: The National Academies Press; 2011.

    Google Scholar 

  2. Burris S, Wagenaar AC, Swanson J, Ibrahim JK, Wood J, Mello MM. Making the case for laws that improve health: a framework for public health law research. Milbank Q. 2010;88(2):169–210.

    Article  PubMed Central  PubMed  Google Scholar 

  3. Brownson RC, Chriqui JF, Stamatakis KA. Understanding evidence-based public health policy. Am J Public Health. 2009;99(9):1576–83.

    Article  PubMed Central  PubMed  Google Scholar 

  4. Brownson RC, Seiler R, Eyler AA. Measuring the impact of public health policy. Prev Chronic Dis. 2010;7(4):A77.

    PubMed Central  PubMed  Google Scholar 

  5. Dodson E, Brownson RC, Weiss S. Policy dissemination research. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: translating science to practice. New York, NY: Oxford University Press; 2012. p. 437–58.

    Chapter  Google Scholar 

  6. Bogenschneider K, Corbett TJ. Evidence-based policymaking: insights from policy-minded researchers and research-minded policymakers. New York, NY: Routledge; 2011.

    Google Scholar 

  7. Weiss CH. The many meanings of research utilization. Pub Admin Rev. 1979;39(5):426–31.

    Article  Google Scholar 

  8. Innvær S, Vist G, Trommald M, Oxman A. Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Policy. 2002;7(4):239–44.

    Article  PubMed  Google Scholar 

  9. Brownson RC, Royer C, Ewing R, McBride TD. Researchers and policymakers: travelers in parallel universes. Am J Prev Med. 2006;30(2):164–72.

    Article  PubMed  Google Scholar 

  10. De Leeuw E, Clavier C, Breton E. Health policy—why research it and how: health political science. Health Res Policy Syst. 2014;12(1):55.

    Article  PubMed Central  PubMed  Google Scholar 

  11. Orton L, Lloyd-Williams F, Taylor-Robinson D, O’Flaherty M, Capewell S. The use of research evidence in public health decision making processes: systematic review. PLoS One. 2011;6(7):e21704.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  12. Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14(1):2.

    Article  PubMed Central  PubMed  Google Scholar 

  13. Hanney SR, Gonzalez-Block MA, Buxton MJ, Kogan M. The utilisation of health research in policy-making: concepts, examples and methods of assessment. Health Res Policy Syst. 2003;1(1):2.

    Article  PubMed Central  PubMed  Google Scholar 

  14. Liverani M, Hawkins B, Parkhurst JO. Political and institutional influences on the use of evidence in public health policy. A systematic review. PLoS One. 2013;8(10):e77404.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  15. Oliver TR. The politics of public health policy. Annu Rev Public Health. 2006;27(1):195–233.

    Article  PubMed  Google Scholar 

  16. Jewell CJ, Bero LA. “Developing good taste in evidence”: facilitators of and hindrances to evidence‐informed health policymaking in state government. Milbank Q. 2008;86(2):177–208.

    Article  PubMed Central  PubMed  Google Scholar 

  17. Caplan N. The two-communities theory and knowledge utilization. Am Behav Sci. 1979;22(3):459.

    Article  Google Scholar 

  18. Ellen ME, Lavis JN, Sharon A, Shemer J. Health systems and policy research evidence in health policy making in Israel: what are researchers’ practices in transferring knowledge to policy makers? Health Res Policy Syst. 2014;12(1):67.

    Article  PubMed Central  PubMed  Google Scholar 

  19. Lindblom CE, Cohen DK. Usable knowledge: social science and social problem solving, vol. 21. New Haven, CT: Yale University Press; 1979.

    Google Scholar 

  20. Lomas J. Diffusion, dissemination, and implementation: who should do what? Ann N Y Acad Sci. 1993;703(1):226–37.

    Article  CAS  PubMed  Google Scholar 

  21. Lester JP, Wilds LJ. The utilization of public policy analysis: a conceptual framework. Eval Program Plann. 1990;13(3):313–9.

    Article  Google Scholar 

  22. Lavis JN, Lomas J, Hamid M, Sewankambo NK. Assessing country-level efforts to link research to action. Bull World Health Organ. 2006;84(8):620–8.

    Article  PubMed Central  PubMed  Google Scholar 

  23. Boyko JA, Lavis JN, Dobbins M, Souza NM. Reliability of a tool for measuring theory of planned behaviour constructs for use in evaluating research use in policymaking. Health Res Policy Syst. 2011;9(1):29.

    Article  PubMed Central  PubMed  Google Scholar 

  24. Dobbins M, Ciliska D, Cockerill R, Barnsley J, DiCenso A. A framework for the dissemination and utilization of research for health‐care policy and practice. Worldviews Evid Based Nurs. 2002;E9(1):149–60.

    Article  Google Scholar 

  25. Glasgow RE, Green LW, Taylor MV, Stange KC. An evidence integration triangle for aligning science with policy and practice. Am J Prev Med. 2012;42(6):646–54.

    Article  PubMed Central  PubMed  Google Scholar 

  26. Samet JM. Epidemiology and policy: the pump handle meets the new millennium. Epidemiol Rev. 2000;22(1):145–54.

    Article  CAS  PubMed  Google Scholar 

  27. Petticrew M, Whitehead M, Macintyre SJ, Graham H, Egan M. Evidence for public health policy on inequalities: 1: the reality according to policymakers. J Epidemiol Community Health. 2004;58(10):811–6.

    Article  PubMed Central  PubMed  Google Scholar 

  28. Panisset U, Campbell S, Lavis J. 2012–2015 Strategic plan: towards a world in which the best available research evidence informs health policy-making. World Health Organization. 2012. http://www.who.int/evidence/OMSEVIPNetStratPlan.pdf?ua=1. Accessed 1 Dec 2015.

  29. World Health Organization. World report on knowledge for better health: strengthening health systems. 2004. http://www.who.int/rpc/meetings/en/world_report_on_knowledge_for_better_health2.pdf. Accessed 1 Dec 2015.

  30. Sibbald SL, Tetroe J, Graham ID. Research funder required research partnerships: a qualitative inquiry. Implement Sci. 2014;9(1):176.

    Article  PubMed Central  PubMed  Google Scholar 

  31. Glasgow RE, Vinson C, Chambers D, Khoury MJ, Kaplan RM, Hunter C. National Institutes of Health approaches to dissemination and implementation science: current and future directions. Am J of Public Health. 2012;102(7):1274–81.

    Article  Google Scholar 

  32. National Institutes of Health. Dissemination and implementation research in mental health. 2002. http://grants.nih.gov/grants/guide/pa-files/PA-02-131.html. Accessed 1 Dec 2015.

  33. National Institutes of Health. Dissemination and implementation research in mental health. 2005. https://grants.nih.gov/grants/guide/pa-files/PAR-06-039.html. Accessed 1 Dec 2015.

  34. Glasgow RE, Chambers DA, Cynkin L. News from the NIH: highlights in implementation science from the National Cancer Institute and the National Institute of Mental Health. Transl Behav Med. 2013;3(4):335–7.

    Article  PubMed Central  PubMed  Google Scholar 

  35. National Institutes of Health. Dissemination and implementation research in health (R01). 2013. http://grants.nih.gov/grants/guide/pa-files/PAR-13-055.html. Accessed 20 Aug 2015.

  36. Tinkle M, Kimball R, Haozous EA, Shuster G, Meize-Grochowski R. Dissemination and implementation research funded by the US National Institutes of Health, 2005–2012. Nurs Res Pract. 2013;2013:1–15.

    Article  Google Scholar 

  37. Neta G, Sanchez MA, Chambers DA, Phillips SM, Leyva B, Cynkin L, et al. Implementation science in cancer prevention and control: a decade of grant funding by the National Cancer Institute and future directions. Implement Sci. 2015;10(1):4.

    Article  PubMed Central  PubMed  Google Scholar 

  38. Research Portfolio Online Reporting Tools (RePORT). http://report.nih.gov/. Accessed 20 Aug 2015.

  39. ClinicalTrials.gov. https://clinicaltrials.gov/. Accessed 20 Aug 2015.

  40. PubMed. http://www.ncbi.nlm.nih.gov/pubmed. Accessed 20 Aug 2015.

  41. Community Preventive Services Taskforce. Guide to community preventive services. Preventing skin cancer: education and policy approaches. http://www.thecommunityguide.org/cancer/skin/education-policy/index.html. Accessed 20 Aug 2015.

  42. Community Preventive Services Taskforce. Guide to community preventive services. Reducing tobacco use and secondhand smoke exposure: smoke-free policies. http://www.thecommunityguide.org/tobacco/smokefreepolicies.html. Accessed 20 Aug 2015.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jonathan Purtle.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

JP conceptualized and designed the study. JP and RP developed the search strategy, reviewed, and coded projects. RB provided guidance and direction throughout the research process. JP drafted the manuscript and was responsible for overseeing revisions. RP and RB provided reviews of the manuscript and suggested edits that were addressed. All authors approved the final version of the manuscript.

Additional file

Additional file 1:

NIH-funded policy D&I projects. (XLSM 583 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Purtle, J., Peters, R. & Brownson, R.C. A review of policy dissemination and implementation research funded by the National Institutes of Health, 2007–2014. Implementation Sci 11, 1 (2015). https://doi.org/10.1186/s13012-015-0367-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-015-0367-1

Keywords