Email updates

Keep up to date with the latest news and content from Implementation Science and BioMed Central.

Journal App

google play app store
Open Access Short report

Community capacity to acquire, assess, adapt, and apply research evidence: a survey of Ontario's HIV/AIDS sector

Michael G Wilson123, Sean B Rourke145*, John N Lavis2367, Jean Bacon1 and Robb Travers89

Author Affiliations

1 The Ontario HIV Treatment Network, Toronto, Canada

2 McMaster Health Forum, Hamilton, Canada

3 Centre for Health Economics and Policy Analysis, McMaster University, Hamilton, Canada

4 Centre for Research on Inner City Health, St. Michael's Hospital, Toronto, Canada

5 Department of Psychiatry, University of Toronto, Toronto, Canada

6 Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada

7 Department of Political Science, McMaster University, Hamilton, Canada

8 Department of Psychology, Wilfrid Laurier University, Waterloo, Canada

9 Dalla Lana School of Public Health, University of Toronto, Toronto, Canada

For all author emails, please log on.

Implementation Science 2011, 6:54  doi:10.1186/1748-5908-6-54

The electronic version of this article is the complete one and can be found online at: http://www.implementationscience.com/content/6/1/54


Received:2 November 2010
Accepted:28 May 2011
Published:28 May 2011

© 2011 Wilson et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Community-based organizations (CBOs) are important stakeholders in health systems and are increasingly called upon to use research evidence to inform their advocacy, program planning, and service delivery. To better support CBOs to find and use research evidence, we sought to assess the capacity of CBOs in the HIV/AIDS sector to acquire, assess, adapt, and apply research evidence in their work.

Methods

We invited executive directors of HIV/AIDS CBOs in Ontario, Canada (n = 51) to complete the Canadian Health Services Research Foundation's "Is Research Working for You?" survey.

Findings

Based on responses from 25 organizations that collectively provide services to approximately 32,000 clients per year with 290 full-time equivalent staff, we found organizational capacity to acquire, assess, adapt, and apply research evidence to be low. CBO strengths include supporting a culture that rewards flexibility and quality improvement, exchanging information within their organization, and ensuring that their decision-making processes have a place for research. However, CBO Executive Directors indicated that they lacked the skills, time, resources, incentives, and links with experts to acquire research, assess its quality and reliability, and summarize it in a user-friendly way.

Conclusion

Given the limited capacity to find and use research evidence, we recommend a capacity-building strategy for HIV/AIDS CBOs that focuses on providing the tools, resources, and skills needed to more consistently acquire, assess, adapt, and apply research evidence. Such a strategy may be appropriate in other sectors and jurisdictions as well given that CBO Executive Directors in the HIV/AIDS sector in Ontario report low capacity despite being in the enviable position of having stable government infrastructure in place to support them, benefiting from long-standing investment in capacity building, and being part of an active provincial network. CBOs in other sectors and jurisdictions that have fewer supports may have comparable or lower capacity. Future research should examine a larger sample of CBO Executive Directors from a range of sectors and jurisdictions.

Findings

Community-based organizations (CBOs) are important stakeholders in the health sector [1,2] as they not only provide a wide spectrum of programs and services to the members of their community but also play an advocacy role for broader system-level supports. As we have outlined in more detail in a previous paper [3], CBOs are typically not-for-profit organizations that: are guided by a specific mission (i.e., an overall goal) shaped by commonly held values within the community they serve; have a governance structure consisting of board members elected from the members in the community; and deliver a specific set of programs or services that are shaped by the mission and values of the organization. Because they are key health system stakeholders, it is important to support their capacity to find and use relevant and high-quality research evidence. Doing so will help ensure that programs, services, and advocacy are informed by the best available evidence.

However, there are many potential challenges related to research use. Barriers that have been consistently identified across sectors include: the complexity of research evidence, organizational barriers, lack of available time, poor access to current literature, lack of timely research, lack of experience and skills for critical appraisal, unsupportive culture for research, lack of actionable messages in research reports, and limited resources for implementation [4-8]. Given these barriers, it is not surprising that a lack of uptake of research evidence has been noted in many different sectors [9-13].

While there are strategies for supporting the use of research by managers of healthcare organizations and by policy makers in government [14,15], there is still an important gap in the availability of a specific strategy for CBOs [3]. Many existing strategies for supporting the use of research evidence are based on experience and anecdotal evidence rather than on rigorous evidence of effects [8,14,16]. Moreover, strategies designed for supporting the use of research evidence by healthcare organizations and governments may not be relevant to the specific contexts and capacity of CBOs. The transferability of these strategies to CBOs is difficult to determine without first collecting evidence about their current capacity to find and use research evidence and then grappling with how to align capacity-building strategies with local realities [17-21].

In order to begin to fill this gap, we conducted an assessment of the capacity of HIV/AIDS CBOs in one Canadian jurisdiction (Ontario). The role of CBOs is particularly important for addressing the HIV/AIDS epidemic in Ontario as approximately half of all people living with HIV/AIDS in Canada reside in that province [22,23]. The province has a strong network of HIV/AIDS CBOs to respond to the local HIV epidemic, with 51 organizations serving 19 different regions at the time of this study (there are now 46 organizations) [24]. As such, this network provides essential on-the-ground support for people living with HIV/AIDS, especially for the most vulnerable [25].

We invited Executive Directors of HIV/AIDS CBOs that are funded by the AIDS Bureau of the Ontario Ministry of Health and Long-Term Care and members of the Ontario AIDS Network (n = 51) to complete an online survey. We asked each Executive Director to work with one or more managers and front-line staff to complete the survey, allowing them to respond to areas that they are most knowledgeable about but also be informed by the specific knowledge of other managers and staff.

The CBO Executive Directors in Ontario were invited to complete an adapted version of the Canadian Health Services Research Foundation's "Is Research Working for You?" survey, which has been previously validated [26,27]. The survey is organized into four areas of assessment: (1) Acquire: Can your organization find and obtain the research findings it needs?; (2) Assess: Can your organization assess research findings to ensure they are reliable, relevant, and applicable to you?; (3) Adapt: Can your organization present the research to decision makers in a useful way?; and (4) Apply: Are there skills, structures, processes, and a culture in your organization to promote and use research findings in decision making [27]? Each domain is then broken down into subsections that ask how well their organization performs specific tasks, and each question uses a five-point Likert scale (question anchors are 1 = Don't do, 2 = Do poorly, 3 = Do inconsistently, 4 = Do with some consistency, and 5 = Do well).

We analyzed the results of the survey by calculating response frequencies for each question. We did not conduct statistical comparisons between organizations due to the small sample available for the survey. Ethics approval was obtained from the University of Toronto to complete this research.

Executive Directors from 25 organizations completed the survey (response rate = 49%). These organizations collectively provide services to approximately 32,000 clients per year with 290 full-time equivalent staff. Participant/organizational characteristics are summarized in Table 1, and the results for each question are presented in Tables 2, 3, 4 to 5. The survey respondents were predominantly from organizations with an exclusive focus on HIV/AIDS. In addition, the sample appears balanced based on region served (approximately half were based in the greater Toronto area) and includes organizations with varying scale (as measured by the number of full-time equivalent staff) and service volume (as measured by the number of clients served each year). Lastly, the organizations included in our sample provide HIV/AIDS services to a broad array of populations, including women, youth, men who have sex with men, injection drug users, specific ethno-racial communities, as well as people living with HIV/AIDS.

Table 1. Participant characteristics

Table 2. Capacity to acquire research evidence*

Table 3. Capacity to assess research evidence

Table 4. Capacity to adapt research evidence

Table 5. Capacity to apply research evidence

Overall, we found organizational capacity to acquire, assess, adapt, and apply research evidence to be low, with a couple of notable exceptions where the majority of organizations provided ratings of "Do with some consistency" or "Do well." Capacity for applying research evidence appears to be relatively strong, with approximately half of organizations indicating they "Do with some consistency" or "Do well" on all but two questions (related to time and engaging on-staff researchers) for the subdomain related to whether their decision-making processes have a place for research (Table 5, subsection VII). In addition, most CBOs indicated that they have a corporate culture that is supportive of research use (64% selected "Do with some consistency" or "Do well").

Few organizations rated any of the questions related to acquiring, assessing, or adapting research evidence higher than "Inconsistent." Areas of relative strength include identifying research evidence through peer networks, "grey" literature reports, and through websites. Capacity was lowest for the domains related to: acquiring research (subsection I); assessing the reliability, quality, relevance, and applicability of research evidence (subsections III and IV); and summarizing results in a user-friendly way (subsection V). A higher proportion of organizations selected the "Don't do" or "Do poorly" response options in these domains. Overall, the results indicate that CBOs are not only limited in skilled staff with time, incentive, and resources for summarizing results in a user-friendly way and assessing the quality and reliability of research, but they also have limited arrangements with external experts to help them.

This study gives initial insight into the capacity of CBOs to acquire, assess, adapt, and apply research in their organizations, which can provide a starting point for developing their capacity to find and use research evidence. Efforts to build capacity among CBOs should also draw on key facilitators for supporting the use of research evidence that have been cited for other groups of stakeholders, such as ongoing interactions between researchers, managers, and policy makers and ensuring research is available in a timely manner (e.g., through searchable databases of research syntheses) [7]. In addition, many strategies to support the use of research evidence highlight the importance of clear summaries that are tailored to the specific audience and highlight the take-home messages from research [7,8,21,28,29].

The primary strength of this study is that it provides a baseline assessment of CBO capacity to acquire, assess, adapt, and apply research, which can be used for identifying areas for future capacity-building strategies. In addition, the data from this study can be used in future evaluations of organizational capacity after capacity-building initiatives have been implemented. However, the results should be interpreted with caution given the low response rate and the small sample size. In addition, the generalizability of our results to other Canadian and international jurisdictions may be limited as we only surveyed organizations in one Canadian jurisdiction.

Given the limited capacity to find and use research evidence, we recommend a capacity-building strategy for HIV/AIDS CBOs that focuses on providing the tools, resources, and skills needed to help CBOs more consistently acquire, assess, adapt, and apply research evidence. Such a strategy may also be appropriate in other sectors and jurisdictions given that CBO Executive Directors in the HIV/AIDS sector in Ontario report low capacity despite being in the enviable position of having stable government infrastructure in place to support them, benefiting from long-standing investment in capacity building, and being part of an active provincial network. CBOs in other sectors and jurisdictions that have fewer supports may have comparable or lower capacity.

Capacity building does not mean that CBOs need to do everything themselves. For instance, capacity building can focus on efforts to facilitate "pull" [28], such as developing and maintaining "one-stop shopping" websites that provide access to high-quality and relevant systematic reviews of the literature with user-friendly summaries (e.g., Health Systems Evidence [http://www.healthsystemsevidence.org webcite] for health system managers and policy makers and http://www.health-evidence.ca webcite for public health) [30,31], and/or "rapid response" services, such as those developed through the Ontario HIV Treatment Network for HIV/AIDS CBOs [32] and the Health Technology Inquiry Service at the Canadian Agency for Drugs and Technologies in Health [33]. Additional efforts could also include facilitating access to research evidence by providing more links to external experts who can assist CBOs with identifying relevant research evidence and assessing its quality, reliability, and local applicability. Such efforts to support the use of research evidence will require in-depth consultation with CBOs to develop approaches that reflect their specific needs.

Competing interests

Some of the organizations included in the study sample may receive funding from the Ontario HIV Treatment Network. JNL was Chair and then a member of the Board of Directors of an ASO in Ontario but he played no role in administering or completing the survey.

Authors' contributions

MGW contributed to the conception and design of the study, the data analysis and interpretation, drafted the original manuscript, and incorporated revisions from the study team. SBR contributed to the conception and design of the study, the data analysis and interpretation, and helped draft and revise the manuscript. JNL contributed to the conception and design of the study, the data interpretation, and provided revisions to the manuscript. JB contributed to the conception and design of the study and provided revisions to the manuscript. RT contributed to the conception and design of the study and provided revisions to the manuscript. All authors read and approved the final manuscript.

Acknowledgements and funding

The study and manuscript preparation was funded by The Ontario HIV Treatment Network. SBR, MGW, and JB each receive a salary from the Ontario HIV Treatment Network. JNL receives salary support as Canadian Research Chair of Knowledge Transfer and Exchange. We would also like to thank Tarik Bereket for coordinating the survey administration.

References

  1. World Health Organization: Ottawa Charter for Health Promotion. Copenhagen: World Health Organization, Division of Health Promotion, Education & Communication; 1986. OpenURL

  2. World Health Organization Regional Office for Europe: Health 21 Health for All in the 21st Century. Copenhagen: World Health Organization Regional Office for Europe; 1999. OpenURL

  3. Wilson MG, Lavis JN, Travers R, Rourke SB: Community-based knowledge transfer and exchange: Helping community-based organizations link research to action.

    Implementation Science 2010, 5:33. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  4. Dobbins M, Rosenbaum P, Plews N, Law M, Fysh A: Information transfer: What do decision-makers want and need from researchers.

    Implementation Science 2007, 2:20. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  5. Haynes RB, Haines A: Getting research findings into practice: Barriers and bridges to evidence based clinical practice.

    BMJ 1998, 317:273-276. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  6. Innvaer S, Vist GE, Trommald M, Oxman AD: Health policy-makers' perceptions of their use of evidence: A systematic review.

    Journal of Health Services Research and Policy 2002, 7:239-244. PubMed Abstract | Publisher Full Text OpenURL

  7. Lavis JN, Davies HTO, Oxman AD, Denis JL, Golden-Biddle K, Ferlie E: Towards systematic reviews that inform health care management and policy-making.

    Journal of Health Services Research and Policy 2005, 10:S1:35-S1:48. OpenURL

  8. Mitton C, Adair CE, McKenzie E, Patten SB, Wayne Perry B: Knowledge Transfer and Exchange: Review and Synthesis of the Literature.

    Milbank Quarterly 2007, 85:729-768. PubMed Abstract | Publisher Full Text OpenURL

  9. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al.: Lost in translation: Time for a map?

    Journal of Continuing Education in the Health Professions 2007, 26:13-24. OpenURL

  10. Grol RP, Dalhuijsen J, Thomas S, Veld C, Rutten G, Mokking H: Attributes of clinical guidelines that influence use of guidelines in general practice: Observational study.

    BMJ 1998, 317:858-861. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  11. Lavis JN: Ideas at the margin or marginalized ideas? Nonmedical determinants of health in Canada.

    Health Affairs 2002, 21:107-112. PubMed Abstract | Publisher Full Text OpenURL

  12. McGlynn E, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al.: The quality of health care delivered to adults in the United States.

    New England Journal of Medicine 2003, 348:2635-2645. PubMed Abstract | Publisher Full Text OpenURL

  13. Schuster M, McGlynn E, Brook RH: How good is the quality of health care in the United States?

    Milbank Quarterly 1998, 76:517-563. PubMed Abstract | Publisher Full Text OpenURL

  14. Walshe K, Rundall TG: Evidence-based management: From theory to practice in health care.

    Milbank Quarterly 2001, 79:429-457. PubMed Abstract | Publisher Full Text OpenURL

  15. Rundall TG, Martelli PF, Arroyo L, McCurdy R, Graetz I, Neuwirth EB, et al.: The informed decisions toolbox: tools for knowledge transfer and performance improvement.

    Journal of Healthcare Management 2007, 52:325-341. PubMed Abstract OpenURL

  16. Young SK: Evidence-based management: A literature review.

    Journal of Nursing Management 2002, 10:145-151. PubMed Abstract | Publisher Full Text OpenURL

  17. Miller RL, Shinn M: Learning from communities: overcoming difficulties in dissemination of prevention and promotion efforts.

    Am J Community Psychol 2005, 35:169-183. PubMed Abstract | Publisher Full Text OpenURL

  18. McDonald PW, Viehbeck S: From evidence-based practice making to practice-based evidence making: creating communities of (research) and practice.

    Health Promotion Practice 2007, 8:140-144. PubMed Abstract | Publisher Full Text OpenURL

  19. Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J: How can research organizations more effectively transfer research knowledge to decision makers?

    Milbank Quarterly 2003, 81:221-248. PubMed Abstract | Publisher Full Text OpenURL

  20. Lavis JN, Posada FB, Haines A, Osei E: Use of research to inform public policymaking.

    The Lancet 2004, 364:1615-1621. Publisher Full Text OpenURL

  21. Jacobson N, Butterill D, Goering P: Development of a framework for knowledge translation: Understanding user context.

    Journal of Health Services & Research Policy 2003, 8:94-99. PubMed Abstract | Publisher Full Text OpenURL

  22. Public Health Agency of Canada: HIV/AIDS Epi Updates. Ottawa: Centre for Infectious Disease Prevention and Control, Public Health Agency of Canada; 2006. OpenURL

  23. Public Health Agency of Canada: HIV/AIDS Epi Updates. Ottawa, ON: Centre for Infectious Disease Prevention and Control, Public Health Agency of Canada; 2007. OpenURL

  24. Ontario AIDS Network. Ontario AIDS Network: Members List[http://ontarioaidsnetwork.on.ca/site/oan-community/members/] webcite

    2011.

  25. Crook J, Browne G, Roberts J, Gafni A: Impact of Support Services Provided by a Community-Based AIDS Service Organization on Persons Living With HIV/AIDS.

    Journal of the Association of Nurses in AIDS Care 2005, 16:39-49. PubMed Abstract | Publisher Full Text OpenURL

  26. Canadian Health Services Research Foundation: Is Research Working for you? A Self-Assessment Tool and Discussion Guide for Health Services and Policy Organizations. Ottawa: Canadian Health Services Research Foundation; 2005. OpenURL

  27. Kothari A, Edwards N, Hamel N, Judd M: Is research working for you? Validating a tool to examine the capacity of health organizations to use research.

    Implementation Science 2009, 4:46. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  28. Lavis JN, Lomas J, Hamid M, Sewankambo NK: Assessing country-level efforts to link research to action.

    Bulletin of the World Health Organization 2006, 84:620-628. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  29. Lavis JN, Wilson MG, Grimshaw JM, Haynes RB, Ouimet M, Raina P, et al.: Supporting the use of health technology assessments in policy making about health systems.

    International Journal of Technology Assessment in Health Care 2010, 26:405-414. PubMed Abstract | Publisher Full Text OpenURL

  30. Health-Evidence[http://www.health-evidence.ca/] webcite

    2009.

  31. McMaster Health Forum. Health Systems Evidence. McMaster Health Forum[http://www.healthsystemsevidence.org/] webcite

    Hamilton, Canada; 2010.

    McMaster Health Forum

  32. Ontario HIV Treatment Network. Rapid Response Service[http://www.ohtn.on.ca/Pages/Knowledge-Exchange/Rapid-Response-Service.aspx] webcite

    2011.

  33. CADTH CADTH: Health Technology Inquiry Service[http://cadth.ca/en/products/rapid-response/rapid-response-service] webcite

    2011.