Skip to main content

Community capacity to acquire, assess, adapt, and apply research evidence: a survey of Ontario's HIV/AIDS sector

Abstract

Background

Community-based organizations (CBOs) are important stakeholders in health systems and are increasingly called upon to use research evidence to inform their advocacy, program planning, and service delivery. To better support CBOs to find and use research evidence, we sought to assess the capacity of CBOs in the HIV/AIDS sector to acquire, assess, adapt, and apply research evidence in their work.

Methods

We invited executive directors of HIV/AIDS CBOs in Ontario, Canada (n = 51) to complete the Canadian Health Services Research Foundation's "Is Research Working for You?" survey.

Findings

Based on responses from 25 organizations that collectively provide services to approximately 32,000 clients per year with 290 full-time equivalent staff, we found organizational capacity to acquire, assess, adapt, and apply research evidence to be low. CBO strengths include supporting a culture that rewards flexibility and quality improvement, exchanging information within their organization, and ensuring that their decision-making processes have a place for research. However, CBO Executive Directors indicated that they lacked the skills, time, resources, incentives, and links with experts to acquire research, assess its quality and reliability, and summarize it in a user-friendly way.

Conclusion

Given the limited capacity to find and use research evidence, we recommend a capacity-building strategy for HIV/AIDS CBOs that focuses on providing the tools, resources, and skills needed to more consistently acquire, assess, adapt, and apply research evidence. Such a strategy may be appropriate in other sectors and jurisdictions as well given that CBO Executive Directors in the HIV/AIDS sector in Ontario report low capacity despite being in the enviable position of having stable government infrastructure in place to support them, benefiting from long-standing investment in capacity building, and being part of an active provincial network. CBOs in other sectors and jurisdictions that have fewer supports may have comparable or lower capacity. Future research should examine a larger sample of CBO Executive Directors from a range of sectors and jurisdictions.

Peer Review reports

Findings

Community-based organizations (CBOs) are important stakeholders in the health sector [1, 2] as they not only provide a wide spectrum of programs and services to the members of their community but also play an advocacy role for broader system-level supports. As we have outlined in more detail in a previous paper [3], CBOs are typically not-for-profit organizations that: are guided by a specific mission (i.e., an overall goal) shaped by commonly held values within the community they serve; have a governance structure consisting of board members elected from the members in the community; and deliver a specific set of programs or services that are shaped by the mission and values of the organization. Because they are key health system stakeholders, it is important to support their capacity to find and use relevant and high-quality research evidence. Doing so will help ensure that programs, services, and advocacy are informed by the best available evidence.

However, there are many potential challenges related to research use. Barriers that have been consistently identified across sectors include: the complexity of research evidence, organizational barriers, lack of available time, poor access to current literature, lack of timely research, lack of experience and skills for critical appraisal, unsupportive culture for research, lack of actionable messages in research reports, and limited resources for implementation [4–8]. Given these barriers, it is not surprising that a lack of uptake of research evidence has been noted in many different sectors [9–13].

While there are strategies for supporting the use of research by managers of healthcare organizations and by policy makers in government [14, 15], there is still an important gap in the availability of a specific strategy for CBOs [3]. Many existing strategies for supporting the use of research evidence are based on experience and anecdotal evidence rather than on rigorous evidence of effects [8, 14, 16]. Moreover, strategies designed for supporting the use of research evidence by healthcare organizations and governments may not be relevant to the specific contexts and capacity of CBOs. The transferability of these strategies to CBOs is difficult to determine without first collecting evidence about their current capacity to find and use research evidence and then grappling with how to align capacity-building strategies with local realities [17–21].

In order to begin to fill this gap, we conducted an assessment of the capacity of HIV/AIDS CBOs in one Canadian jurisdiction (Ontario). The role of CBOs is particularly important for addressing the HIV/AIDS epidemic in Ontario as approximately half of all people living with HIV/AIDS in Canada reside in that province [22, 23]. The province has a strong network of HIV/AIDS CBOs to respond to the local HIV epidemic, with 51 organizations serving 19 different regions at the time of this study (there are now 46 organizations) [24]. As such, this network provides essential on-the-ground support for people living with HIV/AIDS, especially for the most vulnerable [25].

We invited Executive Directors of HIV/AIDS CBOs that are funded by the AIDS Bureau of the Ontario Ministry of Health and Long-Term Care and members of the Ontario AIDS Network (n = 51) to complete an online survey. We asked each Executive Director to work with one or more managers and front-line staff to complete the survey, allowing them to respond to areas that they are most knowledgeable about but also be informed by the specific knowledge of other managers and staff.

The CBO Executive Directors in Ontario were invited to complete an adapted version of the Canadian Health Services Research Foundation's "Is Research Working for You?" survey, which has been previously validated [26, 27]. The survey is organized into four areas of assessment: (1) Acquire: Can your organization find and obtain the research findings it needs?; (2) Assess: Can your organization assess research findings to ensure they are reliable, relevant, and applicable to you?; (3) Adapt: Can your organization present the research to decision makers in a useful way?; and (4) Apply: Are there skills, structures, processes, and a culture in your organization to promote and use research findings in decision making [27]? Each domain is then broken down into subsections that ask how well their organization performs specific tasks, and each question uses a five-point Likert scale (question anchors are 1 = Don't do, 2 = Do poorly, 3 = Do inconsistently, 4 = Do with some consistency, and 5 = Do well).

We analyzed the results of the survey by calculating response frequencies for each question. We did not conduct statistical comparisons between organizations due to the small sample available for the survey. Ethics approval was obtained from the University of Toronto to complete this research.

Executive Directors from 25 organizations completed the survey (response rate = 49%). These organizations collectively provide services to approximately 32,000 clients per year with 290 full-time equivalent staff. Participant/organizational characteristics are summarized in Table 1, and the results for each question are presented in Tables 2, 3, 4 to 5. The survey respondents were predominantly from organizations with an exclusive focus on HIV/AIDS. In addition, the sample appears balanced based on region served (approximately half were based in the greater Toronto area) and includes organizations with varying scale (as measured by the number of full-time equivalent staff) and service volume (as measured by the number of clients served each year). Lastly, the organizations included in our sample provide HIV/AIDS services to a broad array of populations, including women, youth, men who have sex with men, injection drug users, specific ethno-racial communities, as well as people living with HIV/AIDS.

Table 1 Participant characteristics
Table 2 Capacity to acquire research evidence*
Table 3 Capacity to assess research evidence
Table 4 Capacity to adapt research evidence
Table 5 Capacity to apply research evidence

Overall, we found organizational capacity to acquire, assess, adapt, and apply research evidence to be low, with a couple of notable exceptions where the majority of organizations provided ratings of "Do with some consistency" or "Do well." Capacity for applying research evidence appears to be relatively strong, with approximately half of organizations indicating they "Do with some consistency" or "Do well" on all but two questions (related to time and engaging on-staff researchers) for the subdomain related to whether their decision-making processes have a place for research (Table 5, subsection VII). In addition, most CBOs indicated that they have a corporate culture that is supportive of research use (64% selected "Do with some consistency" or "Do well").

Few organizations rated any of the questions related to acquiring, assessing, or adapting research evidence higher than "Inconsistent." Areas of relative strength include identifying research evidence through peer networks, "grey" literature reports, and through websites. Capacity was lowest for the domains related to: acquiring research (subsection I); assessing the reliability, quality, relevance, and applicability of research evidence (subsections III and IV); and summarizing results in a user-friendly way (subsection V). A higher proportion of organizations selected the "Don't do" or "Do poorly" response options in these domains. Overall, the results indicate that CBOs are not only limited in skilled staff with time, incentive, and resources for summarizing results in a user-friendly way and assessing the quality and reliability of research, but they also have limited arrangements with external experts to help them.

This study gives initial insight into the capacity of CBOs to acquire, assess, adapt, and apply research in their organizations, which can provide a starting point for developing their capacity to find and use research evidence. Efforts to build capacity among CBOs should also draw on key facilitators for supporting the use of research evidence that have been cited for other groups of stakeholders, such as ongoing interactions between researchers, managers, and policy makers and ensuring research is available in a timely manner (e.g., through searchable databases of research syntheses) [7]. In addition, many strategies to support the use of research evidence highlight the importance of clear summaries that are tailored to the specific audience and highlight the take-home messages from research [7, 8, 21, 28, 29].

The primary strength of this study is that it provides a baseline assessment of CBO capacity to acquire, assess, adapt, and apply research, which can be used for identifying areas for future capacity-building strategies. In addition, the data from this study can be used in future evaluations of organizational capacity after capacity-building initiatives have been implemented. However, the results should be interpreted with caution given the low response rate and the small sample size. In addition, the generalizability of our results to other Canadian and international jurisdictions may be limited as we only surveyed organizations in one Canadian jurisdiction.

Given the limited capacity to find and use research evidence, we recommend a capacity-building strategy for HIV/AIDS CBOs that focuses on providing the tools, resources, and skills needed to help CBOs more consistently acquire, assess, adapt, and apply research evidence. Such a strategy may also be appropriate in other sectors and jurisdictions given that CBO Executive Directors in the HIV/AIDS sector in Ontario report low capacity despite being in the enviable position of having stable government infrastructure in place to support them, benefiting from long-standing investment in capacity building, and being part of an active provincial network. CBOs in other sectors and jurisdictions that have fewer supports may have comparable or lower capacity.

Capacity building does not mean that CBOs need to do everything themselves. For instance, capacity building can focus on efforts to facilitate "pull" [28], such as developing and maintaining "one-stop shopping" websites that provide access to high-quality and relevant systematic reviews of the literature with user-friendly summaries (e.g., Health Systems Evidence [http://www.healthsystemsevidence.org] for health system managers and policy makers and http://www.health-evidence.ca for public health) [30, 31], and/or "rapid response" services, such as those developed through the Ontario HIV Treatment Network for HIV/AIDS CBOs [32] and the Health Technology Inquiry Service at the Canadian Agency for Drugs and Technologies in Health [33]. Additional efforts could also include facilitating access to research evidence by providing more links to external experts who can assist CBOs with identifying relevant research evidence and assessing its quality, reliability, and local applicability. Such efforts to support the use of research evidence will require in-depth consultation with CBOs to develop approaches that reflect their specific needs.

References

  1. World Health Organization: Ottawa Charter for Health Promotion. 1986, Copenhagen: World Health Organization, Division of Health Promotion, Education & Communication

    Google Scholar 

  2. World Health Organization Regional Office for Europe: Health 21 Health for All in the 21st Century. 1999, Copenhagen: World Health Organization Regional Office for Europe

    Google Scholar 

  3. Wilson MG, Lavis JN, Travers R, Rourke SB: Community-based knowledge transfer and exchange: Helping community-based organizations link research to action. Implementation Science. 2010, 5: 33-10.1186/1748-5908-5-33.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Dobbins M, Rosenbaum P, Plews N, Law M, Fysh A: Information transfer: What do decision-makers want and need from researchers. Implementation Science. 2007, 2: 20-10.1186/1748-5908-2-20.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Haynes RB, Haines A: Getting research findings into practice: Barriers and bridges to evidence based clinical practice. BMJ. 1998, 317: 273-276.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Innvaer S, Vist GE, Trommald M, Oxman AD: Health policy-makers' perceptions of their use of evidence: A systematic review. Journal of Health Services Research and Policy. 2002, 7: 239-244. 10.1258/135581902320432778.

    Article  PubMed  Google Scholar 

  7. Lavis JN, Davies HTO, Oxman AD, Denis JL, Golden-Biddle K, Ferlie E: Towards systematic reviews that inform health care management and policy-making. Journal of Health Services Research and Policy. 2005, 10: S1:35-S1:48.

    Article  Google Scholar 

  8. Mitton C, Adair CE, McKenzie E, Patten SB, Wayne Perry B: Knowledge Transfer and Exchange: Review and Synthesis of the Literature. Milbank Quarterly. 2007, 85: 729-768. 10.1111/j.1468-0009.2007.00506.x.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W: Lost in translation: Time for a map?. Journal of Continuing Education in the Health Professions. 2007, 26: 13-24.

    Article  Google Scholar 

  10. Grol RP, Dalhuijsen J, Thomas S, Veld C, Rutten G, Mokking H: Attributes of clinical guidelines that influence use of guidelines in general practice: Observational study. BMJ. 1998, 317: 858-861.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Lavis JN: Ideas at the margin or marginalized ideas? Nonmedical determinants of health in Canada. Health Affairs. 2002, 21: 107-112. 10.1377/hlthaff.21.2.107.

    Article  PubMed  Google Scholar 

  12. McGlynn E, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A: The quality of health care delivered to adults in the United States. New England Journal of Medicine. 2003, 348: 2635-2645. 10.1056/NEJMsa022615.

    Article  PubMed  Google Scholar 

  13. Schuster M, McGlynn E, Brook RH: How good is the quality of health care in the United States?. Milbank Quarterly. 1998, 76: 517-563. 10.1111/1468-0009.00105.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Walshe K, Rundall TG: Evidence-based management: From theory to practice in health care. Milbank Quarterly. 2001, 79: 429-457. 10.1111/1468-0009.00214.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Rundall TG, Martelli PF, Arroyo L, McCurdy R, Graetz I, Neuwirth EB: The informed decisions toolbox: tools for knowledge transfer and performance improvement. Journal of Healthcare Management. 2007, 52: 325-341.

    PubMed  Google Scholar 

  16. Young SK: Evidence-based management: A literature review. Journal of Nursing Management. 2002, 10: 145-151. 10.1046/j.1365-2834.2002.00309.x.

    Article  PubMed  Google Scholar 

  17. Miller RL, Shinn M: Learning from communities: overcoming difficulties in dissemination of prevention and promotion efforts. Am J Community Psychol. 2005, 35: 169-183. 10.1007/s10464-005-3395-1.

    Article  PubMed  Google Scholar 

  18. McDonald PW, Viehbeck S: From evidence-based practice making to practice-based evidence making: creating communities of (research) and practice. Health Promotion Practice. 2007, 8: 140-144. 10.1177/1524839906298494.

    Article  PubMed  Google Scholar 

  19. Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J: How can research organizations more effectively transfer research knowledge to decision makers?. Milbank Quarterly. 2003, 81: 221-248. 10.1111/1468-0009.t01-1-00052.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Lavis JN, Posada FB, Haines A, Osei E: Use of research to inform public policymaking. The Lancet. 2004, 364: 1615-1621. 10.1016/S0140-6736(04)17317-0.

    Article  Google Scholar 

  21. Jacobson N, Butterill D, Goering P: Development of a framework for knowledge translation: Understanding user context. Journal of Health Services & Research Policy. 2003, 8: 94-99. 10.1258/135581903321466067.

    Article  Google Scholar 

  22. Public Health Agency of Canada: HIV/AIDS Epi Updates. 2006, Ottawa: Centre for Infectious Disease Prevention and Control, Public Health Agency of Canada

    Google Scholar 

  23. Public Health Agency of Canada: HIV/AIDS Epi Updates. 2007, Ottawa, ON: Centre for Infectious Disease Prevention and Control, Public Health Agency of Canada

    Google Scholar 

  24. Ontario AIDS Network. Ontario AIDS Network: Members List: 2011, [http://ontarioaidsnetwork.on.ca/site/oan-community/members/]

  25. Crook J, Browne G, Roberts J, Gafni A: Impact of Support Services Provided by a Community-Based AIDS Service Organization on Persons Living With HIV/AIDS. Journal of the Association of Nurses in AIDS Care. 2005, 16: 39-49. 10.1016/j.jana.2005.05.003.

    Article  PubMed  Google Scholar 

  26. Canadian Health Services Research Foundation: Is Research Working for you? A Self-Assessment Tool and Discussion Guide for Health Services and Policy Organizations. 2005, Ottawa: Canadian Health Services Research Foundation

    Google Scholar 

  27. Kothari A, Edwards N, Hamel N, Judd M: Is research working for you? Validating a tool to examine the capacity of health organizations to use research. Implementation Science. 2009, 4: 46-10.1186/1748-5908-4-46.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Lavis JN, Lomas J, Hamid M, Sewankambo NK: Assessing country-level efforts to link research to action. Bulletin of the World Health Organization. 2006, 84: 620-628. 10.2471/BLT.06.030312.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Lavis JN, Wilson MG, Grimshaw JM, Haynes RB, Ouimet M, Raina P: Supporting the use of health technology assessments in policy making about health systems. International Journal of Technology Assessment in Health Care. 2010, 26: 405-414. 10.1017/S026646231000108X.

    Article  PubMed  Google Scholar 

  30. Health-Evidence: 2009, [http://www.health-evidence.ca/]

  31. McMaster Health Forum. Health Systems Evidence. McMaster Health Forum: Hamilton, Canada, 2010 [http://www.healthsystemsevidence.org/]

  32. Ontario HIV Treatment Network. Rapid Response Service: 2011, [http://www.ohtn.on.ca/Pages/Knowledge-Exchange/Rapid-Response-Service.aspx]

  33. CADTH CADTH: Health Technology Inquiry Service: 2011, [http://cadth.ca/en/products/rapid-response/rapid-response-service]

Download references

Acknowledgements and funding

The study and manuscript preparation was funded by The Ontario HIV Treatment Network. SBR, MGW, and JB each receive a salary from the Ontario HIV Treatment Network. JNL receives salary support as Canadian Research Chair of Knowledge Transfer and Exchange. We would also like to thank Tarik Bereket for coordinating the survey administration.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sean B Rourke.

Additional information

Competing interests

Some of the organizations included in the study sample may receive funding from the Ontario HIV Treatment Network. JNL was Chair and then a member of the Board of Directors of an ASO in Ontario but he played no role in administering or completing the survey.

Authors' contributions

MGW contributed to the conception and design of the study, the data analysis and interpretation, drafted the original manuscript, and incorporated revisions from the study team. SBR contributed to the conception and design of the study, the data analysis and interpretation, and helped draft and revise the manuscript. JNL contributed to the conception and design of the study, the data interpretation, and provided revisions to the manuscript. JB contributed to the conception and design of the study and provided revisions to the manuscript. RT contributed to the conception and design of the study and provided revisions to the manuscript. All authors read and approved the final manuscript.

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Wilson, M.G., Rourke, S.B., Lavis, J.N. et al. Community capacity to acquire, assess, adapt, and apply research evidence: a survey of Ontario's HIV/AIDS sector. Implementation Sci 6, 54 (2011). https://doi.org/10.1186/1748-5908-6-54

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1748-5908-6-54

Keywords