Open Access Highly Accessed Open Badges Study protocol

Policy to implementation: evidence-based practice in community mental health – study protocol

Rinad S Beidas1*, Gregory Aarons2, Frances Barg3, Arthur Evans14, Trevor Hadley1, Kimberly Hoagwood5, Steven Marcus6, Sonja Schoenwald7, Lucia Walsh1 and David S Mandell1

Author Affiliations

1 Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market Street, 3rd floor, Philadelphia, PA, 19104, USA

2 Department of Psychiatry, University of California San Diego, San Diego, CA, USA

3 Department of Family Medicine and Community Health, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA

4 Department of Behavioral Health and Intellectual DisAbility Services, Philadelphia, PA, USA

5 Department of Psychiatry, New York University, New York, NY, USA

6 Center for Health Equity Research and Promotion, Philadelphia Veterans Affairs Medical Center; School of Social Policy and Practice, University of Pennsylvania, Philadelphia, PA, USA

7 Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA

For all author emails, please log on.

Implementation Science 2013, 8:38  doi:10.1186/1748-5908-8-38

Published: 24 March 2013



Evidence-based treatments (EBTs) are not widely available in community mental health settings. In response to the call for implementation of evidence-based treatments in the United States, states and counties have mandated behavioral health reform through policies and other initiatives. Evaluations of the impact of these policies on implementation are rare. A systems transformation about to occur in Philadelphia, Pennsylvania, offers an important opportunity to prospectively study implementation in response to a policy mandate.


Using a prospective sequential mixed-methods design, with observations at multiple points in time, we will investigate the responses of staff from 30 community mental health clinics to a policy from the Department of Behavioral Health encouraging and incentivizing providers to implement evidence-based treatments to treat youth with mental health problems. Study participants will be 30 executive directors, 30 clinical directors, and 240 therapists. Data will be collected prior to the policy implementation, and then at two and four years following policy implementation. Quantitative data will include measures of intervention implementation and potential moderators of implementation (i.e., organizational- and leader-level variables) and will be collected from executive directors, clinical directors, and therapists. Measures include self-reported therapist fidelity to evidence-based treatment techniques as measured by the Therapist Procedures Checklist-Revised, organizational variables as measured by the Organizational Social Context Measurement System and the Implementation Climate Assessment, leader variables as measured by the Multifactor Leadership Questionnaire, attitudes towards EBTs as measured by the Evidence-Based Practice Attitude Scale, and knowledge of EBTs as measured by the Knowledge of Evidence- Based Services Questionnaire. Qualitative data will include semi-structured interviews with a subset of the sample to assess the implementation experience of high-, average-, and low-performing agencies. Mixed methods will be integrated through comparing and contrasting results from the two methods for each of the primary hypotheses in this study.


Findings from the proposed research will inform both future policy mandates around implementation and the support required for the success of these policies, with the ultimate goal of improving the quality of treatment provided to youth in the public sector.

Evidence-based practice; Community mental health; Policy; Implementation; Fidelity; Organizational variables