Email updates

Keep up to date with the latest news and content from Implementation Science and BioMed Central.

Journal App

google play app store
Open Access Open Badges Methodology

Developing a measure of provider adherence to improve the implementation of behavioral health services in primary care: a Delphi study

Gregory P Beehler123*, Jennifer S Funderburk456, Kyle Possemato45 and Christina L Vair17

Author Affiliations

1 VA Center for Integrated Healthcare, VA WNY Healthcare System, Buffalo, NY, USA

2 School of Nursing, University at Buffalo, The State University of New York, Buffalo, NY, USA

3 School of Public Health and Health Professions, University at Buffalo, The State University of New York, Buffalo, NY, USA

4 VA Center for Integrated Healthcare, Syracuse VA Medical Center, Syracuse, NY, USA

5 Department of Psychology, Syracuse University, Syracuse, NY, USA

6 Department of Psychiatry, University of Rochester, Rochester, NY, USA

7 Department of Psychology, University of Colorado, Colorado Springs, CO, USA

For all author emails, please log on.

Implementation Science 2013, 8:19  doi:10.1186/1748-5908-8-19

Published: 13 February 2013



The integration of behavioral health services into primary care is increasingly popular, yet fidelity of implementation in this area has been infrequently assessed due to the few measurement tools available. A sentinel indicator of fidelity of implementation is provider adherence, or utilization of prescribed procedures and engagement in model-specific behaviors. This study aimed to develop the first self-report measure of behavioral health provider adherence for co-located, collaborative care, a commonly adopted model of behavioral health service delivery in primary care.


A preliminary 56-item measure was developed by the research team to represent critical components of adherence among behavioral health providers. To ensure the content validity of the measure, a modified Delphi study was conducted using a panel of co-located, collaborative care model experts. During three rounds of emailed surveys, panel members provided qualitative feedback regarding item content while rating each item’s relevance for behavioral health provider practice. Items with consensus ratings of 80% or greater were included in the final adherence measure.


The panel consisted of 25 experts representing the Department of Veterans Affairs, the Department of Defense, and academic and community health centers (total study response rate of 76%). During the Delphi process, two new items were added to the measure, four items were eliminated, and a high level of consensus was achieved on the remaining 54 items. Experts identified 38 items essential for model adherence, six items compatible (although not essential) for model adherence, and 10 items that represented prohibited behaviors. Item content addressed several domains, but primarily focused on behaviors related to employing a time-limited, brief treatment model, the scope of patient concerns addressed, and interventions used by providers.


This study yielded the first content valid self-report measure of critical components of collaborative care adherence for use by behavioral health providers in primary care. Although additional psychometric evaluation is necessary, this measure may assist implementation researchers in clarifying how provider behaviors contribute to clinical outcomes. This measure may also assist clinical stakeholders in monitoring implementation and identifying ways to support frontline providers in delivering high quality services.

Primary healthcare; Mental health services; Guideline adherence; Delphi technique