Email updates

Keep up to date with the latest news and content from Implementation Science and BioMed Central.

Open Access Highly Accessed Research

The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study

Gregory A Aarons12*, Charles Glisson3, Phillip D Green3, Kimberly Hoagwood45, Kelly J Kelleher6, John A Landsverk2 and The Research Network on Youth Mental Health

Author Affiliations

1 University of California, San Diego, Department of Psychiatry, 9500 Gilman Drive (0812), La Jolla, CA, 92093-0812, USA

2 Child and Adolescent Services Research Center at Rady Children’s Hospital San Diego, 3020 Children’s Way MC-5033, San Diego, CA, 92132, USA

3 Children's Mental Health Services Research Center, Henson Hall, University of Tennessee, Knoxville, TN, 37996, USA

4 New York University School of Medicine, One Park Avenue at East 33rd, 8th Floor, New York, NY, 10016, USA

5 New York State Office of Mental Health, 44 Holland Avenue, Albany, NY, 12229, USA

6 The Ohio State University, Pediatrics and Public Health, 700 Children’s Drive, Columbus, OH, 43205, USA

For all author emails, please log on.

Implementation Science 2012, 7:56  doi:10.1186/1748-5908-7-56


The electronic version of this article is the complete one and can be found online at: http://www.implementationscience.com/content/7/1/56


Received:30 August 2011
Accepted:7 June 2012
Published:22 June 2012

© 2012 Aarons et al.; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Evidence-based practices have not been routinely adopted in community mental health organizations despite the support of scientific evidence and in some cases even legislative or regulatory action. We examined the association of clinician attitudes toward evidence-based practice with organizational culture, climate, and other characteristics in a nationally representative sample of mental health organizations in the United States.

Methods

In-person, group-administered surveys were conducted with a sample of 1,112 mental health service providers in a nationwide sample of 100 mental health service institutions in 26 states in the United States. The study examines these associations with a two-level Hierarchical Linear Modeling (HLM) analysis of responses to the Evidence-Based Practice Attitude Scale (EBPAS) at the individual clinician level as a function of the Organizational Social Context (OSC) measure at the organizational level, controlling for other organization and clinician characteristics.

Results

We found that more proficient organizational cultures and more engaged and less stressful organizational climates were associated with positive clinician attitudes toward adopting evidence-based practice.

Conclusions

The findings suggest that organizational intervention strategies for improving the organizational social context of mental health services may contribute to the success of evidence-based practice dissemination and implementation efforts by influencing clinician attitudes.

Keywords:
EBPAS; OSC; Evidence-based practice; Evidence-based practice attitude scale; Mental health services; Provider attitudes; Organizational social context; Culture; Climate

Background

Multiple factors at system, organizational, and individual levels influence the implementation of evidence-based practices (EBPs) and other innovations in public sector social service and mental healthcare settings [1-4]. These factors include policy, social and economic characteristics, characteristics of the innovation itself, characteristics of the organization attempting to implement the innovation, and characteristics of service providers and clients [1,3-8]. One of the most proximal influences on service providers’ attitudes and behaviors is the social context of the organizations in which they work [9]. This paper examines the association of organizational social context, other organizational factors, and clinician demographics with clinician attitudes toward adopting EBPs in a United States national sample of mental health clinicians.

The role of organizational social context in the development of individual-level work attitudes and behavior has been the focus of organizational research for several decades. This long line of research has established that work-related attitudes of individuals in a variety of different types of work environments are influenced by the organizational social contexts in which they work [5,10-12]. Organizational social context includes the norms and expectations (i.e., culture) of the organization for its members as well as the psychological impact of the work environment on the individual workers (i.e., climate). Culture and climate are related, complex, multidimensional constructs, but there is evidence that they are distinct and that each affects work attitudes in unique ways [13]. Culture is expected to influence the work attitudes of individual workers through their accommodation to the expectations (e.g., an emphasis on proficiency) that govern their day-to-day work behavior. Climate is expected to influence work attitudes through the psychological impact of the work environment (e.g., the level of stress) on the workers.

While the vast majority of research on culture and climate has been conducted in business and industrial organizations, there is a growing interest in the cultures and climates of healthcare, mental health, social service, and other human service organizations. While few controlled studies have examined actual organization context change efforts [14], research provides evidence that culture, climate, and work attitudes compose an organizational social context that is associated with worker turnover, new program sustainability, service quality, and service outcomes [9,13,15-20]. Here, we examine variance in mental health clinicians’ attitudes towards EBPs explained by specific dimensions of organizational culture and climate in order to better understand the role that organizational social context might play in complementing or inhibiting efforts to disseminate and implement EBPs.

Mental health clinicians have varying and complex attitudes toward EBPs [21,22]. While an overall positive or negative view of EBPs may be held by clinicians, clinicians can also simultaneously hold somewhat contradictory attitudes. That is, clinicians can be positively predisposed to EBPs on one dimension and negatively predisposed on another. For example, the idea that the practices clinicians use in providing mental healthcare should be informed by empirical evidence might be appealing to some clinicians who might also be unwilling to implement specific EBPs that are required by their employer or by a state mental health agency [23]. Or, clinicians may be open to learning more about EBPs, but at the same time believe that what they have learned in the world of practice experience is much more salient to selecting appropriate psychotherapeutic approaches than evidence provided by published research [23].

Aarons et al. [21,23,24] captured this complexity in the Evidence-Based Practice Attitude Scale (EBPAS) and documented four distinct dimensions in clinicians’ attitudes toward EBPs. The four dimensions include the intuitive appeal of EBPs, the likelihood of adopting EBPs as a result of institutional requirements, the perceived divergence between research-based practices and current practice, and general openness to learning new practices. Moreover, they found that certain clinician characteristics at the individual level (e.g., years of experience, education level) influence each of these various dimensions differently. For example, clinicians with more advanced degrees described EBPs as more appealing but at the same time were less willing to implement EBPs simply because they were required.

EBPAS scores have been associated with provider characteristics related to implementing new service technologies (See Aarons et al., 2012 for a review [25]). For example, female service providers have higher scores on the Appeal [26,27] and Requirements subscales [26] as well as higher total EBPAS scores [26]. Higher educational attainment has been associated with higher scores on the Appeal subscale [23,26-28] and lower scores on the Requirements subscale [26]. Aarons [23] found no difference in EBPAS scores by discipline (e.g., social work, psychology), but a later study found that providers with their highest degree in social work had higher Openness subscale scores and higher Total EBPAS scores than those with their highest degree in psychology. Clinicians with a degree in psychology, however, endorsed lower perceived Divergence between EBP and usual practice compared to clinicians with a degree in social work [26].

EBPAS scores have also been associated with knowledge and use of EBPs in other healthcare disciplines and settings. In a study of physicians, more positive EBPAS scores were associated with higher levels of physician knowledge of technological innovations supporting evidence-based clinical practice [29]. In a study of implementation of Parent–child Interaction Therapy (PCIT), more positive scores on Appeal were related to increased clinician attendance in PCIT consultation, while lower perceived Divergence between EBP and usual care was associated with increased phone coaching [30]. In substance abuse treatment settings, more positive attitudes toward EBP have been demonstrated to predict self-reported use of evidence-based cognitive/motivational treatment approaches [31] and adoption of contingency management, an evidence-based substance abuse treatment approach [32].

Studies examining transfer of learning in varied settings and types of health practice have also found relationships between attitudes and EBP. One study found that occupational therapy students in Ireland endorsed willingness to practice EBP [33]. Another study found that while medical residents in Japan endorsed generally positive attitudes toward EBP, few could articulate the construct or were familiar with the use of standard research sources in medical decision making [34]. However, a survey of physicians in the United Kingdom found a greater prevalence of positive attitudes toward EBP and greater ability to articulate elements of evidence-based medicine [35]. Thus, while studies across countries and healthcare practice settings have indicated variability in attitudes and knowledge related to EBP, none have examined the link between organizational context and attitudes in nationally representative samples.

Studies have indicated that change in organizational culture and climate in healthcare settings is a complex proposition that must take into account the impacts of autonomy and influence that occur in complex healthcare organizations [36,37]. In addition, individually held attitudes constitute but one factor in a complex context that may influence adoption and use of EBP [6]. Still, providing conceptual linkages between system and organizational context, provider attitudes and characteristics, and adoption and use of EBP is needed to help understand and develop strategies to improve EBP implementation and sustainment.

The organizational social contexts of providers’ work environments are also related to providers’ attitudes toward EBP. Positive climate measured by the Organizational Readiness for Change scale was negatively related to Divergence scores on the EBPAS [38], and demoralizing or negative organizational climate has been found to be positively related to Divergence scores [15]. These results indicate that providers in organizations with positive climates perceive less Divergence between EBP and their usual practice, while those in organizations with poor climates perceive more Divergence. An organizational culture that promotes achievement and mutual encouragement has been positively associated with higher scores on Appeal and Openness subscales, as well as higher total EBPAS scores, while an organizational culture emphasizing rules and orders (i.e., ‘command and control’) over personal beliefs and ideas, has been associated with higher Divergence scores [15]. Finally, mental health providers working for private non-profit organizations endorsed more positive attitudes toward EBP relative to public sector mental health departments [39].

Previous studies linking organizational social context and attitudes toward EBPs were conducted with limited samples and fewer organizational context dimensions, so further validation with more representative samples using multidimensional measures of both organizational culture and climate is needed. The present study builds on previous work by examining the association of attitudes toward EBP as measured by the EBPAS [23,26], with multiple dimensions of organizational social context as measured by the Organizational Social Context (OSC) scales [16] in a nationally representative sample of mental health agencies from all regions of the United States.

We developed several hypotheses based on the extant literature. By including predictors at both organizational and individual levels in a multi-level analysis, it is possible to partition the variance in attitudes explained by factors at each level. This is important because we know that culture and climate characteristics at the organizational level are not completely independent from individual clinician level characteristics. For example, organizational policies and leader behaviors can signal to providers the important cultural values of the organization [40]. Therefore, analyses at a single level—whether the organizational level or individual level—could confound the association of attitudes toward EBPs with variables at the other level. In addition, it is important to consider the independent and combined roles played by multiple dimensions of culture and climate simultaneously in explaining different dimensions of clinicians’ (often conflicted) attitudes toward EBPs. Thus, we hypothesized that variance in attitudes toward EBPs would be explained by variables at both the organizational and individual levels.

Organizational culture and climate are each expected to influence attitudes about EBPs in different ways. Organizational culture captures the expectations and values about what is important in a specific organization. These expectations and values, either implicitly or explicitly expressed in the behavior of fellow workers, have the capacity to socialize members of the organizations who may seek to behave in ways that meet the expectations of their workplace [9]. As a result, culture should influence attitudes about EBPs in ways that comply with organizational expectations. For example, clinicians who work in cultures that expect organizational members to be proficient in their work (i.e., have up-to-date knowledge, be effective in their work) should have more positive views of EBPs. Thus, we hypothesized that providers working in organizations with proficient cultures would have more positive overall attitudes toward EBPs as evidenced by higher scores on the EBPAS total scale score. In addition, we hypothesized that more proficient cultures would be associated with higher scores on the openness, appeal, and requirements scales, and lower scores on the divergence scale. We hypothesized that clinicians working in organizations with rigid or resistant cultures would be more likely to have lower EBPAS total, Appeal, Openness and Requirements scores, and higher Divergence Scores.

Organizational climate is expected to influence attitudes about EBPs somewhat differently. Climate describes the psychological impact of the work environment on clinicians (e.g., stress) [9]. Therefore, although an organization might expect its members to be proficient, clinicians in high-stress work environments might feel that requirements imposed on them by implementing new EBPs are overwhelming. Thus, we hypothesized that providers working in organizations with more stressful climates would endorse more negative attitudes toward EBPs as evidenced by lower scores on the Appeal, Openness, and Requirements scales, and higher scores on the Divergence scale. We also hypothesized that clinicians working in organizations in more functional climates would report more positive attitudes toward EBP. Finally, we hypothesized that clinicians working in organizations with engaged climates would endorse higher scores on the Appeal scale because this scale emphasizes the importance of colleague perceptions and practicality of EBP.

Methods

The national onsite, in-person survey of mental health clinicians who provided mental health services in the sampled clinics is one of two surveys composing the Clinical Systems Project (CSP) of the Research Network on Children’s Mental Health funded by the John D. and Catherine T. MacArthur Foundation [16]. The other survey was a telephone survey of mental health clinic directors [41]. The sampling frame for these surveys began with the counties selected for the National Survey of Child and Adolescent Well-being (described in [42]). The sample of 100 clinics participating in the clinician survey represents clinics from the CSP that met minimum size criteria (five or more clinicians) and whose director agreed to an onsite survey of clinicians in a scheduled staff meeting. Comparisons of the characteristics of clinics represented in the clinic directors survey that did and did not meet these criteria show similar characteristics in the number of clinicians employed in the clinic, clinician turnover rates, and the proportions of clinicians who are psychiatrists, Ph.D. psychologists, and MSW social workers. However, a slightly higher proportion of therapists in the clinics that participated in the clinician survey were BSW social workers (13% versus 8%).

Participants

Trained external research assistants who had no formal working relationships with the clinics conducted the onsite surveys in person with clinicians who treated either children or both children and adults in each of the 100 participating mental health clinics. Respondents in each clinic completed the surveys simultaneously during a scheduled staff meeting with no upper managers present, after receiving assurances of confidentiality from the external research assistant. The respondents returned the surveys at the end of the meeting directly to the research assistant using sealed envelopes. In some clinics, more than one meeting was necessary to obtain data from all clinicians. The response rate for the clinicians per site ranged from 30% to 100% with an overall average response rate of 76%.

Clinician demographics

Demographics were assessed through clinician self-report of age, gender, race/ethnicity, educational level, major of highest degree, years of experience, and job tenure. As shown in Table 1, 1,112 clinicians from 100 clinics participated in 75 cities in 26 states from all parts of the United States. Among the respondents, 24% worked for public mental health agencies, 2% for private for-profit agencies, and 74% for private not-for-profit agencies. As shown in Table 2, the age of the participating clinicians by clinic ranged from 21 to 74 years, and clinicians had up to 50 years of experience with a mean of 11 years of experience. Most of the clinicians were female (76%) and Caucasian (71%), and had masters degrees (67%), with majors in either social work (41%) or psychology (32%). In addition, smaller proportions held doctorates (7%) and were either African American (15%) or Hispanic (7%).

Table 1. Number and location of clinics

Table 2. Characteristics of clinicians

Measures

Evidence-Based Practice Attitudes Scale (EBPAS)

The EBPAS consists of 15 items measured on a 5-point Likert scale ranging from 0 (not at all) to 4 (to a very great extent) [23,24,26]. The EBPAS is conceptualized as consisting of four lower-order factors/subscales and a higher-order factor or total score that represents the respondent’s global attitude toward the use of EBPs [26]. Previous studies suggest moderate to good internal consistency reliability in two samples (Cronbach’s alpha total scale ranging from 0.77 to 0.79, subscales ranging from 0.59 to 0.93) [23,24]. Construct validity is supported by two previous scale development studies [23,24] and by studies of associations between EBPAS and mental health clinic structure and policies [23], culture and climate [15] and leadership [27].

Scores for each subscale were calculated for the present analyses by summing the items composing each subscale. A Total Scale Score was calculated by summing all 15 items after reverse coding the items composing the Divergence Scale. The alpha reliability for measuring the Total Score was 0.76 in this sample.

Among the four lower-order factors that compose the Total Score, the Appeal Score assesses the extent to which the provider would adopt an EBP if it were intuitively appealing, could be used correctly, or was being used by colleagues who were happy with it. The alpha reliability for measuring Appeal was 0.80 in this sample. The Requirements Score indicates the extent to which the provider would adopt an EBP if it were required by an agency, supervisor, or state. The alpha reliability for measuring Requirements was 0.91 in this sample. The Openness Score indicates the extent to which the provider is generally open to trying new interventions and would be willing to try or use more structured or manualized interventions. The alpha for measuring Openness was 0.84 in this sample. The Divergence Score indicates the extent to which the provider perceives EBPs as not useful and less important than clinical experience. The alpha reliability for measuring Divergence was 0.66 in this sample.

Organizational Social Context (OSC)

The OSC includes 105 items that measure seven primary scales that were developed in multiple validity and reliability studies over a period of three decades to assess three domains of the social context of mental health and social service organizations: Organizational Culture, Organizational Climate, and Work Attitudes [16]. The OSC was completed by individual clinicians in staff meetings as described above and the scale scores were aggregated by clinic site after confirming agreement among clinicians within each clinic [16]. The OSC measures of culture and climate are included in the present analyses and described below.

Organizational Culture is defined as the expectations that govern how work is done in an organization and the OSC assesses organizational culture on three second-order dimensions: rigidity, proficiency, and resistance. Previous studies have provided evidence of the validity and reliability of these scales and associated organizational culture with individualized care, service quality, staff turnover, staff attitudes toward their work, productivity, efficiency, and the sustainability of new treatment programs in a variety of mental health and social service organizations [5,12,19,43-46].

A Rigid Organizational Culture is characterized by expectations that clinicians will have little discretion or flexibility in carrying out their jobs, provide limited input into key management decisions, and carefully follow a host of bureaucratic rules and regulations. This dimension is assessed with items measuring centralization (e.g., ‘I have to ask a supervisor or coordinator before I do almost anything’) and formalization (e.g., ‘the same steps must be followed in processing every piece of work’). The alpha reliability for measuring Rigidity in this sample is 0.81.

A Proficient Organizational Culture is characterized by expectations that clinicians will place the well-being of each client first and that clinicians will be competent and have up-to-date knowledge. Proficient cultures expect clinicians to be both skilled and attentive to the needs of individual clients. Proficiency is assessed with items measuring responsiveness (e.g., ‘members of my organizational unit are expected to be responsive to the needs of each client’) and competence (e.g., ‘members of my organizational unit are expected to have up-to-date knowledge’). The alpha reliability for measuring Proficiency in this sample is 0.94.

A Resistant Organizational Culture is characterized by expectations that clinicians will show little interest in change or in new ways of providing service, and that clinicians will suppress any interest in change with criticism and apathy. Resistance is assessed with items measuring apathy (e.g., ‘members of my organizational unit are expected to not make waves’) and suppression (e.g., ‘members of my organizational unit are expected to be critical’). The alpha reliability for measuring Resistance in this sample is 0.81.

Organizational Climate is defined as the employees’ perceptions of the psychological impact of the work environment on their own well-being and functioning in the organization [47]. An organizational climate is formed when employees in the same organizational unit share similar perceptions about the psychological impact of their work environment. The OSC measures climate on three second-order factors: engagement, functionality, and stress. Previous studies have provided evidence of the validity and reliability of these scales and associated organizational climate with service quality, staff turnover, job satisfaction and commitment, and service outcomes in a variety of mental health and social service organizations [5,12,19,20,45,48].

An Engaged Climate is characterized by employee perceptions that they are able to personally accomplish many worthwhile things, remain personally involved in their work and sustain concern about their clients. Engagement is assessed with items measuring personalization (e.g., ‘I feel I treat some of the clients I serve as impersonal objects’ – reverse coded) and personal accomplishment (e.g., ‘I have accomplished many worthwhile things in this job’). The alpha reliability for measuring Engagement in this sample is 0.78.

A Functional Climate is characterized by employee perceptions that they receive the cooperation and help they need from coworkers and administrators to do a good job, have opportunities for personal advancement and growth, and have a clear understanding of how they fit in, and can work successfully within the organization. Functionality is assessed with items measuring growth and advancement (e.g., ‘this agency provides numerous opportunities to advance if you work for it’), role clarity (e.g., ‘my job responsibilities are clearly defined’), and cooperation (e.g., ‘there is a feeling of cooperation among my coworkers’). The alpha reliability for measuring Functionality in this sample is 0.90.

A Stressful Climate is characterized by employee perceptions that they are emotionally exhausted from their work, overloaded in their work, and unable to get the necessary things done. Stress is assessed with items measuring emotional exhaustion (e.g., ‘I feel like I am at the end of my rope’), role conflict (e.g., ‘interests of the clients are often replaced by bureaucratic concerns, e.g., paperwork’), and role overload (e.g., ‘the amount of work I have to do keeps me from doing a good job’). The alpha reliability for measuring Stressful Climate in this sample is 0.94.

Analyses

Pearson product–moment correlations were computed at the individual level for responses to the OSC scales assessing culture and climate and the EBPAS scales assessing attitudes toward EBPs. The clinic response rate was also included in the correlation matrix to assess whether the response rate by clinic was correlated with clinician scale responses. In addition, the level of agreement of clinician responses to each culture and climate scale were assessed for each clinic using the rwg coefficient [45].

Each of the EBPAS scales (the total score and the four subscales) were regressed on predictors in two-level random regression models (individuals nested within organizations) using HLM 6 statistical modeling software [49]. For each analysis, an unconditional model was estimated first including only the intercept to compute the intraclass correlation (ICC) and organizational variance for each EBPAS scale. In the second analysis, individual-level, clinician characteristics and group centered responses to the culture and climate measures were added in the first level. Organization-level characteristics, including measures of organizational culture and climate assessed as the mean responses of clinicians in each clinic, were added in the second level. This strategy allowed the between-organizational differences in the EBPAS scales to be distinguished from the within-organizational level differences in EBPAS scales for each unit difference in the OSC scales at the organization and individual levels, respectively.

In addition to estimating regression coefficients for each predictor, the effects of categorical variables at the clinic level (e.g., type of agency) and clinician level (e.g., race/ethnicity) were assessed using a model comparison approach, based on likelihood ratio tests of nested models. This was calculated as the difference in deviance statistics (−2 log likelihood) between the full model and the model without the contrasts (parameters) that compose the particular categorical variable of interest [49]. The significance of the difference in deviance scores is tested as a chi-square value with the degrees of freedom equal to the number of contrasts composing the variable.

Results

The correlation matrix in Table 3 shows that the clinic response rate (i.e., the proportion of clinicians completing scales within each clinic) is unrelated to the OSC or EBPAS scale values, as indicated by correlations that range from −0.05 to 0.03 (all non-significant). The pattern of correlations within the matrix also suggests that the responses to the OSC and EBPAS are not explained by common method error variance. The size of the correlations between scales from the two instruments (OSC and EBPAS) range from a low of 0.00 (p < 0.990) for rigid culture and EBPAS total to a high of 0.24 (p < 0.000) for proficient culture and EBPAS total. Moreover, despite the statistical power of the large sample (n = 1,112), only half of the 35 correlations between the two sets of scales are statistically significant. Among the dimensions of culture and climate, proficient culture is the most highly correlated with attitudes toward EBPs. Significant correlations include the relationships between proficient culture and the EBPAS Total (0.24, p < 0.001), Requirements (0.19, p < 0.001), Appeal (0.20, p < 0.001), and Openness (0.20, p < 0.001).

Table 3. Correlation matrix for Clinic Response Rate, EBPAS, and OSC Scales (n = 1,112)

The within clinic agreement in clinician descriptions of their clinic’s culture and climate was assessed with rwg[45]. The average rwg for each scale ranged between 0.91 and 0.96 for each clinic. The lowest rwg values for each scale across all clinics ranged between 0.58 and 0.87, and the highest values for each scale ranged between 0.98 and 0.99. These statistics represent strong within clinic agreement among clinicians completing the OSC and support the aggregation of those responses for clinic level measures of organizational culture and climate.

Intraclass correlation coefficients (ICCs) in Tables 4 and 5 provide an estimate of the degree of clustering of responses to the EBPAS scales within groups (represented here by mental health clinics). Although ICCs are typically small in relation to estimates of the proportion of between group to total variation provided by an ANOVA-based eta-squared coefficient, the effect of clustering with ICCs as low as 0.01 to 0.05 can be dramatic on estimates using ordinary least squares regression that assumes ICC = 0.00 [50]. The ICCs reported in Tables 4 and 5 for the five EBPAS scales range between 0.02 to 0.06, and four of the five are statistically significant as reported in Table 5, indicating that attitudes about EBPs are clustered by clinic and thus need random regression models that account for the clustering. The ICCs and eta-squared coefficients (which range from 0.11 to 0.15) reported in Tables 4 and 5 confirm that a significant proportion of the variance in clinicians’ attitudes toward the use of EBPs is a function of the clinic in which the clinician works. Tables 4 and 5 also show the proportions of the organizational-level or clinic variance in attitudes toward EBPs that is explained by organizational social context and by all organizational and individual predictors combined.

Table 4. Hierarchical linear model analysis of total EBPAS total score

Table 5. Hierarchical linear model analyses of EBPAS total scale and subscales

The correlations in Table 3 provided evidence that the relationship between clinician-level responses to the scales measuring proficient culture and the total EBPAS was stronger than the relationship between any other pair of OSC and EBPAS dimensions. The higher order analysis in Table 4 provides further support for our hypothesis that clinicians who work in organizations with proficient cultures have more positive attitudes toward the use of EBPs. The results shown in Table 4 describe a significant unique positive effect of proficient culture on attitudes toward EBPs after controlling for other dimensions of organizational culture, dimensions of organizational climate, type of agency, service structure, clinician level perceptions of culture and climate, and other clinician characteristics (e.g., age, experience, education, et al.). Clinicians who work in clinics that expect them to place the well-being of each client first, to be competent and to have up-to-date knowledge are more likely to have more favorable views toward using EBPs than clinicians who work in clinics that do not have those expectations, regardless of their own demographic characteristics, experience, or training.

It’s important to note that this result was obtained after controlling for type of agency (for profit, non-profit, or public), clinic service structure (child and adult, child only, or separate child division), and clinician gender, education level, years of experience, race, profession (education, medicine, nursing, social work, other or psychology) and individual level responses to the culture and climate scales. Although not shown here, the same association between the clinicians’ attitudes toward EBP and their organizational culture was confirmed without the additional covariates in the analysis.

The HLM model shown above was repeated with each of the EBPAS first order scales (Appeal, Requirements, Openness, Divergence) and is shown in Table 5. Clinicians in clinics with proficient cultures report higher overall EBPAS Total and lower Divergence scores. In contrast, Organizational Climate has a stronger association with the EBPAS Appeal scale. In more engaged climates (where clinicians share the perception that they are able to personally accomplish many worthwhile things, remain personally involved in their work, and sustain concern about their clients), clinicians indicate they would be more likely to adopt an EBP if it were intuitively appealing, could be used correctly, or was being used by colleagues who were pleased with it. Clinicians in more stressful climates (those work environments where clinicians shared the perception that they are emotionally exhausted from their work, overloaded in their work, and unable to get the necessary things done at work) report that they would not be as likely to use an EBP if it were required by their agency, supervisor, or state. This suggests that formal requirements for using EBPs would not be as successful in organizational climates characterized by high levels of stress and that other, or additional implementation strategies should be considered.

The analyses also show that the service structure of the clinic was not associated with the clinicians’ attitudes toward EBPs but the type of agency did play a role in three of the five EBPAS scales (Total, Appeal and Openness). Clinicians in private for profit clinics generally had more negative attitudes toward the use of EBPs.

As shown in Tables 4 and 5, the dimensions of organizational culture and climate together explained the highest proportion of organizational variance in the EBPAS dimensions of Appeal and Openness. In addition, culture and climate together also accounted for the majority of the organizational variance that was explained by the full model in these two criteria and in the EBPAS Requirements scale.

At the individual level, several clinician characteristics were associated with attitudes toward the use of EBPs with years of experience, race/ethnicity and academic major having consistent associations across all five EBPAS scales. Clinicians with more years of experience and African American clinicians reported more negative attitudes toward EBPs after controlling of all other characteristics at the clinician and clinic level. Although the effects of academic major were not entirely consistent across all dimensions, social workers generally held the most positive views of EBPs as indicated by three of the five EBPAS scales.

Discussion

The important finding in this study is that mental health clinic organizational culture and climate are associated with clinician’s attitudes toward adopting EBPs, even when controlling for the effects of a variety of organizational-level and individual-level characteristics. Clinicians working in organizations characterized by proficient cultures, engaged climates, and less stressful climates have more positive attitudes toward the use of EBPs, but each dimension of culture and climate is uniquely associated with a different dimension of the clinicians’ attitudes. The best unique predictor among the culture and climate dimensions of clinicians having positive overall attitudes toward EBPs is working in a proficient organizational culture. Clinicians in proficient cultures (i.e., the organization expects them to place the well-being of the clients first, to be competent, and have up-to-date knowledge) endorsed more positive overall attitudes toward adopting EBPs and lower perceived divergence between ‘in the trenches’ clinical work and the use of EBPs.

Two dimensions of organizational climate were associated with clinician attitudes toward EBPs, but each was associated with a different attitudinal dimension. Clinicians working in clinics with more stressful climates were less likely to adopt an EBP if it was required by a supervisor, agency, or state. This means clinicians working in settings with high levels of emotional exhaustion and role overload are less likely to respond favorably to policy or regulatory mandates requiring the use of EBPs.

Clinicians working in engaged organizational climates where there is a sense of personal accomplishment, personal involvement in and concern for clients reported a greater likelihood of adopting an EBP if it fit their views of clinical practice and their ability to effectively learn and use (i.e., to have high self-efficacy) a given EBP.

Given our conservative approach of testing hypothesized relationships after controlling for the effects of all other dimensions and predictors at the organizational and individual levels, a number of our hypotheses were not confirmed. The HLM analyses provided tests of the unique effects of each dimension of culture and climate at the organizational level after controlling for the effects of all other culture and climate dimensions and all other predictors at both the organizational and individual levels. This is important because all dimensions of culture and climate were related to at least one dimension of the EBPAS in the zero-order correlations shown in Table 3. Moreover, all dimensions of culture and climate at either the organizational or individual level were related to one or more dimensions of the EBPAS in the HLM analysis. However, our hypotheses and findings focus on the independent association of each of the organizational level dimensions of culture and climate with individual level attitudes toward EBP.

In regard to organizational culture, there were no significant organizational level findings related to rigid or resistant cultures (when controlling for the expectations associated with proficient cultures, the three climate dimensions, and all individual-level predictors). Rigid and resistant cultures were associated with attitudes toward EBP at the zero order level, but these relationships were attenuated when the other dimensions of culture and climate were included in the analyses.

Some of our climate related hypotheses were also not supported. While stressful and engaged climates were associated with poorer attitudes toward adopting EBP, our findings illustrated that only two EBPAS subscales were related to climate. Moreover, although functional climates were associated with EBPAS total, Requirements, Appeal, and Openness scales at the zero-order level, all of these relationships were not statistically significant when other dimensions of culture and climate were included at the organizational and individual levels.

It is possible that additional effects could be found if we utilized profile analysis to characterize clinics by culture/climate profiles (e.g., depicting the levels of culture and climate dimensions in relation to each other) rather than assessing the unique contributions of individual culture and climate dimensions after controlling for the others. That is, there may be profiles of positive and negative culture paired with positive or negative climate that provide a more complex characterization of organizations and are associated with EBP attitudes. Thus, future studies should examine profiles of organizational context that may be associated with attitudes and adoption and use of EBPs.

While organizational culture and climate and attitudes toward EBP have been explored in a previous study [15], the present study is the first to focus on these aspects of the complex relationship between multiple dimensions of social context and attitudes. However, it should be noted that the previous study utilized an earlier measure of organizational context with some overlap with the present measure of organizational social context. The present study advances this work by utilizing a more refined measure of organizational social context validated in a United States nationally representative sample of mental healthcare providers. In particular, the conceptualization of the culture and climate dimensions have been honed to be most relevant to health and allied health service settings and predictive of clinical outcomes and EBP implementation [50].

These findings suggest that the successful dissemination and implementation of EBPs among mental health service providers can be facilitated by the support of strategies for developing positive organizational cultures and climates that might otherwise present barriers to adoption and implementation [50]. In addition to implementation of clinical interventions, broad changes are now occurring in behavioral health settings with significant incentives to deploy accountability systems tied to monitoring and reimbursement. Such initiatives may impact organizational context or conversely be affected by organizational context. Changing organizational culture and climate in mental health and social service systems to prepare for implementation requires strategic, well-planned, and evidence-based strategies specifically designed for those systems [1,40,51]. Some promising developments in this regard are being tested in such organizations using randomized controlled trials, showing that organizational interventions can create improvements not only in organizational social context, but also in the implementation of EBPs and client outcomes [50,52]. In related work, effective leadership and coordinated organizational strategies to develop a positive work environment have been suggested to promote strategic cultures and climates that support the implementation and use of EBPs [40,53,54].

Beyond organizational culture and climate, we identified other factors associated with attitudes toward EBPs that are important to understanding barriers to dissemination and implementation. The service structure of the clinic was not associated with clinician attitudes toward EBPs but the type of agency did play a role. Clinicians working in private for-profit clinics reported more negative attitudes toward the use of EBPs. The finding that clinicians working in private for-profit clinics reported more negative attitudes toward EBPs highlights the complexity of supporting EBP implementation and use. private for-profit clinics generally operate under different financing and reimbursement structures and processes such as fee-for-service. This is in contrast to private non-profits that may rely more heavily on contracts and grants to provide funding for services. The links between organization type and attitudes toward EBPs was explored in another study that found that clinicians working inprivate non-profit clinics, compared to public sector clinics, had more positive attitudes toward EBPs [39]. The authors attributed the finding to the more bureaucratic, hierarchical, and less flexible nature of programs that are part of large public sector mental health authorities.

At the individual level and consistent with previous studies, we found that female clinicians and social workers were more positive in their assessments of the use of EBPs while clinicians with more years of experience were more negative in their assessments. Further research is needed to determine why these characteristics impact attitudes and how to take advantage of what is being done in psychology, social work, and other academic and applied mental health, substance abuse, and social service training programs to promote the adoption and use of EPBs.

The implementation of EBPs is a particularly critical change effort that can improve the quality and outcomes of mental health services. In light of recent healthcare policy change under the Patient Protection and Affordable Care Act it is likely that the landscape of health services in the United States will also change. For example, mental health and physical health services may be more integrated in the same service settings, and federally qualified health centers (FQHCs) are likely to be implementing EBPs at an accelerated pace. Understanding the impact of organizational social context on staff readiness and willingness to implement EBPs in light of these changes can suggest ways to improve implementation. Finally, changes in healthcare policy and the financing of behavioral health, and the reorganization of the systems delivering these services (i.e., primary care and specialty mental health payment and provider networks) are likely to affect both attitudes towards EBPs (may reinforce positive attitudes or not) and organizational processes (may produce different values, may increase stress and burnout, may privilege proficient organizations) that are central to successful implementation.

A number of innovations being adopted by mental health authorities expand the definition of EBP beyond efficacious treatment models. Some examples include continuous data monitoring systems, alerts to target prescribing practices, business engineering to target high volume service users, and client outcomes monitoring and feedback to clinicians. Some of these may be considered to be evidence-based if they are proven to improve outcomes. The EBP referent for the EBPAS is more traditional psychological or psychosocial treatment that has research support and may be manualized and structured. Thus, attitudes towards therapies, clinical interventions, and treatments may be a special case of EBPs tapped by the EBPAS. Further research is needed to better understand the role of social context in attitudes toward a variety of other technical innovations as described above. Attitudes toward clinical practices and treatments may not reflect attitudes toward such innovations. The first author is currently piloting an adapted version of the EBPAS to be used to assess attitudes toward adopting specific innovations or practices.

Limitations

Some limitations of the present work should be noted. First, we assessed four established domains of attitudes toward EBP. Recent work that occurred after data collection for this study identified an additional eight domains of attitudes for EBP including: EBP limitations, fit with clinical practice, impact of fidelity monitoring, balance of art and science in psychotherapy, workplace burden, impact of job security, perceived organizational support, and the impact of getting feedback on job and clinical performance [21]. Thus, further work is needed to establish the associations of these additional scales with individual differences and organizational social context. Second, while our sample is representative of clinicians working in community-based mental health clinics, our findings may not generalize to clinicians in individual practice or to sectors other than mental health (e.g., alcohol/drug, social services). However, there are a number of similarities across public sectors that would support the impact of common organizational processes for implementation [1].

We recognize that determinants of clinician behavior are multifaceted. In the present study, we examined attitudes toward EBP and attitudes are but one among a group of determinants affecting whether someone will or will not engage in a behavior. Other determinants include social norms (some of which are captured by the OSC) and networks [55,56], self-efficacy [57], locus of control and behavioral intentions [58], expectancies, habits, and environmental constraints. Further, changing behavior might also be facilitated by maximizing the fit of an innovation or practice with the needs of the clinic, and needs of clients [59]. While attitudes are associated with organizational social context dimensions, changing clinician behavior will require strategic attention to potential mediators and moderators of the link between attitudes and behavior change. Conversely, it may be the case that engaging in a behavior such as continued use of an EBP until familiarity is developed, using a data monitoring system to track specific indicators of change, or attending learning collaborative meetings with one’s peers, may change attitudes and beliefs about EBPs. That is, behavioral activation itself may influence attitudes [60].

Although a number of covariates were included in the model to control for confounding effects at both the organizational and individual levels, the observed association between organizational social context and individual attitudes toward EBPs provides no evidence of a causal relationship. To better understand the causal links between context and attitudes, we are currently conducting randomized controlled studies with organizational interventions that focus on improving both social context and the use of EBPs as a way of linking changes in social context at the organizational level with the adoption and implementation of EBPs by clinicians.

Finally, it is important to note that utilizing quantitative measures alone presents challenges in adequately measuring and describing a construct as complex as social context. The need for mixed-methods in understanding the impacts of EBP implementation is emerging as important and critical for a more comprehensive and balanced understanding of implementation process and outcomes [61,62]. In addition, the need for multilevel conceptual approaches and analysis is critical to understanding the complexity of outer and inner contexts of services in healthcare and allied healthcare settings [1,63,64].

Conclusions

The present study adds to the research base demonstrating that organizational social context is associated with clinician attitudes toward the adoption and use of EBPs [50,65]. While focusing on the organization is only one aspect of the landscape of EBP implementation [1], it is an important one for a number of reasons. First, most EBPs are implemented within organizations and organizational characteristics such as social context predict provider attitudes toward adopting EBP, even when accounting for individual-level covariates. Second, organizational social context is malleable. That is, randomized controlled studies have documented that change in organizational culture and climate, as well as associated improvements in EBP implementation, can be accomplished with targeted improvement strategies [40,50-52,66]. Third, we know that the availability of clinical training and funding may be a necessary but not sufficient condition for successful EBP implementation and that successful implementation strategies must address the context in which EBPs are offered [1,4]. In conclusion, this paper adds to the growing evidence that organizational social context must be addressed in efforts to provide effective mental health treatment services through implementation of the most effective treatments. By improving the organizational social context for EBP we not only improve the work environments for clinicians, but also support the implementation of effective clinical services that decrease symptoms and improve functioning and quality of life of those who are served.

Competing interests

Gregory A. Aarons is an Associate Editor of Implementation Science. The authors declare that they have no other competing interests.

Authors’ contributions

All authors were involved in the design of the research study. GAA and CG conceptualized the specific research questions and analytic approach for this manuscript. Analyses were conducted by CG and PG. All authors wrote, edited, and revised portions of the manuscript. The MacArthur Research Network on Youth Mental participated in the conceptualization and research design of the study. All authors read and approved the final manuscript.

Acknowledgements

This research was supported by the MacArthur Research Network on Youth Mental Health (PI: John R. Weisz, Network Chair) and by NIMH grants R21MH082731 and 2R01MH072961 (PI: GAA), R01MH084855 (PI: CG) and P30MH074678 (PI: JL). The authors thank The Research Network on Youth Mental Health whose members at the time this work was performed included John R. Weisz, Judge Baker Children’s Center, Harvard University; Bruce Chorpita, Department of Psychology, University of California, Los Angeles; Robert Gibbons, Center for Health Statistics, University Chicago; Charles Glisson; Evelyn Polk Green, Chicago Public Schools Office of Early Childhood Education, Chicago, Illinois; Kimberly Hoagwood; Peter S. Jensen, Center for the Advancement of Children’s Mental Health, Columbia University College of Physicians and Surgeons; Kelly Kelleher; John Landsverk; Stephen Mayberg, California Department of Mental Health, Sacramento, California; Jeanne Miranda, Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles; Lawrence Palinkas, Schools of Social Work, Anthropology, and Preventive Medicine, University of Southern California; Sonja Schoenwald, Department of Psychiatry and Behavioral Sciences and the Family Services Research Center, Medical University of South Carolina.

References

  1. Aarons GA, Hurlburt M, Horwitz SM: Advancing a conceptual model of evidence-based practice implementation in child welfare.

    Adm Policy Ment Health and Ment Health Serv Res 2011, 38(1):4-23. Publisher Full Text OpenURL

  2. Bond GB, Becker DR, Drake RE, Rapp CA, Meisler N, Lehman AF, Bell MD, Blyler CR: Implementing supported employment as an evidenced-based practice.

    Psychiatr Serv 2001, 52:313-322. PubMed Abstract | Publisher Full Text OpenURL

  3. Glisson C, Schoenwald S: The ARC organizational and community intervention strategy for implementing evidence-based children's mental health treatments.

    Ment Health Serv Res 2005, 7(4):243-259. PubMed Abstract | Publisher Full Text OpenURL

  4. Raghavan R, Bright CL, Shadoin AL: Toward a policy ecology of implementation of evidence-based practices in public mental health settings.

    Implement Sci 2008, 3(26):3-26. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  5. Glisson C, Schoenwald S, Kelleher K, Landsverk J, Hoagwood K, Mayberg S, Green P: Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate and service structure.

    Adm Policy Ment Health and Ment Health Serv Res 2008, 35:124-133. Publisher Full Text OpenURL

  6. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffusion of innovations in service organizations: Systematic review and recommendations.

    Milbank Q 2004, 82(4):581-629. PubMed Abstract | Publisher Full Text OpenURL

  7. Grol R, Wensing M: What drives change? Barriers to and incentives for achieving evidence-based practice.

    Med J Aust 2004, 180:S57-S60. PubMed Abstract | Publisher Full Text OpenURL

  8. Mendel P, Meredith L, Schoenbaum M, Sherbourne C, Wells K: Interventions in organizational and community context: a framework for building evidence on dissemination and implementation in health services research.

    Adm Policy Ment Health and Ment Health Serv Res 2008, 35(1–2):21-37. OpenURL

  9. Glisson CA: The organizational context of children's mental health services.

    Clin Child Fam Psychol Rev 2002, 5(4):233-253. PubMed Abstract | Publisher Full Text OpenURL

  10. Brown SP: A meta-analysis and review of organizational research on job involvement.

    Psychol Bull 1996, 120(2):235-255. OpenURL

  11. James LR, Jones AP: Organizational climate: A review of theory and research.

    Psychol Bull 1974, 81(12):1096-1112. OpenURL

  12. Glisson C, Durick M: Predictors of job satisfaction and organizational commitment in human service organizations.

    Adm Sci Q 1988, 33(1):61-81. Publisher Full Text OpenURL

  13. Aarons GA, Sawitzky AC: Organizational climate partially mediates the effect of culture on work attitudes and staff turnover in mental health services.

    Adm Policy Ment Health and Ment Health Serv Res 2006, 33(3):289-301. Publisher Full Text OpenURL

  14. Parmelli E, Flodgren G, Beyer F, Baillie N, Schaafsma ME, Eccles MP: The effectiveness of strategies to change organisational culture to improve healthcare performance: a systematic review.

    Implement Sci 2011., 6(33) OpenURL

  15. Aarons GA, Sawitzky AC: Organizational culture and climate and mental health provider attitudes toward evidence-based practice.

    Psychol Serv 2006, 3(1):61-72. PubMed Abstract | PubMed Central Full Text OpenURL

  16. Glisson C, Landsverk J, Schoenwald SK, Kelleher K, Hoagwood KE, Mayberg S, Green P: Assessing the Organizational Social Context (OSC) of mental health services: Implications for research and practice.

    Adm Policy Ment Health and Ment Health Serv Res 2008, 35(1–2):98-113. OpenURL

  17. Jameson JP, Chambless DL, Blank MB: Empirically supported treatments in rural community mental health centers: A preliminary report on current utilization and attitudes toward adoption.

    Community Ment Health J 2009, 45(6):463-467. PubMed Abstract | Publisher Full Text OpenURL

  18. Kirsh B: Organizational culture, climate and person-environment fit: Relationships with employment outcomes for mental health consumers.

    Work: A Journal of Prevention, Assessment and Rehabilitation 2000, 14(2):109-122. OpenURL

  19. Glisson C, Hemmelgarn A: The effects of organizational climate and interorganizational coordination on the quality and outcomes of children's service systems.

    Child Abuse Negl 1998, 22(5):401-421. PubMed Abstract | Publisher Full Text OpenURL

  20. Glisson C, Green P: Organizational climate, services and outcomes in child welfare systems.

    Child Abuse Negl

    in press

    OpenURL

  21. Aarons GA, Cafri G, Lugo L, Sawitzky A: Expanding the domains of attitudes towards evidence-based practice: The Evidence-based Practice Attitude Scale-50.

    Adm Policy Ment Health and Ment Health Serv Res 2010.

    Online first, 2010, DOI: 10.1007/s10488-010-0302-3

    OpenURL

  22. Borntrager CF, Chorpita BF, Higa-McMillan C, Weisz JR: Provider attitudes toward evidence-based practices: Are the concerns with the evidence or with the manuals?

    Psychiatr Serv 2009, 60:677-681. PubMed Abstract | Publisher Full Text OpenURL

  23. Aarons GA: Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS).

    Ment Health Serv Res 2004, 6(2):61-74. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  24. Aarons GA, McDonald EJ, Sheehan AK, Walrath-Greene CM: Confirmatory factor analysis of the evidence-based practice attitude scale (EBPAS) in a geographically diverse sample of community mental health providers.

    Adm Policy Ment Health and Ment Health Serv Res 2007, 34:465-469. Publisher Full Text OpenURL

  25. Aarons GA, Green AE, Miller E: Researching readiness for implementation of evidence-based practice: A comprehensive review of the evidence-based practice attitude scale (EBPAS). In Cambridge handbook of implementation science for psychology in education. Edited by Kelly B, Perkins D. Cambridge University Press, Cambridge and New York; 2012. OpenURL

  26. Aarons GA, Glisson C, Hoagwood K, Kelleher K, Landsverk J, Cafri G: Psychometric properties and United States national norms of the Evidence-Based Practice Attitude Scale (EBPAS).

    Psychol Assess 2010, 22(2):356-365. PubMed Abstract | Publisher Full Text OpenURL

  27. Aarons GA: Transformational and transactional leadership: Association with attitudes toward evidence-based practice.

    Psychiatr Serv 2006, 57(8):1162-1169. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  28. Stahmer AC, Aarons GA: Attitudes toward adoption of evidence-based practices: A comparison of Autism early intervention teachers and children’s mental health providers.

    Psychol Serv 2009, 6(3):223-234. PubMed Abstract | PubMed Central Full Text OpenURL

  29. Melas CD, Zampetakis LA, Dimopoulou A, Moustakis V: Evaluating the properties of the Evidence-Based Practice Attitude Scale (EBPAS) in health care.

    Psychol Assess

    In press

    OpenURL

  30. Nelson MM, Shanley JR, Funderburk BW, Bard E: Therapists' attitudes toward evidence-based practices and implementation of parent–child interaction therapy.

    Child Maltreat 2012, 17(1):47-55. PubMed Abstract | Publisher Full Text OpenURL

  31. Smith BD, Manfredo IT: Frontline counselors in organizational contexts: A study of treatment practices in community settings.

    J Subst Abuse Treat 2011, 41(2):124-136. PubMed Abstract | Publisher Full Text OpenURL

  32. Henggeler SW, Chapman JE, Rowland MD, Halliday-Boykins CA, Randall J, Shackelford J, Schoenwald SK: Statewide adoption and initial implementation of contingency management for substance abusing adolescents.

    J Consult Clin Psychol 2008, 76(4):556-567. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  33. Stronge M, HCahill M: Self-reported knowledge, attitudes and behaviour toward evidence-based practice of occupational therapy students in Ireland.

    Occup Ther Int 2012, 19(1):7-16. PubMed Abstract | Publisher Full Text OpenURL

  34. Risahmawati RR, Emura SS, Nishi TT, Koizumi SS: Japanese resident physicians' attitudes, knowledge, and perceived barriers on the practice of evidence based medicine: A survey.

    BMC Res Notes 2011, 4:374-383. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  35. Davies KS: Evidence-based medicine: UK doctors' attitudes and understanding.

    Int J of Evid Based Healthcare 2011, 9(3):265-273. OpenURL

  36. Dopson S, Fitzgerald L: The role of the middle manager in the implementation of evidence-based health care.

    J Nurs Manag 2006, 14(1):43-51. PubMed Abstract | Publisher Full Text OpenURL

  37. Dopson S, FitzGerald L, Ferlie E, Gabbay J, Locock L: No magic targets! Changing clinical practice to become more evidence based.

    Health Care Manage Rev 2010, 35(1):2-12. PubMed Abstract | Publisher Full Text OpenURL

  38. Saldana L, Chapman JE, Henggeler SW, Rowland MD: The organizational readiness for change scale in adolescent programs: Criterion validity.

    J Subst Abuse Treat 2007, 33(2):159-169. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  39. Aarons GA, Sommerfeld DH, Walrath-Greene CM: Evidence-based practice implementation: The impact of public vs. private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice.

    Implement Sci 2009, 4(83):1-13. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  40. Schein E: Organizational culture and leadership. 4th edition. John Wiley and Sons, San Francisco, CA; 2010. OpenURL

  41. Schoenwald SK, Chapman JE, Kelleher K, Hoagwood KE, Landsverk J, Stevens J, Glisson C, Rolls-Reutz J: A survey of the infrastructure for children’s mental health services: Implications for the implementation of empirically supported treatments (EBTs).

    Adm Policy Ment Health and Ment Health Serv Res 2008, 35(1-2):78-83. OpenURL

  42. Webb MB, Dowd K, Harden BJ, Landsverk J, Testa MF: Child welfare and child well-being: New perspectives from the national survey of child and adolescent well-being. Oxford University Press, New York; 2010. OpenURL

  43. Glisson CA, Martin PY: Productivity and efficiency in human service organizations as related to structure, size, and age.

    Acad Manage J 1980, 23(1):21-37. PubMed Abstract | Publisher Full Text OpenURL

  44. Glisson C, Green P: The effects of organizational culture and climate on the access to mental health care in child welfare and juvenile justice systems.

    Adm Policy Ment Health and Ment Health Serv Res 2006, 33(4):433-448. Publisher Full Text OpenURL

  45. Glisson C, James LR: The cross-level effects of culture and climate in human service teams.

    J Organ Behav 2002, 23:767-794. Publisher Full Text OpenURL

  46. Glisson CA: Dependence of technological routinization on structural variables in human service organizations.

    Adm Sci Q 1978, 23(3):383-395. PubMed Abstract | Publisher Full Text OpenURL

  47. James LA, James LR: Integrating work environment perceptions: Explorations into the measurement of meaning.

    J Appl Psychol 1989, 74(5):739-751. OpenURL

  48. Glisson C, Schoenwald S, Kelleher K, Landsverk J, Hoagwood K, Mayberg S, Green P: Assessing the Organizational Social Context (OSC) of mental health services: Implications for research and practice.

    Adm Policy Ment Health and Ment Health Serv Res Special Issue: Improving mental health services 2008, 35(1–2):98-113. OpenURL

  49. Raudenbush SW, Bryk AS, Cheong YF, Congdon R: HLM 6: Hierarchical Linear and Nonlinear Modeling. Scientific Software International, Lincolnwood, IL; 2004. OpenURL

  50. Glisson C, Schoenwald SK, Hemmelgarn A, Green P, Dukes D, Armstrong KS, Chapman JE: Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy.

    J Consult Clin Psychol 2010, 78(4):537-550. PubMed Abstract | Publisher Full Text OpenURL

  51. O'Reilly CA, Caldwell DF, Chatman JA, Lapiz M, Self W: How leadership matters: The effects of leaders' alignment on strategy implementation.

    Leadersh Q 2010, 21(1):104-113. Publisher Full Text OpenURL

  52. Glisson C, Dukes D, Green P: The effects of the ARC organizational intervention on caseworker turnover, climate, and culture in children's service systems.

    Child Abuse Negl 2006, 30(8):849-854. Publisher Full Text OpenURL

  53. Schneider B: Welcome to the world of services management.

    Acad Manage J 2004, 18(2):144-150. OpenURL

  54. Aarons GA, Sommerfeld DH: Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation.

    J Am Acad Child Adolesc Psychiatry 2012, 51(4):423-431. PubMed Abstract | Publisher Full Text OpenURL

  55. McPherson M, Smith-Lovin L, Cook JM: Birds of a feather: Homophily in social networks.

    Annu Rev Sociol 2001, 27:415-444. Publisher Full Text OpenURL

  56. Moynihan DP, Pandey SK: The ties that bind: Social networks, person-organization value fit, and turnover intention.

    Journal of Public Administration Research and Theory 2008, 18(2):205-227. OpenURL

  57. de Vries H, Dijkstra M, Kuhlman P: Self-efficacy: the third factor besides attitude and subjective norm as a predictor of behavioural intentions.

    Health Educ Res 1988, 3(3):273-282. Publisher Full Text OpenURL

  58. Ajzen I: Perceived behavioral control, self-efficacy, locus of control, and the theory of planned behavior.

    J Appl Soc Psychol 2002, 32(4):665-683. Publisher Full Text OpenURL

  59. Klein KJ, Sorra JS: The challenge of innovation implementation.

    Acad Manage Rev 1996, 21(4):1055-1080. OpenURL

  60. Thoresen CJ, Kaplan SA, Barsky AP, Warren CR, de Chermont K: The affective underpinnings of job perceptions and attitudes: a meta-analytic review and integration.

    Psychol Bull 2003, 129(6):914-945. PubMed Abstract | Publisher Full Text OpenURL

  61. Aarons GA, Fettes DL, Sommerfeld DH, Palinkas LA: Mixed methods for implementation research: application to evidence-based practice implementation and staff turnover in community-based organizations providing child welfare services.

    Child Maltreat 2012, 17(1):67-79. PubMed Abstract | Publisher Full Text OpenURL

  62. Creswell JW: Plano Clark VL: Designing and conducting mixed methods research. Sage Publications, Thousand Oaks, CA; 2007. OpenURL

  63. Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J: Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science.

    Implement Sci 2009, 4:50. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  64. Rycroft-Malone J, Burton C: Paying attention to complexity in implementation research.

    Worldviews Evid Based Nurs 2010, 7(3):121-122. PubMed Abstract | Publisher Full Text OpenURL

  65. Jacobs JA, Dodson EA, Baker EA, Deshpande AD, Brownson RC: Barriers to evidence-based decision making in public health: a national survey of chronic disease practitioners.

    Public Health Rep 2010, 125(5):736-742. PubMed Abstract | PubMed Central Full Text OpenURL

  66. Glisson C, Hemmelgarn A, Green P, Dukes D, Atkinson S, Williams NJ: Randomized trial of the ARC organization intervention with community-based mental health programs and clinicians serving youth.

    J Am Acad Child Adolesc Psychiatry 2012, 51(3):780-787.

    In press

    PubMed Abstract | Publisher Full Text OpenURL