Email updates

Keep up to date with the latest news and content from Implementation Science and BioMed Central.

Open Access Highly Accessed Systematic review

To what extent do nurses use research in clinical practice? A systematic review

Janet E Squires1*, Alison M Hutchinson2, Anne-Marie Boström3, Hannah M O'Rourke4, Sandra J Cobban45 and Carole A Estabrooks4

Author Affiliations

1 Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada

2 Faculty of Nursing, Deakin University, and Cabrini-Deakin Centre for Nursing Research, Cabrini Institute, Cabrini Health, Melbourne, Australia

3 Department of Neurobiology, Care Sciences and Science, Division of Nursing, Karolinska Institutet, Stockholm, Sweden

4 Faculty of Nursing, University of Alberta, Edmonton, Alberta, Canada

5 Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada

For all author emails, please log on.

Implementation Science 2011, 6:21  doi:10.1186/1748-5908-6-21


The electronic version of this article is the complete one and can be found online at: http://www.implementationscience.com/content/6/1/21


Received:19 July 2010
Accepted:17 March 2011
Published:17 March 2011

© 2011 Squires et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

In the past forty years, many gains have been made in our understanding of the concept of research utilization. While numerous studies exist on professional nurses' use of research in practice, no attempt has been made to systematically evaluate and synthesize this body of literature with respect to the extent to which nurses use research in their clinical practice. The objective of this study was to systematically identify and analyze the available evidence related to the extent to which nurses use research findings in practice.

Methods

This study was a systematic review of published and grey literature. The search strategy included 13 online bibliographic databases: Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, MEDLINE, CINAHL, EMBASE, HAPI, Web of Science, SCOPUS, OCLC Papers First, OCLC WorldCat, ABI Inform, Sociological Abstracts, and Dissertation Abstracts. The inclusion criteria consisted of primary research reports that assess professional nurses' use of research in practice, written in the English or Scandinavian languages. Extent of research use was determined by assigning research use scores reported in each article to one of four quartiles: low, moderate-low, moderate-high, or high.

Results

Following removal of duplicate citations, a total of 12,418 titles were identified through database searches, of which 133 articles were retrieved. Of the articles retrieved, 55 satisfied the inclusion criteria. The 55 final reports included cross-sectional/survey (n = 51) and quasi-experimental (n = 4) designs. A sensitivity analysis, comparing findings from all reports with those rated moderate (moderate-weak and moderate-strong) and strong quality, did not show significant differences. In a majority of the articles identified (n = 38, 69%), nurses reported moderate-high research use.

Conclusions

According to this review, nurses' reported use of research is moderate-high and has remained relatively consistent over time until the early 2000's. This finding, however, may paint an overly optimistic picture of the extent to which nurses use research in their practice given the methodological problems inherent in the majority of studies. There is a clear need for the development of standard measures of research use and robust well-designed studies examining nurses' use of research and its impact on patient outcomes. The relatively unchanged self-reports of moderate-high research use by nurses is troubling given that over 40 years have elapsed since the first studies in this review were conducted and the increasing emphasis in the past 15 years on evidence-based practice. More troubling is the absence of studies in which attempts are made to assess the effects of varying levels of research use on patient outcomes.

Background

Scholars have expressed long-held concerns about whether nurses' practice is in accordance with the best available scientific evidence [1-9]. The disparity between the findings of research evidence and actual practice is frequently referred to as the research-practice gap [6,10-12]. Despite increasing quantities of, and more convenient access to, clinically relevant research, the slow and haphazard uptake or failure to adopt such evidence persists. Many examples of the research-practice gap have been highlighted in the nursing literature over the past thirty years [13-15]. However, most of the evidence is anecdotal, highlighting the difficulties surrounding attempts to measure whether or not a practice is based on research [16]. Bostrom and Wise [17] suggested that the research-practice gap is in the order of ten to fifteen years, while Landrum [18] proposed that an eight to thirty year time lag exists between publication and adoption of research findings.

Concerns about this gap are related to widely held assumptions that patients who receive evidence-based care achieve better outcomes. There is some evidence in support of this assumption [19-22]. However, because such evidence has often resulted from studies conducted under research conditions, Estabrooks [6] recommends caution when drawing this conclusion. Estabrooks argued that we do not know whether and to what extent nurses adapt research findings according to the circumstances and context in which they practice, and consequently, the effectiveness of nursing interventions under such conditions is largely unknown. The gold standard of evidence in healthcare intervention (including interventions to promote research utilization) research is commonly held to be the prospective randomized controlled trial (RCT). RCTs can be either explanatory or pragmatic in nature. Explanatory trials test whether an intervention is efficacious (i.e., whether it is beneficial in an 'ideal' situation), while pragmatic trials measure effectiveness (the degree of beneficial effect in 'real' practice). Hence, the pragmatic trial is more often a reflection of the 'real world' and therefore, if used, would address the concerns raised by Estabrooks [6] about the extent to which nurses adapt research findings according to the circumstances and context in which they practice.

Research utilization is defined by Estabrooks and colleagues [23] as 'that process by which specific research-based knowledge (science) is implemented in practice.' It is a complex and multi-facetted construct, as evidenced by the multiple and diverse conceptualizations that abound the literature. For instance, while some scholars define research utilization as a general or omnibus construct [24,25], others describe it as the use of specific research-based findings or practices [26,27]. Within these views, research utilization is often conceptualized further as a variable (or discrete event) [24,28,29] or a process (as consisting of a number of consecutive steps or stages) [26,30]. Some scholars, in addition to using a variable conceptualization, also propose several different kinds of research utilization exist; instrumental, conceptual, and persuasive uses of research have been described [31-33]. Instrumental utilization refers to the concrete application of specific knowledge to practice; conceptual utilization refers to a change in thinking, but not necessarily behavior, in response to research findings; and persuasive utilization refers to the use of certain knowledge to persuade others regarding a predetermined position [23,34]. Estabrooks [29] embarked on a study to explore and provide some empirical support for a conceptual structure of research utilization and concluded that 'instrumental, conceptual, and persuasive research utilization exist and that a global measure of research utilization may be defensible.' She argued, however, that we have little understanding of the correct measures of research utilization, and that the most common methods employed measure only instrumental research use [35]. She further contended that in failing to measure conceptual and persuasive research utilization, the findings of such studies underestimate nurses' overall research utilization (i.e., the use of any kind of research in any way in clinical practice) [29].

Several published reviews have identified the complexity of, and challenges associated with, integrating research evidence into practice [36,37]. A number of factors related to characteristics of the evidence, the individual practitioner, and the context in which care is delivered have been identified as being influential in the translation of research to practice. A 2003 review [38] reported six categories of potential individual determinants of research utilization: beliefs and attitudes, involvement in research activities, information seeking, professional characteristics, education, and other socio-economic factors. More recently, Meijers et al. [39] examined the relationships between characteristics of organizational context and research use. They reported statistically significant relationships between research use and six contextual factors: the role of the nurse, multi-faceted access to resources, organizational climate, multifaceted support, time for research activities, and provision of education. Despite these findings, attempts to measure the extent of research use have not captured the complexity of the phenomenon.

In addition to the reviews identified above, reviews exist on studies measuring research use by nurses [23], interventions designed to increase nurses' use of research [40], and instruments used to measure nurses' attitudes towards research use [41]. However, we could locate no reviews on the question of whether, and to what extent, nurses use research. The purpose of the systematic review reported in this paper was therefore to examine existing evidence on the extent to which nurses use research in clinical practice and by so doing, contribute to the body of work assessing the 'state of the science' in this field.

Methods

Inclusion criteria

Types of studies

Experimental (intervention) and non-experimental designs that examined the use of research by nurses in clinical practice were eligible for inclusion. An experiment was defined as 'a study in which an intervention is deliberately introduced to observed its effects' [42]. Experimental designs include RCTs, clinical trials, and quasi-experimental (e.g., pre/post test designs) studies. Non-experimental designs refer to observational studies (e.g., cohort, case-control, cross-sectional) [42]. Case reports and non-systematic/narrative literature reviews were excluded. Studies were limited to those published in English and Scandinavian languages -- the languages represented on the research team. There were no restrictions on the basis of country of origin or publication date.

Types of participants and outcomes

Studies that examined nurses' use of research in clinical practice were considered for inclusion. A nurse was defined as a professional who provides patient care in a clinical setting -- e.g., registered nurse (RN), registered psychiatric nurse (RPN), licensed practical nurse (LPN). The outcome of interest was use of research findings. Research, in this study, was defined as information that is empirically and systematically derived. The research findings could be reported in a primary research article, review/synthesis report, or protocol. Measures of research use needed to be expressed quantitatively.

We excluded articles that reported on nurses' adherence to clinical practice guidelines, the rationale being that clinical practice guidelines can be based on non-research evidence (e.g., expert opinion). We did not have the capacity to review the evidence base for recommendations provided in each guideline reported in the literature. We did however include nurses' use of protocols where the research-base of the protocol was made explicit in the research report. It is also possible that studies assessing research utilization using omnibus (general) questions may include nurses' use of guidelines if the nurse answered with a specific guideline in mind. We also excluded articles that reported on: predictors or barriers to research utilization if they did not also report on nurses' use of research in their practice; nurses' use of one specific research-based practice if the purpose of the study was not to examine nurses' use of research in practice; and studies in which research use of healthcare professionals other than nurses were included if a separate analysis of nurses' use of research was not provided or could not be extracted. We also excluded articles where a quantitative measure of the extent of research use was not provided or could not be derived from the data reported.

Search strategy for identification of studies

The search strategy for this review was developed in consultation with a health sciences librarian. We searched the following bibliographic databases: Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, CINAHL, MEDLINE, EMBASE, HAPI, Web of Science, SCOPUS, OCLC PapersFirst, OCLC WorldCat, ABI Inform, Sociological Abstracts, and Dissertation Abstracts. Key words and subject headings related to research use and known instruments to assess research use in nurses were identified prior to initiating the search. The derivatives of the search terms were captured with the use of truncation symbols appropriate to the respective databases searched. Due to the differences in meaning between the terms 'research utilization' and 'evidence-based practice' (i.e., evidence-based practice is sometimes used as a broader concept which can incorporate forms of knowledge other than research) we decided, in consultation with a health sciences librarian, to exclude the term 'evidence-based practice' from the search strategy. See Table 1 for a summary of the search strategy.

Table 1. Search Strategy

Study identification

Two team members (JES and HMO) independently screened the titles and abstracts of the 12,418 citations identified by the search strategy noted in Table 1 to identify potentially relevant studies. Full text copies were retrieved of all citations identified as: having potential relevance to the objective of the review and where there was insufficient information to make a decision as to relevance (n = 133). Two reviewers (JES and HMO) then independently assessed all retrieved articles for relevance. A total of 55 articles were retained. All discrepancies with respect to relevance were resolved through consensus.

Quality assessment

All included articles (n = 55) were independently assessed for methodological quality by two reviewers (two of JES, AMH, AMB, HMO). To assess methodological quality we adapted two previously used tools. Disagreements in quality assessment were resolved through consensus.

The first tool, the Estabrooks' Quality Assessment and Validity Tool for Cross-Sectional Studies, originally developed based on Cochrane guidelines (in existence in 2001) and the medical literature [43,44], has been used in other systematic reviews by our group [38,45]. The tool contains a maximum of 16 points and assesses studies in three core areas: sampling, measurement, and statistical analysis. To derive a final quality score for each article, we divided the total points scored by the total points possible (16 -- the number of points not applicable for the article). Each study was then classified as weak (≤0.50), moderate-weak (0.51 to 0.65), moderate-strong (0.66 to 0.79), or strong (≥0.80). This rating system was used in a recent review [45] and is based on a scoring system developed by de Vet et al. [46]. This tool was used to assess the methodological quality of all cross-sectional studies included in the review (n = 51). That is, all studies providing a descriptive snapshot at one point in time of the extent to which nurses use research in practice.

The second quality assessment tool used in this review was the Quality Assessment Tool for Quantitative Studies (http://www.city.hamilton.on.ca/phcs/EPHPP/ webcite), developed by the Effective Public Health Practice Project, Canada. This tool has been judged suitable to be used in systematic reviews of effectiveness (measuring interventions) [47], and been shown to have content and construct validity [48]. The tool assesses studies on the basis of six areas; the six areas are selection bias, study design, confounders, blinding, data collection methods, and withdrawals/drop-outs. Each article is scored as weak, moderate, or strong in each of these six areas according to preset criteria within the tool. The tool developers do not provide a means for calculating an overall quality score. However, in order to compare the quality scores for each included article that used an intervention design (assessed with this tool) to the included articles that used cross-sectional designs (assessed with Estabrooks' Quality Assessment and Validity Tool described above), we derived an overall quality score for each article. To derive this score, we assigned values of 1, 2, and 3 to the categorizations of weak, moderate, and strong respectively. A final quality score for each article was then obtained by dividing the summative score obtained by the total amount of points possible. Each study was classified as weak (1 to 1.5), moderate-weak (1.6 to 2.0), moderate-strong (2.1 to 2.5), or strong (>2.5) by applying the same categorization system used (and published) in the cross-sectional tool. The Quality Assessment Tool for Quantitative Studies tool was used to assess all intervention studies included in the review (n = 4). That is, all studies testing an intervention to improve nurses' use of research.

Data extraction

Two reviewers performed data extraction on all included articles; one reviewer extracted data, which was then checked for accuracy by a second reviewer. Data were extracted on: study design, country, sample and subject characteristics, setting, measure of research use, reliability and validity, main finings with respect to use of research, and the intervention (where applicable). For the four intervention studies, data were extracted on both pre- and post-research use scores. All disagreements in data extraction were resolved through consensus.

Data analysis

The use of many different measures of research use across different healthcare contexts prevented us from performing a meta-analysis. Therefore, the findings from the review are presented in narrative form. That is, we synthesized the extracted data descriptively, according to the type of measure used to assess nurses' use of research as follows: Nurses Practice Questionnaire (NPQ), Research Utilization Questionnaire (RUQ), other multi-item measures, and single-item measures. A sensitivity analysis, comparing findings from all reports with those rated moderate (moderate-weak and moderate-strong) and strong quality, was performed to assess the impact of methodological quality on the review findings; no significant differences were noted. Therefore, we have elected to report findings from all 55 included reports.

To determine the extent to which nurses use research in their practice across all included reports, we categorized the findings from each study onto a common metric: low research use, moderate-low research use, moderate-high research use, and high research use (these findings are summarized in Table 2). We did this by creating equal quartiles for each of the different scoring systems used in the selected articles. For example, 12 articles [5,26,27,33,49-56] used the NPQ to measure nurses' use of research. The NPQ provides a total innovation adoption behavior (TIAB) score for each nurse (which represents their overall research use) based on responses to a series of questions about specific research-based practices. The TIAB score can range from 0 to 4, with 4 indicating maximum research use; by equally dividing the possible range of scores into quartiles we were able to categorize the extent to which nurses use research in their clinical practice as follows: low (0 to 0.99), moderate-low (1.00 to 1.99), moderate-high (2.00 to 2.99), and high (3.0 to 4.00). We used similar processes to create quartiles for the research use scores provided in all 55 included articles. For the four intervention studies, we used the pretest research utilization scores to calculate quartiles. The pretest scores were used for two reasons: posttest scores could be lower or higher due to chance, and posttest scores may not be sustained overtime. Therefore, we hypothesized the pretest scores would be a more accurate reflection of the extent to which nurses use research in their clinical practice. A description of the processes used to calculate extent of research use in each of the included articles can be found in Additional Files 1, 2, 3 and 4. We elected to use this (quartile) method to synthesize the findings because it allowed us to compare all 55 included reports to provide an overall conclusion on the extent to which nurses use research use in practice. While a few instruments (e.g., NPQ, RUQ) were used in multiple studies, we elected not to calculate additional summary statistics for these instruments because: it would not progress our overall aim of determining the extent to which nurses use research overall, several of the multi-use instruments were modified significantly, limiting ability to combine scores, and there was little variability in scores between studies using the same instrument, and therefore, it would not provide added value.

Table 2. Summary of Findings

Additional file 1. Characteristics of articles using the NPQ to assess research use. A summary of data extraction and extent calculation on studies that used the NPQ.

Format: DOC Size: 53KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

Additional file 2. Characteristics of articles using the RUQ to assess research use. A summary of data extraction and extent calculation on studies that used the RUQ.

Format: DOC Size: 56KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

Additional file 3. Characteristics of articles using other multi-item measures to assess research use. A summary of data extraction and extent calculation on studies that used other multi-item instruments.

Format: DOC Size: 51KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

Additional file 4. Characteristics of articles using single-item measures of research use. A summary of data extraction and extent calculation on studies that used single-item measures of research utilization.

Format: DOC Size: 140KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

Results

Description of studies

The database search yielded 12,418 unique (after removal of duplicate articles) titles and/or abstracts. Of these, 133 were identified as potentially relevant and were retrieved in full text. From the 133 retrieved articles, 78 did not meet our inclusion criteria (see Additional File 5). This resulted in a final sample of 55 articles (see Figure 1) representing 47 individual studies. Of the final 55 articles, the majority (n = 51) reported a cross-sectional survey design [4,5,9,24,26,27,29,33,50-88], while the remainder (n = 4) used a quasi-experimental design [89-92].

Additional file 5. Excluded Articles based on Full-Text (n = 78). Citation and reason for exclusion for 78 articles retrieved but not included in the review.

Format: DOC Size: 99KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

thumbnailFigure 1. Search and Retrieval Process.

The majority of articles examine research use by nurses in North America (n = 39, 71%) followed by Europe (n = 12, 22%), Asia (n = 3, 5%) and Oceania (n = 1, 2%). Most studies were conducted in acute care (hospital) settings (n = 44, 80%) followed by multiple settings (e.g., sampling through a register) (n = 9, 16%), educational programs (n = 1, 2%) and nursing homes (n = 1, 2%). With respect to year of publication, the first report included in this review was published in 1981 [90]. Earlier reports on research use in nurses dating back to the 1970's [93] were excluded from this review because their purpose was to measure the use of a practice and not the concept of research use per se. There has been a trend of increased published reports on nurses' use of research in the past decade, with 38 (69%) of the 55 reports in this review being published between 1996 and 2007. The number of published articles peaked in 1995 to 1999, followed by a gradual fall in the early 2000's (Figure 2); this may be due, in part, to a shift in focus by some researchers away from research utilization and towards evidence-based practice. Examination of the articles in chronological order reveals that early studies found low, moderate-low, or moderate-high research use only (Figure 2). The first study in which high research use was found was published in 1996 [53] and since then moderate-low, moderate-high, and high reports of research use have been published, with no studies falling into the low research use category. Characteristics of the 55 included articles can be found in Additional Files 1, 2, 3, and 4.

thumbnailFigure 2. Extent of research use by year of publication (n = 55 articles).

Methodological quality of studies

Methodological quality of the 55 included articles is presented in Additional File 6. Of the included reports, 3 (5%) were rated as strong [27,76,86], 12 (22%) as moderate-strong [5,26,29,50-52,54-56,66,91,94], 23 (42%) as moderate-weak [9,24,33,49,53,61,68-70,72,73,77,78,80,81,83,84,87-89,92,95,96], and 17 (31%) as weak [4,57,59,60,63-65,67,71,74,75,79,82,85,90,97,98]. Discrepancies in quality assessment between reviewers related primarily to representativeness of the sample, treatment of missing data, and appropriateness of statistical tests, and were settled through consensus. In the majority of articles, the research use measures employed were reported to be reliable and valid. However, stringent assessments were not performed. Reliability (of the multi-item measures) was limited to tests of internal consistency (Cronbach's alpha) while assessment of validity (of both multi and single items) was predominantly limited to content validity and in many cases was 'assumed' based on a review of the literature and/or a statement from the index (first) study of the measure stating it was a valid instrument. Other common methodological weaknesses included: low response rates in cross-sectional studies (acceptable response rates (>50%) were only achieved by 42% (n = 21) of the cross sectional studies), and minimal use of control for confounding factors in the study design and/or analysis.

Additional file 6. Quality Assessment. A summary of quality assessment of the 55 articles included in the review.

Format: DOC Size: 143KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

Study outcome: extent of research utilization

Because nurses' use of research was assessed with several different measures of research use, we have grouped findings on the extent of research use according to the class of instrument used in its assessment. A summary of these findings is presented in Table 2. Within each instrument group we also emphasize the extent of research use according to whether it was measured as research use in general, the use of specific research findings, or according to a kind of research use -- i.e., instrumental, conceptual, persuasive, or overall research use.

Nurses practice questionnaire

Twelve articles (ten studies) [5,26,27,33,49-56] assessed nurses' use of research using the NPQ. The NPQ consists of brief descriptions of a set of specific nursing practice innovations (research-based practices). Six questions, which measure the nurse's stage of innovation adoption, are posed for each of the nursing practice innovations. Nurses are then classified as being unaware of, aware of, persuaded of, use sometimes, or use always for each of the practices and for all practices overall. While the adoption scores varied slightly by the specific practices assessed in the included studies, overall adoption scores were similar across studies. Nurses in nine [5,26,27,33,50,51,54-56] of the twelve NPQ articles, on average, reported some use of the practices and were classified as being in the 'persuasion' stage of adoption overall, according to the TIAB classification system developed by Brett [26]. With respect to our extent of research use classification, most NPQ articles (10 of 12) [5,26,27,33,49,50,52,54-56] fell in the moderate-high research use category; one study [51] fell in the moderate-low research use category, while another article [53] fell in the high research use category. Characteristics of the twelve articles using the NPQ to assess nurses' research use can be found in Additional File 1.

Research utilization questionnaire

Ten articles (ten studies) [57,63,67-69,71,75,83,91,95] assessed nurses' use of research using the RUQ. The RUQ, developed by Champion and Leach [63], is a general measure of research use consisting of 42 self-descriptive statements comprising four subscales of which research utilization is one subscale. The research utilization subscale contains ten items, each scored on a 5-point Likert scale, assessing the degree to which nurses' perceive they incorporate research findings into their daily practice. The ten items are then summed and a mean is taken to obtain an overall research use score, with higher values indicating higher levels of research use. Eight [63,67-69,71,83,91,95] of the ten articles reported an overall score indicative of moderate-high research use. The remaining two articles [57,75] reported moderate-low research use scores. Characteristics of the ten articles using the RUQ to assess nurses' research use can be found in Additional File 2.

Other multi-item measures

We located an additional five articles (four studies) [73,74,85,87,90] using different multi-item measures to assess nurses' use of research. All articles reported moderate research use. One article [90] reported moderate-low research use in general, while three articles [73,74,85] reported moderate-high research use in general. The remaining article [87] in this category assessed both nurses' use of specific research-based practices (with moderate-high research use scores) and their use of research in general (with moderate-low research use scores). Characteristics of the five articles using individual multi-item measures to assess nurses' research use are found in Additional File 3.

Single-item measures

In 28 published papers (23 studies), investigators used single-item questions to assess nurses' research use. A combination of specific practices and general research use were targeted with the single-item questions. We further categorized the single-item questions as follows: past, present, and future use; Parahoo's measure; Estabrooks' kinds of research use; and other single items. Characteristics of the 28 articles using single-item measures are found in Additional File 4.

Past, present, and future research use

Four articles (four studies) [4,59-61] assessed the proportion of nurses reporting past use of research in general (more than six months ago), present use of research in general (most recent six months), and/or future (within the next year) intention to use research in general. While intention to use research in the future was reported as moderate-low [4] to high [59], current use was reported as low at 15.9% [4,60] or moderate-low at 30.3% [59]. In all studies, past use was reported slightly higher at 23.4% [60], 24.6% [4], and 52.6% [59], compared to present use. One article, Brown [61], assessed past use without assessing present use and found that the extent of reported use was moderate-high but varied by type of use: past use of research to change practice (66%), and past use for patient care (71%). Similar to previous studies, Brown [61] also found future intention to use research to be very high (86%) (See Additional File 4).

Parahoo's measure

Seven articles (three studies) [9,24,78-81,86] assessed nurses' use of research in general using a single-item question developed by Parahoo [24]. This question asked nurses to indicate the frequency with which they used research in clinical practice. The majority of nurses (50.0% to 54.7%), regardless of context (setting) or role, reported moderate research use, with three articles [78,80,86] reporting scores indicative of moderate-low research use and four articles [9,24,79,81] reporting scores indicative of moderate-high research use (See Additional File 4).

Estabrooks' kinds of research use

Six articles (five studies) [25,64,66,70,76,84] assessed nurses' use of four kinds of research use: instrumental, conceptual, persuasive, and overall. Five [25,64,66,70,84] of the six articles used a 7-point frequency scale; mean scores showed that nurses, on average, reported using research on half of their shifts (score of 5). Nurses also commonly reported using research conceptually and overall more frequently than instrumentally and persuasively. For instance, Estabrooks [29] reported mean scores of 5.20, 4.71, 4.36, and 3.60 (on a 7-point scale) for conceptual, overall, instrumental, and persuasive research use, respectively. The remaining study [76] used a 5-point frequency scale and found a similar pattern for staff nurses' use of research. The extent of research use for nurses reporting kinds of research use ranged from moderate-low to high depending on the kind of research use (with conceptual use generally scoring higher) and nurse group (educator groups generally scored higher compared to staff nurses) (See Additional File 4 and Table 2).

Other single items

Eleven additional articles (eleven studies) [65,72,77,82,88,89,92,94,96-98] assessed nurses' use of research using their own single-item question(s), which have not been used in subsequent studies. Of these studies, one measured use of specific practices [99], while the remainder measured nurses' use of research in general. Findings varied widely from a low extent of research use [89] to a high extent of research use [65,77]. However, most studies reported moderate-low or moderate-high research use, with some studies reporting both levels of moderate use, depending on the item used to assess research use (See Additional File 4).

Extent of research use in general, use of specific research findings and research use according to kinds

The use of research in general was measured in 36 studies, including those in which the RUQ was used [57,63,67-69,71,75,83,91,95], studies that used other multi-item measures [73,74,85,87,90], single-item (past present and future) measures [4,59-61], the single-item Parahoo measure [9,24,78-81,86], and other single-item measures [58,72,77,82,88,89,92,96-98] (Table 2). Research use in general was found to range from low through to high use. Specifically, two studies reported low use [4,60], one study [89] reported low and moderate-low use on the basis of responses to two separate survey items, one study [59] reported moderate-low and moderate-high use dependent on the role of nurses, and one study [77] reported high use. The remainder reported moderate-low (n = 13) and/or moderate-high (n = 20) use (Table 2). Figure 3 illustrates the extent of research use in general by year of publication. A peak in the number of articles reporting general research use occurred in 1995 to 1999, and reports of high general research use starting in 2005.

thumbnailFigure 3. Extent of research use in general by year of publication (N = 36 articles). Three studies (Butler, 1995; Linde, 1989; and Pettingill, 1994) report two levels of research use, either based on responses to separate survey questions or for categories of nurses. Thus, 36 articles account for 39 measures reported in this figure.

Use of specific research findings was measured in 14 studies, including those in which the NPQ was administered [5,26,27,33,49-56], one study that used a other multi-item measure [87], and an additional study that employed a single-item measure [99]. Use of specific research ranged from moderate-low to high with the overwhelming majority of studies reporting moderate-high research use. The study that used a single-item measure [99] reported moderate-low, moderate-high or high use, dependent on the practices assessed. Figure 4 illustrates the extent of use of specific research findings by year of publication. Similar to reports of general research use, there was a peak in publications during the 1995 to 1999 year range. The only reports of high use of specific research findings occurred between 1995 and 1999.

thumbnailFigure 4. Extent of use of specific research findings by year of publication (n = 14 articles). One study (Davies, 1999) reports three levels of research use dependent on practices assessed. Thus, 14 articles account for 16 measures reported in this figure.

Kinds (including instrumental, conceptual, persuasive and overall) of research use were measured in six studies [25,64,66,70,76,84], each of which employed Estabrooks' measures. Across these studies, instrumental research use was reported as moderate-high to high, conceptual and persuasive research use ranged from moderate-low to high, and overall research use ranged from moderate-high to high. The majority of publications examining kinds of research use are from 2005 onwards. There were no reports of low research use; the first reports of high research use occurred in 2003 (conceptual research use), 2005 (instrumental and overall research use), and 2007 (persuasive research use).

Discussion

The various ways in which research use is conceptualized (i.e., as a process or an outcome, as a general concept or as kinds -- instrumental, conceptual, persuasive, overall) coupled with the use of multiple instruments to assess nurses' use of research, challenges clinicians' and investigators' ability to directly compare findings from various studies to determine the extent to which nurses use research in clinical practice. In this review, by quantifying nurses' use of research as low, moderate-low, moderate-high, or high, we were able to indirectly compare the results of the 55 included articles and conclude that the extent to which nurses report using research in clinical practice is, on average, moderate-high (with 38 of the 55 articles reporting research use in the moderate-high range) (Table 2). Caution must be used when interpreting this finding, however, because we combined different instruments (and conceptualizations of research use) in reaching this conclusion.

Specific versus general research use

An examination of the extent of research use elicited by different instruments revealed little variation in the scores regardless of whether nurses were asked to report on their use of specific research-based practices (e.g., NPQ) or on their use of research generally (e.g., RUQ). Most articles that used the NPQ (n = 10 of 12) were ranked in the moderate-high research use category. Of the ten articles that used the RUQ, eight were classified in the moderate-high category. Limited variation in reported research use for the NPQ and the RUQ suggests that an instrument effect may be at play. As such, a propensity towards moderately high use of research may reflect either a regression to the mean effect, or a lack of sensitivity of the instruments to detect changes in research use over time.

Anecdotally we know that nurses find it challenging to respond to general questions about the extent to which they use research in practice. Importantly, such omnibus questions require nurses to first be aware that they are using research. If nurses are using research but are not aware they are, this type of question should lead to under-reporting of research utilization. However, in this review, nurses reported, on average, moderate-high use. Instruments containing questions that relate to the use of specific research evidence, on the other hand, provide a context for respondents and enable them to relate their responses to their work. This was illustrated in a recent international study with nursing service providers in Canada and Sweden [100]. In this study, the investigators needed to provide concrete examples of research-based practices to stimulate nurses' reflection of their use of research in practice. This, in turn, however has the potential for increasing any social desirability effect that may exist.

Different kinds of research use

Six reports [25,64,66,70,76,84] in this review assessed the extent to which nurses reported different kinds of research use. Overall, use was highest for conceptual research use, followed by instrumental and persuasive use, with two exceptions [64,76]. The first exception is from a study of research use in Canadian nursing homes; Connor [64] reported high persuasive research use (RNs mean = 6.07, LPNs mean = 5.27 on a 7-point scale), followed by instrumental and lastly conceptual research use. This may be explained however by the context (setting/work environment) in which the nurses in this study were employed. Nurses in Canadian long-term care facilities have a largely supervisory role in overseeing the practice of healthcare aides who provide the majority of direct care. Therefore, nurses in this setting are more likely to use research persuasively in order to convince direct care providers (healthcare aides) to provide research-based care.

The second exception can be seen in a study conducted by Milner and colleagues [76]. In this study, Milner found that staff nurses and advanced practice nurses (educators and managers) both reported similar patterns (high conceptual followed by instrumental and persuasive use) in the extent to which they use research in clinical practice. However, the extent of use (for all kinds of research) reported by advanced practice nurses was higher when compared to staff nurses. Similar findings were also noted by Veeramah [9] in a study of graduate nurses and midwives in the United Kingdom. Veeramah [9] found that 67% of nurses reported a high extent of research use. The majority (63%) of nurses in this study, however, occupied senior positions with varying degrees of managerial responsibilities, autonomy, and authority, which may have been responsible for the higher extent of research use reported; nurses in management roles have greater authority to use research to implement change. These findings with respect to role are consistent with past research. While Estabrooks and colleagues [38] and Meijers and colleagues [39] located too few studies investigating role to reach a conclusion on its effect on research use, they did find consistent findings in the studies they located, with nurses in leadership roles reporting higher research use compared to staff nurses.

The state of the science when examining extent of research use

This review describes the range of measures of research use that have been used with nurses. It paints a somewhat discouraging portrait. Although the use of research evidence to underpin practice is viewed as fundamentally important, this review demonstrates several major limitations in this area of the field.

The first major limitation relates to methodological quality. Few studies examining nurses' use of research were strong (or even moderately strong) methodologically, illustrating a need for better-designed studies. Of the 55 articles included in this review, 51 reported a cross-sectional design. This design enables researchers to capture nurses' perceptions of their use of research at a single point in time. However, restricting study to cross-sectional research limits advances in the field. For example, evidence for causal inferences that can be used to develop interventions to increase nurses' use of research and consequently improve patient care is limited with cross-sectional designs. Four studies included in this review used a quasi-experimental design. All four studies measured research use pre- and post-implementation of an intervention designed to improve nurses' use of research in practice. Three of the studies implemented a control or comparison group alongside the experimental group, but reported little consideration of confounding variables, limiting the internal validity of the studies. Future studies in the field need to use more robust quasi-experimental and experimental (e.g., pragmatic RCTs) designs that take into consideration, and control for, threats to internal validity.

The second major limitation relates to the measures in use; these measures have several problems. First, there is inconsistency in the measures used, including widely varying use of language. While we were able to develop a method with which to compare findings on the extent of research use by dividing research use scores into quartiles, the lack of standard language makes it difficult to compare, contrast, and evaluate findings collected with the various instruments. Second, with the exception of the NPQ, none of the research use measures identified were developed explicitly using a relevant theoretical framework. As well, none of the studies examined reported the use of measurement theory (i.e., classical test score theory or item response theory [101,102]) in the design or evaluation of the instrument. Finally, all of the instruments used self-report measures of research use. The advantages of self-report are well known. Whether done using paper and pencil, online, or computer-assisted telephone or personal interview, it has the benefits of cost efficiency, convenience, and time efficiency for researchers. Despite these advantages, self-report measures are also often criticized. They are reports of 'perception' and therefore, not 'objective' measures. With respect to the measurement of research use, self-report instruments are further criticized because of an inability to clarify items and thus what is meant by 'research,' an inability to probe to more fully understand what nurses mean when they report research use (or non-use), and reduced ability by nurses to recall how often they use research. The most frequent criticisms however are that such measures offer the potential for social desirability bias [103] and rely on nurses' ability to recognize that they are using research. One way to reduce social desirability bias is to pay careful attention to instrument design (e.g., attention to item wording, item order, response options, and pre-testing) [103-105]. To this end, and positively, some of the research use measures identified in this review, while reliant on self-report measurement, have undergone extensive feasibility and pre-testing [24,25,27]. Further, if social desirability bias were an issue for the studies identified, we would expect to see an increase in the extent of nurses reported research use in recent years, given the current drive towards evidence-based nursing practice, of which research use is one component. Instead, extent scores have remained relatively consistent over time into the early 2000's, when the shift towards evidence-based practice emerged. We recommend that future studies be conducted that: examine the scores obtained from the self-report research use instruments identified in this review along side other forms of assessment (e.g., chart audit, think-aloud, observation), and attempt to causally associate research use scores with practice improvements and/or improved patient outcomes. Such studies will not only assess the validity of the research use scores obtained with these instruments and concerns with using self-report measurement, but also significantly advance the field. Until we have accurate and reliable measures of research use, it will not be possible to know, with any degree of certainty, whether intervention efforts are increasing nurses' use of research. Thus, in order to progress the field, robust measures of research use are critical.

Third, is conceptualization (theory) and resulting operationalization (the scales and scoring methods) of current research utilization measures. The published literature is characterized by multiple conceptualizations of research utilization. This influences how we define research utilization and, importantly, how we measure the construct and interpret the scores obtained from such measurement. Two conceptualizations dominating the field are: research utilization as a process (i.e., consists of a series of stages/steps) and research utilization as a variable or discrete event (also referred to as the 'variance' approach). Both types of measures were evident in this review. Assigning meaning to the scales used, however, regardless of whether the measure follows a process or variable conceptualization, in many cases remains unclear. For example, scores (called Total Innovation Adoption Scores) ranging from 0 to 4 are theoretically possible (and have been reported) in studies using the NPQ [26] (a process measure) to assess research utilization. A stage of adoption (as per Rogers' Innovation Decision Process Theory [106,107]) is then assigned to the resulting score: 0 to 0.49 (unaware), 0.5 to 1.49 (aware), 1.5 to 2.49 (persuasion), 2.5 to 3.49 (use sometimes), and 3.5 to 4.0 (use always). Using this schematic, a research utilization score of '1' is feasible. This score is interpreted as the respondent is aware of the research findings and is 1 (on a 0 to 4 scale) with respect to 'using research.' What is unclear is how this is interpreted as 'using research' when no 'use' is actually occurring? While no one would disagree that awareness is desirable and in many cases, necessary, for research utilization to occur, awareness is not 'research use' per se nor does it guarantee that research use will occur. In line with Rogers' Diffusion of Innovations theory (from which this scoring is stated to have been developed), an individual may be aware of the innovation (research findings) and still choose not to use it in practice if they are not persuaded of its effectiveness. This scoring method gives the impression research use is occurring when it is not, painting a more optimistic picture of research use than actually exists. Similar pictures are painted with variable measures of research utilization. For example, items in the RUQ [63] are scored on a 5-point Likert agreement scale (1 - strongly disagree, 2 - disagree, 3 - neither agree nor disagree, 4 - agree, 5 - strongly agree). But it is unclear how to interpret these scores as quantitative measures of research use. For instance, an overall scale score of '2' is interpreted as the nurse is just below average with respect to their 'use' of research. However, a score of '2,' according to the scale descriptors, would imply the same nurse 'disagreed' with most of the statements about their use of research. As with the NPQ, we contend that these scores also paint a more optimistic picture than actually exists. Similar scaling issues can be found in the remaining research utilization measures. As a result of these scoring problems, we believe that the extent to which nurses use research in their practice that is portrayed in the literature (and by association, in our synthesis) is higher than what actually exits.

Fourth, although nurses' research use has been measured by various instruments and has been studied for nearly 40 years, no benchmarking has been done. We thus have no 'gold standard' against which to compare the findings of any studies measuring nurses' use of research. A standard measure or set of measures of research use would help in such an effort. Equally, if not more important, is work that enables researchers and decision makers to evaluate the effect of different levels of research use on patient outcomes.

Progress in this field depends on having robust measures of research use. Fundamental to achieving this, we believe, are: an understanding of the validity and reliability of the measures that have been used to date; instrument development work that focuses on strengthening measurement accuracy; development of benchmarks for research use; and investigating the impact of varying levels of research use on patient outcomes. To date, there has been little emphasis on examination of the effects of varying levels of research use on patient and other outcomes (e.g., system, provider). Despite a strongly held assumption that integrating research evidence into practice will improve patient outcomes, none of the 55 articles included in this review examined associations between research use and patient or provider outcomes. Therefore, we could not assess the effect of research use on patient outcomes.

Limitations

A limitation of our work is that we deliberately excluded the terms 'evidence-based practice' and 'decision making' from our search. Our rationale for doing so was that evidence-based practice refers to the application of a range of sources of evidence, including patient preference, the clinician's expertise, and resources, in addition to research evidence [108]. Decision making also takes into account multiple information-seeking processes and resources. Research use, on the other hand, refers very specifically to the use of research in practice. We acknowledge that by excluding studies that measured evidence-based practice and/or decision making, we may have failed to capture the role of research use revealed in such studies. Further, our finding of moderate-high use may not hold if these additional studies were included. Future research should examine the extent to which nurses use other information sources, in addition to and in combination with research, to make clinical decisions.

Conclusion

From this review we are able to conclude that the extent to which nurses report research use in their daily practice is, on average, moderate-high, and has remained fairly consistent over time into the early 2000's, when increasing awareness of the evidence-based practice movement may have influenced the number of reports published on research use alone. Our finding suggests a more optimistic picture than we believe exists. Our combined nursing experience, which exceeds eight decades, causes us to question the accuracy of this finding. In an attempt to understand this finding, we turn to other plausible explanations. It is possible that the method we used to calculate the extent of research use was not sensitive enough to detect changes in research use over time and thus provided an overly positive result. Assuming, however, that our method was sufficiently sensitive, other plausible explanations for the finding include: lack of construct validity evidence of the research use scores obtained; scaling and other instrument concerns; self-report measurement biases (i.e., recall and social desirability); and low methodological quality of the studies included in the review. The results of this review, coupled with our hesitancy in accepting the finding that nurses' use of research is moderate-high, suggest a need for advances in the field, starting with a systematic and detailed assessment of the validity of current research use instruments used with nurses; more robust research designs including longitudinal research programs and programmatic research that examines nurses' research use and its link to patient and system outcomes; and the establishment of a benchmark against which to compare findings of studies measuring nurses' use of research.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

JES designed the review; developed the search strategy; undertook the article selection; participated in data extraction, quality assessment, and data synthesis; and made major contributions to drafting the manuscript. AMH, AMB, and HMO participated in data extraction and quality assessment, data synthesis, and contributed to drafting the manuscript. SJC participated in data synthesis and drafting the manuscript. CAE provided guidance throughout the study and critical commentary on manuscript drafts. All authors provided commentary on and approved the final manuscript.

Acknowledgements

JES is a postdoctoral fellow at the Ottawa Hospital Research Institute and is funded by Canadian Institutes for Health Research (CIHR) Postdoctoral and Bisby Fellowships; at the time of this research she was a PhD Candidate at the University of Alberta, funded by Killam, Alberta Heritage Foundation for Medical Research (AHFMR), and CIHR fellowships. At the time of this study, AMH and AMB were postdoctoral fellows at the University of Alberta, funded by AHFMR and CIHR fellowships. HMO is funded by AHFMR and KT Canada (CIHR) doctoral scholarships. SJC is funded by the Canadian Foundation for Dental Hygiene Research and Education. CAE holds a CIHR Canada Research Chair in knowledge translation.

References

  1. Shore HL: Adopters and laggards.

    The Canadian Nurse 1972, 68(7):36-39. PubMed Abstract OpenURL

  2. Closs S, Cheater F: Utilization of nursing research: culture, interest and support.

    J Adv Nurs 1994, 19(4):762-773. PubMed Abstract | Publisher Full Text OpenURL

  3. English I: Nursing as a research based profession: 22 years after Briggs.

    British Journal of Nursing 1994, 3(8):402-406. PubMed Abstract OpenURL

  4. Rizzuto C, Bostrom J, Suter WN, Chenitz WC: Predictors of nurses' involvement in research activities.

    Western Journal of Nursing Research 1994, 16(2):193-204. PubMed Abstract | Publisher Full Text OpenURL

  5. Michel Y, Sneed NV: Dissemination and use of research findings in nursing practice.

    Journal of Professional Nursing 1995, 11(5):306-311. PubMed Abstract | Publisher Full Text OpenURL

  6. Estabrooks CA: Will Evidence-Based Nursing Practice Make Practice Perfect?

    Canadian Journal of Nursing Research 1998, 30(1):15-36. PubMed Abstract OpenURL

  7. Dobbins M, Ciliska D, Mitchell A: Dissemination and use of research evidence for policy and practice by nurses: A model of development and implementation strategies. University of Toronto: Author; 1998.

  8. Parahoo K: Barriers to, and facilitators of, research utilization among nurses in Northern Ireland.

    Journal of Advanced Nursing 2000, 31(1):89-98. PubMed Abstract | Publisher Full Text OpenURL

  9. Veeramah V: Utilization of research findings by graduate nurses and midwives.

    Journal of Advanced Nursing 2004, 47(2):183-191. PubMed Abstract | Publisher Full Text OpenURL

  10. Glaser EM, Abelson HH, Garrison KN: Putting Knowledge to Use: Facilitating the Diffusion of Knowledge and the Implementation of Planned Change. San Francisco: Jossey-Bass; 1983. OpenURL

  11. Haines A, Kuruvilla S, Borchert M: Bridging the implementation gap between knowledge and action for health.

    Bulletin of the World Health Organization 2004, 82:724-732. PubMed Abstract | PubMed Central Full Text OpenURL

  12. Crane J: The future of research utilization.

    Nursing Clinics of North America 1995, 30(3):565-577. PubMed Abstract OpenURL

  13. Walsh M, Ford P: Nursing rituals: research, and rational action. Oxford: Butterworth Heinemann; 1989. OpenURL

  14. Hunt J: Indicators for nursing practice: the use of research findings.

    Journal of Advanced Nursing 1981, 6:189-194. PubMed Abstract | Publisher Full Text OpenURL

  15. Gould D: Pressure sore prevention and treatment: an example of nurses' failure to implement research findings.

    Journal of Advanced Nursing 1986, 11(4):389-394. PubMed Abstract | Publisher Full Text OpenURL

  16. Bircumshaw D: The utilization of research findings in clinical nursing practice.

    Journal of Advanced Nursing 1990, 15:1272-1280. PubMed Abstract | Publisher Full Text OpenURL

  17. Bostrom J, Wise L: Closing the gap between research and practice.

    Journal of Nursing Administration 1994, 24:22-27. PubMed Abstract | Publisher Full Text OpenURL

  18. Landrum BJ: Marketing innovations to nurses, Part 2: Marketing's role in the adoption of innovations.

    Journal WOCN 1998, 25:227-232. Publisher Full Text OpenURL

  19. Heater BS, Becker AM, Olson RK: Nursing interventions and patient outcomes: A meta-analysis of studies.

    Nursing Research 1988, 37(5):303-307. PubMed Abstract | Publisher Full Text OpenURL

  20. Balogh Z, Caldwell E, Heetveld M, D'Amours S, Schlaphoff G, Harris I, et al.: Institutional practice guidelines on management of pelvic fracture-related hemodynamic instability: do they make a difference?

    Journal of Trauma 2005, 58(4):778-782. PubMed Abstract | Publisher Full Text OpenURL

  21. Barr J, Hecht M, Flavin K, Khorana A, Gould M: Outcomes in critically ill patients before and after the implementation of an evidence-based nutritional management protocol.

    Chest 2004, 125(4):1446-1457. PubMed Abstract | Publisher Full Text OpenURL

  22. Beaupre L, Cinats J, Senthilselvan A, Lier D, Jones C, Scharfenberger A, et al.: Reduced morbidity for elderly patients with a hip fracture after implementation of a perioperative evidence-based clinical pathway.

    Quality and Safety in Health Care 2006, 15(5):375-379. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  23. Estabrooks CA, Wallin L, Milner M: Measuring knowledge utilization in health care.

    International Journal of Policy Analysis & Evaluation 2003, 1:3-36. OpenURL

  24. Parahoo K: Research utilization and research related activities of nurses in Northern Ireland.

    International Journal of Nursing Studies 1998, 35(5):283-291. PubMed Abstract | Publisher Full Text OpenURL

  25. Estabrooks CA: Modeling the individual determinants of research utilization.

    Western Journal of Nursing Research 1999, 21(6):758-772. PubMed Abstract | Publisher Full Text OpenURL

  26. Brett JL: Use of nursing practice research findings.

    Nursing Research 1987, 36(6):344-349. PubMed Abstract OpenURL

  27. Rodgers SE: The extent of nursing research utilization in general medical and surgical wards.

    Journal of Advanced Nursing 2000, 32(1):182-193. PubMed Abstract | Publisher Full Text OpenURL

  28. Stetler C: Research utilization: Defining the concept.

    Image:The Journal of Nursing Scholarship 1985, 17:40-44. PubMed Abstract | Publisher Full Text OpenURL

  29. Estabrooks CA: The conceptual structure of research utilization.

    Research in Nursing and Health 1999, 22(3):203-216. PubMed Abstract | Publisher Full Text OpenURL

  30. Belkhodja O, Amara N, Landry Rj, Ouimet M: The Extent and Organizational Determinants of Research Utilization in Canadian Health Services Organizations.

    Science Communication 2007, 28:377-417. Publisher Full Text OpenURL

  31. Stetler CB: Problems and issues of research utilization. In Nursing issues in the 1990's. Edited by Strickland OL, Fishman DL. New York: Delmar; 1994:459-470. OpenURL

  32. Stetler CB: Refinement of the Stetler/Marram model for application of research findings to practice.

    Nursing Outlook 1994, 42:15-25. PubMed Abstract | Publisher Full Text OpenURL

  33. Berggren A: Swedish midwives' awareness of, attitudes to and use of selected research findings.

    Journal of Advanced Nursing 1996, 23(3):462-470. PubMed Abstract | Publisher Full Text OpenURL

  34. Beyer J, Trice H: The utilization process: A conceptual framework and synthesis of empirical findings.

    Administrative Science Quarterly 1982, 27:591-622. Publisher Full Text OpenURL

  35. Estabrooks CA: Mapping the research utilization field in nursing.

    Canadian Journal of Nursing Research 1999, 31(1):53-72. PubMed Abstract OpenURL

  36. Grol R: Successes and failures in the implementation of evidence-based guidelines for clinical practice.

    Medical Care 2001., 39(8 SUPPL. 2) OpenURL

  37. Ploeg J, Davies B, Edwards N, Gifford W, Elliott Miller P: Factors influencing best-practice guideline implementation: Lessons learned from administrators, nursing staff and project leaders.

    Worldviews on Evidence-Based Nursing 2007, 4(4):210-219. PubMed Abstract | Publisher Full Text OpenURL

  38. Estabrooks CA, Floyd JA, Scott-Findlay S, O'Leary KA, Gushta M: Individual determinants of research utilization: A systematic review.

    Journal of Advanced Nursing 2003, 43(5):506-520. PubMed Abstract | Publisher Full Text OpenURL

  39. Meijers JMM, Janssen MAP, Cummings GG, Wallin L, Estabrooks CA, YG Halfens R: Assessing the relationships between contextual factors and research utilization in nursing: Systematic literature review.

    Journal of Advanced Nursing 2006, 55(5):622-635. PubMed Abstract | Publisher Full Text OpenURL

  40. Thompson DS, Estabrooks CA, Scott-Findlay S, Moore K, Wallin L: Interventions aimed at increasing research use in nursing: A systematic review.

    Implementation Science 2007, 2:15. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  41. Frasure J: Analysis of instruments measuring nurses' attitudes towards research utilization: A systematic review.

    Journal of Advanced Nursing 2008, 61(1):5-18. PubMed Abstract | Publisher Full Text OpenURL

  42. Shadish W, Cook T, Campbell D: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. 2nd edition. Boston: Houghton Mifflin; 2002. OpenURL

  43. Kmet L, Lee R, Cook L: Standard quality assessment criteria for evaluating primary research papers from a variety of fields. Edmonton: Alberta Heritage Foundation for Medical Research; 2004:1-22.

  44. T Khan K, Ter Riet G, Popay J, Nixon J, Kleijnen J, Satge I: Conducting the review: Phase 5 study quality assessment.

    In Undertaking systematic reviews of research effectiveness CDC's guidance for those carrying out or commissioning reviews Edited by Centre of Reviews and Dissemination UoY. 2001, 1-20. OpenURL

  45. Estabrooks C, Cummings G, Olivo S, Squires J, Gibin C, Simpson N: Effects of shift length on quality of patient care and health provider outcomes: systematic review.

    Quality and Safety in Healthcare 2009, 18:181-188. Publisher Full Text OpenURL

  46. De Vet HCW, De Bie RA, Van Der Heijden GJMG, Verhagen AP, Sijpkes P, Knipschild PG: Systematic reviews on the basis of methodological criteria.

    Physiotherapy 1997, 83(6):284-289. Publisher Full Text OpenURL

  47. Deeks JJ, Dinnes J, D'Amico R, Sowden AJ, Sakarovitch C, Song F, Petticrew M, Altman DG: Evaluating non-randomised intervention studies.

    Health Technology Assessment (Winchester, England) 2003., 7(27) PubMed Abstract | Publisher Full Text OpenURL

  48. Jackson N, Waters E: Criteria for the systematic review of health promotion and public health interventions.

    Health Promotion International 2005, 20(4):367-374. PubMed Abstract | Publisher Full Text OpenURL

  49. Carlson CL: Prior conditions influencing nurses' decisions to adopt evidence-based postoperative pain assessment practices. In Ph.D. Indiana University; 2006. OpenURL

  50. Brett JL: Organizational integrative mechanisms and adoption of innovations by nurses.

    Nursing Research 1989, 38(2):105-110. PubMed Abstract OpenURL

  51. Coyle LA, Sokop AG: Innovation adoption behavior among nurses.

    Nursing Research 1990, 39(3):176-180. PubMed Abstract | Publisher Full Text OpenURL

  52. Barta KM: Information-seeking, research utilization, and barriers to research utilization of pediatric nurse educators.

    Journal of professional nursing: official journal of the American Association of Colleges of Nursing 1995, 11(1):49-57. PubMed Abstract | Publisher Full Text OpenURL

  53. Rutledge DN, Greene P, Mooney K, Nail LM, Ropka M: Use of research-based practices by oncology staff nurses.

    Oncology Nursing Forum 1996, 23(8):1235-1244. PubMed Abstract OpenURL

  54. Thompson CJ: Extent and factors influencing research utilization among critical care nurses. In Dissertation/Thesis. Texas Woman's University, College of Nursing; 1997. OpenURL

  55. Rodgers SE: A study of the utilization of research in practice and the influence of education.

    Nurse Education Today 2000, 20(4):279-287. PubMed Abstract | Publisher Full Text OpenURL

  56. Squires JE, Moralejo D, LeFort SM: Exploring the role of organizational policies and procedures in promoting research utilization in registered nurses.

    Implementation Science 2007., 2(1) PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  57. Nash MA: Research utilization among Idaho nurses. In Thesis/dissertation (deg); Manuscript (mss). Gonzaga University; 2005. OpenURL

  58. Tsai S: Nurses' participation and utilization of research in the Republic of China.

    International Journal of Nursing Studies 2000, 37(5):435-444. PubMed Abstract | Publisher Full Text OpenURL

  59. Butler L: Valuing research in clinical practice: a basis for developing a strategic plan for nursing research.

    The Canadian journal of nursing research 1995, 27(4):33-49. PubMed Abstract OpenURL

  60. Bostrom J, Suter WN: Research utilization: making the link to practice.

    Journal of Nursing Staff Development 1993, 9(1):28-34. PubMed Abstract OpenURL

  61. Brown DS: Nursing education and nursing research utilization: is there a connection in clinical settings?

    Journal of Continuing Education in Nursing 1997, 28(6):258-262.

    quiz 284

    PubMed Abstract OpenURL

  62. Carlson CL: Prior conditions influencing nurses' decisions to adopt evidence-based postoperative pain assessment practices. In Dissertation/Thesis. Indiana University; 2006. OpenURL

  63. Champion VL, Leach A: Variables related to research utilization in nursing: an empirical investigation.

    Journal of Advanced Nursing 1989, 14(9):705-710. PubMed Abstract | Publisher Full Text OpenURL

  64. Connor N: The relationship between organizational culture and research utilization practices among nursing home departmental staff. In Dissertation/Thesis. Dalhousie University (Canada); 2007. OpenURL

  65. Davies S: Practice nurses' use of evidence--based research.

    Nursing Times 1999, 95(4):57-60. PubMed Abstract OpenURL

  66. Estabrooks CA, Kenny DJ, Adewale AJ, Cummings GG, Mallidou AA: A comparison of research utilization among nurses working in Canadian civilian and United States Army healthcare settings.

    Research in Nursing and Health 2007, 30(3):282-296. PubMed Abstract | Publisher Full Text OpenURL

  67. Hansen HE, Biros MH, Delaney NM, Schug VL: Research utilization and interdisciplinary collaboration in emergency care.

    Academic Emergency Medicine 1999, 6(4):271-279. PubMed Abstract | Publisher Full Text OpenURL

  68. Hatcher S, Tranmer J: A survey of variables related to research utilization in nursing practice in the acute care setting.

    Canadian Journal of Nursing Administration 1997, 10(3):31-53. PubMed Abstract OpenURL

  69. Humphris D, Hamilton S, O'Halloran P, Fisher S, Littlejohns P: Do diabetes nurse specialists utilise research evidence?

    Practical Diabetes International 1999, 16(2):47-50. Publisher Full Text OpenURL

  70. Kenny DJ: Nurses' use of research in practice at three US Army hospitals.

    Canadian journal of nursing leadership 2005, 18(3):45-67. PubMed Abstract | Publisher Full Text OpenURL

  71. Lacey EA: Research utilization in nursing practice--a pilot study.

    Journal of Advanced Nursing 1994, 19(5):987-995. PubMed Abstract | Publisher Full Text OpenURL

  72. Logsdon C, Davis DW, Hawkins B, Parker B, Peden A: Factors related to research utilization by registered nurses in Kentucky.

    Kentucky Nurse 1998, 46(1):23-26. PubMed Abstract OpenURL

  73. McCleary L, Brown GT: Use of the Edmonton research orientation scale with nurses.

    Journal of Nursing Measurement 2002, 10(3):263-275. PubMed Abstract | Publisher Full Text OpenURL

  74. McCleary L, Brown GT: Association between nurses' education about research and their reseach use.

    Nurse Education Today 2003, 23(8):556-565. PubMed Abstract | Publisher Full Text OpenURL

  75. McCloskey DJ: The relationship between organizational factors and nurse factors affecting the conduct and utilization of nursing research. In Dissertation/Thesis. George Mason University; 2005. OpenURL

  76. Milner FM, Estabrooks CA, Humphrey C: Clinical nurse educators as agents for change: increasing research utilization.

    International Journal of Nursing Studies 2005, 42(8):899-914. PubMed Abstract | Publisher Full Text OpenURL

  77. Niederhauser VP, Kohr L: Research endeavors among pediatric nurse practitioners (REAP) study.

    Journal of Pediatric Health Care 2005, 19(2):80-89. PubMed Abstract | Publisher Full Text OpenURL

  78. Parahoo K: A comparison of pre-Project 2000 and Project 2000 nurses' perceptions of their research training, research needs and of their use of research in clinical areas.

    Journal of Advanced Nursing 1999, 29(1):237-245. PubMed Abstract | Publisher Full Text OpenURL

  79. Parahoo K: Research utilization and attitudes towards research among psychiatric nurses in Northern Ireland.

    Journal of Psychiatric and Mental Health Nursing 1999, 6(2):125-135. PubMed Abstract | Publisher Full Text OpenURL

  80. Parahoo K, Barr O, McCaughan E: Research utilization and attitudes towards research among learning disability nurses in Northern Ireland.

    Journal of Advanced Nursing 2000, 31(3):607-613. PubMed Abstract | Publisher Full Text OpenURL

  81. Parahoo K, McCaughan EM: Research utilization among medical and surgical nurses: A comparison of their self reports and perceptions of barriers and facilitators.

    Journal of Nursing Management 2001, 9(1):21-30. PubMed Abstract | Publisher Full Text OpenURL

  82. Pettengill MM, Gillies DA, Clark CC: Factors encouraging and discouraging the use of nursing research findings.

    Image--the journal of nursing scholarship 1994, 26(2):143-147. PubMed Abstract | Publisher Full Text OpenURL

  83. Prin PL, Mills ME: Nurses' MEDLINE usage and research utilization.

    Studies in health technology and informatics 1997, 46:451-456. PubMed Abstract OpenURL

  84. Profetto-McGrath J, Hesketh KL, Lang S, Estabrooks CA: A study of critical thinking and research utilization among nurses.

    West J Nurs Res 2003, 25(3):322-337. PubMed Abstract | Publisher Full Text OpenURL

  85. Stiefel KA: Career commitment, nursing unit culture, and nursing research utilization. In Dissertation/Thesis. University of South Carolina; 1996. OpenURL

  86. Valizadeh L, Zamanzadeh V: Research utilization and research attitudes among nurses working in teaching hospitals in Tabriz, Iran.

    Journal of Clinical Nursing 2003, 12(6):928-930. PubMed Abstract | Publisher Full Text OpenURL

  87. Varcoe C, Hilton A: Factors affecting acute-care nurses' use of research findings.

    Canadian Journal of Nursing Research 1995, 27(4):51-71. PubMed Abstract OpenURL

  88. Veeramah V: A study to identify the attitudes and needs of qualified staff concerning the use of research findings in clinical practice within mental health care settings.

    Journal of Advanced Nursing 1995, 22(5):855-861. PubMed Abstract | Publisher Full Text OpenURL

  89. Linde BJ: The effectiveness of three interventions to increase research utilization among practicing nurses. In Dissertation/Thesis. The University of Michigan; 1989. OpenURL

  90. Pelz DC, A HJ, Ciarlo JA: Measuring utilization of nursing research. In Utilizing evaluation: Concepts and measurement techniques. Edited by Anonymous. Beverly Hills, CA: Sage; 1981:125-149. OpenURL

  91. Tranmer JE, LochhausGerlach J, Lam M: The effect of staff nurse participation in a clinical nursing research project on attitude towards, access to, support of and use of research in the acute care setting.

    Canadian Journal of Nursing Leadership 2002, 15(1):18-26. PubMed Abstract OpenURL

  92. Tsai SL: The effects of a research utilization in-service program on nurses.

    International Journal of Nursing Studies 2003, 40(2):105-113. PubMed Abstract | Publisher Full Text OpenURL

  93. Ketefian S: Application of selected nursing research findings into nursing practice. A pilot study.

    Nursing Research 1975, 24(2):89-92. PubMed Abstract OpenURL

  94. Tsai SL: Nurses' participation and utilization of research in the Republic of China.

    International Journal of Nursing Studies 2000, 37(5):435-444. PubMed Abstract | Publisher Full Text OpenURL

  95. Wallin L, Bostrom A, Wikblad K, Ewald U: Sustainability in changing clinical practice promotes evidence-based nursing care.

    Journal of Advanced Nursing 2003, 41(5):509-518. PubMed Abstract | Publisher Full Text OpenURL

  96. Walczak JR, McGuire DB, Haisfield ME, Beezley A: A survey of research-related activities and perceived barriers to research utilization among professional oncology nurses.

    Oncology Nursing Forum 1994, 21(4):710-715. PubMed Abstract OpenURL

  97. Wright A, Brown P, Sloman R: Nurses' perceptions of the value of nursing research for practice.

    Australian Journal of Advanced Nursing 1996, 13(4):15-18. PubMed Abstract OpenURL

  98. Youngstrom LH: Nursing staff development educators and research utilization. In Dissertation/Thesis. Widener University School of Nursing; 1996. OpenURL

  99. Davies S: Practice nurses' use of evidence-based research.

    Nursing Times 1999, 95(4):57-60. PubMed Abstract OpenURL

  100. Squires J, Adachi A, Estabrooks C: Developing a Valid and Reliable Measure of Research Utilization Technical Report. Edmonton, AB: Faculty of Nursing, University of Alberta; 2008.

  101. Hambleton R, Swaminathan H, Rogers J: Fundamentals of Item Response Theory. Newbury Park, CA: Sage; 1991. OpenURL

  102. Ellis BB, Mead AD: Item analysis: Theory and practice using classical and modern test theory. In Handbook of Research Methods in Industrial and Organizational Psychology. Blackwell Publications; 2002:324-343. OpenURL

  103. Waltz CF, Strickland O, Lenz ER: Measurement in Nursing and Health Research. 3rd edition. New York: Springer Pub; 2005. OpenURL

  104. Streiner D, Norman G: Health Measurement Scales: A practical Guide to their Development and Use. 4th edition. Oxford: Oxford University Press; 2008. OpenURL

  105. DeVellis R: Scale development: Theory and applications. 2nd edition. London: SAGE Publications; 2003. OpenURL

  106. Rogers EM: Diffusion of Innovations. 3rd edition. New York: The Free Press; 1983. OpenURL

  107. Rogers E: Diffusion of Innovations. 4th edition. New York: The Free Press; 1995. OpenURL

  108. DiCenso A, Cullum N: Implementing evidence-based nursing: Some misconceptions.

    Evidence-Based Nursing 1998, 1(2):38-40. Publisher Full Text OpenURL