Table 3 |
|||||||||||
|
Summary of the measures used by included studies, methods of analysis and results of comparisons |
|||||||||||
|
Study |
Proxy measure |
Direct Measure (DM) |
Analysis |
||||||||
|
|
|||||||||||
|
Description 1. Method V = Clinical vignette (No. of case simulations) CI/Q = Clinician interview/questionnaire MR = Medical Record review PI/Q = Patient interview/questionnaire 2. Timing |
Clinician self report (SR) |
Medical Record Review (MR) |
Patient report (PR) |
Description 1. Method SP = Simulated Patients DO = Direct Observation VR = Video recording AR = Audio recording 2. Timing |
SP Training reported |
Psychometrics (IRR) |
Compared Item by Item |
Compared Summary Scores |
Agreement between measures: Co-efficient r; kappa (k); Structural equation modelling (SEM); Sensitivity (Sens) & Specificity (Spec) Difference between mean scores: ANOVA; T-test |
P |
|
|
|
|||||||||||
|
Stange [5] 1998 |
1. MR; PQ 2. At end of consultation |
√ |
√ |
DO |
0.39 to 1.00 (kappa) |
√ |
MR Sens = 8% (diet advice) – 92% (Lab tests) Spec = 83% (social history) – 100% (counselling services, physical exam, lab tests) k = 0.12 to 0.92 (79 comparisons) PR Sens = 17% (mammogram) – 89% (Pap test) Spec = 85% (in-office referral) – 99% (immunisation, physical exam, lab tests) k = 0.03 to 0.86 (53 comparisons) |
NR |
|||
|
Flocke [6] 2004 |
1. PQ 2. At end of consultation (24%) or postal return (76%) |
√ |
DO |
NR |
√ |
Sens* = 11% (substance use) – 76% (smoking cessation) |
NA |
||||
|
Wilson [7] 1994 |
1. MR; PQ 2. At end of consultation |
√ |
√ |
AR |
0.79 to 1.00 |
√ |
MR Sens = 31%, Spec* = 99% 28.6 (Alcohol) Sens = 29%, Spec* = 100% 83.3 (BP) Sens = 83%, Spec* = 93% % agreement between DM & MR: 45.5 (Smoking) PR Sens = 74%, Spec* = 94% 75.0 (Alcohol) Sens = 75%, Spec* = 94% 100 (BP) Sens = 100%, Spec* = 90% % agreement between DM & PR: 81.8 (Smoking) |
NA |
|||
|
Ward [8] 1996 |
1. PQ 2. Questionnaire mailed to patient within 2 days of consultation |
√ |
AR |
0.74 to 0.94 (kappa) |
√ |
Sens = 93% (smoking status) Spec = 79% Sens = 92% (cessation advice) Spec = 82% |
NA |
||||
|
Zuckerman [9] 1975 |
1. MR 2. At end of consultation |
√ |
AR |
NR |
√ |
Sens* = 0% (side effects) – 100% (Diagnosis) Spec* = 9% (Diagnosis) – 100% (side effects) |
NA |
||||
|
Luck [10] 2000 |
1. MR 2. At end of consultation |
√ |
SP (27) each role-playing 1 of 8 case simulations |
√ |
NR |
√ |
√ |
ANOVA (4-way) Necessary care: Sens = 70%, Spec = 81% Unnecessary care: Sens = 65%' Spec = 64%. |
<0.0001 NA |
||
|
Page [11] 1980 |
1. V (4) 2. Upto 6 weeks before or 3 weeks after SP visit |
√ |
SP (4) each role-playing 1 case simulation |
√ |
0.76 |
√ |
√ |
r = .56 & .68 r = .26 & .37 "Must do" actions Sens* = 97%, Spec* = 33% "Must not do" actions Sens* = 30%, Spec* = 98% |
>0.05 <0.05 |
||
|
Gerbert [12] 1988 |
1. CI; MR; PI 2. At end of consultation |
√ |
√ |
√ |
√R |
NR |
√ |
k = 0.67 (SR) k = 0.54 (MR) k = 0.50 (PR) |
<0.001 <0.001 <0.001 |
||
|
Pbert [13] 1999 |
1. CI; PI 2. At end of consultation |
√ |
√ |
AR. |
NR |
√ |
√ |
r = 0.77 (SR) r = 0.67 (PR) |
<0.0001 <0.0001 |
||
|
Gerbert [14] 1986 |
1. CI; MR; PI 2. At end of consultation |
√ |
√ |
√ |
√R |
0.52 to 0.93 (kappa) |
√ |
Median % agreement (All categories): 0.84 (SR) 0.88 (MR) 0.86 (PR) |
NA |
||
|
Dresselhaus [15] 2000 |
1.V (8); MR 2. NR |
√ |
√ |
SP (4) each role-playing a simple and complex case presentation |
√ |
NA |
√ |
ANOVA (3-way) |
<0.01 |
||
|
Rethans [16] 1987 |
1. V (1). 2. Completed 2 months after SP visit |
√ |
SP (3) each role-playing same case simulation |
√ |
0.78 to 1.0 (kappa) |
√ |
√ |
T-test: Overall "Obligatory" "Intermediate" "Superfluous" |
ns <0.005 <0.05 <0.05 |
||
|
Rethans [17] 1994 |
1. MR 2. Charts reviewed two years after SP visit. |
√ |
SP (4) each role-playing 1 of 4 case simulations |
√ |
0.93 (kappa) |
√ |
√ |
r = 0.54 (Overall) r = 0.17 (History taking) r = 0.45 (Physical exam) r = 0.75 (Lab exam) r = 0.50 (Advice) r = 0.43 (Medication) r = -0.04 (Follow-up) |
<0.05) ns ns <0.01 <0.05 ns ns |
||
|
Peabody [18] 2000 |
1. V (8); MR 2. Completed "several weeks" after SP visit |
√ |
√ |
SP (4) each role-playing a simple and complex case presentation |
√ |
NA |
√ |
ANOVA (4-way) |
<0.001 |
||
|
O'Boyle [19] 2001 |
1. % time practiced hand hygiene 2. Up to one month prior to observation period |
√ |
DO Nurses observed for 2 hours or until 10 indications for handwashing had occurred |
0.94 to 0.98 |
√ |
r = 0.21 SEM = 0.201 |
<0.05 <0.05 |
||||
|
|
|||||||||||
|
* Calculated by authors NA = Not applicable NR = Not reported ns = non-significant |
|||||||||||
|
Hrisos et al. Implementation Science 2009 4:37 doi:10.1186/1748-5908-4-37 |
|||||||||||
