  |
|
|
||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
| Journal of the New Zealand Medical Association,
27-September-2002, Vol 115 No 1162
|
||||||||||||||||||||||||||||||||||||||
|
The Ottawa ankle rules for the use of diagnostic X-ray in
after hours medical centres in New Zealand
Simon Wynn-Thomas, Tom Love, Deborah McLeod, Sue Vernall,
Marjan Kljakovic, Antony Dowell and John Durham
The aims of this study were to measure baseline use of the
Ottawa ankle rules (OAR) for suspected fracture of the ankle, validate the OAR
in a New Zealand primary health care setting and, if appropriate, explore the
impact of implementing the rules in general practice after hours
settings.
The OAR were developed upon the basis of a series of studies
of ankle injuries,1–3 which were analysed
to find clinical indicators for an X-ray of the ankle or foot. The full
formulation of the OAR is shown in Figure
1.4
The Accident Compensation Corporation (ACC) has recommended
that the OAR be used in New Zealand.5 There has
been one attempt to validate the OAR in the New Zealand
setting,6 which showed that up to 7% of
fractured ankles might have been missed if the OAR had been implemented. This
study has been criticised upon the methodological grounds that it used an
earlier version of the OAR and only tested one part of the
OAR.7 The authors of the New Zealand study
contended that the failed validation reflected the inexperience of junior staff
in the assessment of limb injuries and in applying the
OAR.8
A pilot study, conducted before this study began, suggested
that use of the OAR had the potential to reduce the number of unnecessary X-rays
in a New Zealand setting.
Figure 1. The Ottawa ankle rules
Data concerning the utilisation of X-rays in the management
of 159 patients with ankle injuries presenting to Wellington’s After Hours
Medical Centre (AMC) over a three month period were collected with a
retrospective notes audit. Analysis of these data showed that ankle injuries are
common (approximately 600 cases per annum), frequently X-rayed (73% of ankle
injuries), and the majority of these X-rays (75%) are normal. Similar data have
been published in other centres throughout the
world.9–11 Implementation of the OAR
overseas has resulted in a reduction of up to 34% in the number of X-rays
performed, without a corresponding increase in the number of undiagnosed
fractures.3, 12–16 Even where the Rules
have not been fully validated, they were found to be more sensitive than
clinical suspicion alone.17
MethodsSurvey
To find the baseline use of the OAR, postal questionnaires were sent to
general practitioners (GPs) on the rosters for the Wellington and Christchurch
AMCs. The questionnaire asked about the use of six guidelines to avoid specific
prompting about the OAR. GPs were asked to indicate on five point scales, from
never to always, how often they used guidelines for heavy menstrual bleeding,
lipid disorders, back pain, ankle injuries, depression and youth suicide. A
reply-paid envelope was included. A second questionnaire and reminder letter was
sent to non respondents.
Validation of Ottawa ankle rules The research was conducted at the Wellington AMC and the Christchurch 24 Hour Surgery. Prospective data were collected on eligible patients over a six-month period. In each case, the general practitioner seeing the patient was asked to determine whether the patient was eligible to participate in the study and to seek consent from the patient. The GPs managed patients in their usual manner and recorded their findings on a data collection sheet that included (but did not emphasise) the observations necessary to apply the OAR. X-ray results were later recorded by the research team. Patients who did not receive X-rays were followed up with a phone call to determine their outcome seven to ten days after the initial consultation. The outcome measures were X-ray utilisation and diagnosis of fracture. The sample size was based on the number of patients required to detect a 20% difference in X-ray rates before and after the implementation of the OAR. A sample size of 200 patients from each centre would enable us to detect a 20% difference with a power of 80% at the 95% confidence level. Inclusion and exclusion criteria Patients were included in the study if they were aged 18 years or over, had pain and/or tenderness secondary to blunt trauma due to any mechanism of injury, including twisting, falling or direct blow, affecting any of the following anatomical sites: the distal 6 cm of the tibia and/or fibula; the talus; the navicular, cuboid and cuneiform bones; the anterior process of the calcaneus and/or overlying soft tissues. Patients were excluded if they: were pregnant; had isolated injuries of the skin; had an injury sustained more than 10 days prior to the consultation; had an altered mental state (such as alcohol intoxication); a penetrating injury; multiple trauma; or an underlying physical condition preventing application of the OAR. Clinically insignificant fractures were defined as bone fragments less than 3 mm in breadth, following the initial study that validated the OAR.3 Data analysis Data were entered into a Microsoft Access database. Ottawa positive and negative status was determined retrospectively from the data for each patient by two GPs working independently and blinded to the X-ray results. Where there was disagreement, a third general practitioner reviewed the cases. Ineligibility due to insufficient information was determined in the same manner. Statistical tests were carried out in EpiInfo. Exact confidence limits were obtained from Documenta Geigy Scientific Tables for sample sizes of less than 100 and approximated for larger samples.18 The study was approved by the Wellington Ethics Committee. ResultsSurvey
410 (87.1%) GPs completed the survey; 291 (94.5%) from Christchurch and
119 (73.0%) from Wellington. Eighty nine per cent of GPs reported that they
never or hardly ever used ankle guidelines (Table 1), the lowest self-reported
use for any of the guidelines.
Table 1. General practitioners’ self-reported use
of guidelines
Recruitment Data
were collected from May to December 2001 for 109 consenting patients from
Christchurch and 107 from Wellington. Comparison of recruited patients with
completed ACC forms indicated a coverage of slightly more than 50% had been
achieved. Seven patients from Christchurch and nine from Wellington were
subsequently excluded from the analysis because the data collection forms had
not been completed sufficiently to allow application of the OAR (13), they were
ineligible due to an underlying physical condition (1), or they had fractures of
the lower leg (2).
Recruited patients included 94 males and 106 females. There
was no difference between Wellington and Christchurch with regard to
age
(χ2
= 1.97, p =
0.742) and gender (χ2 = 0.34, p =
0.559) of recruited eligible patients.
Data comparing recruited patients to non-recruited patients
were available from Wellington, where ACC forms were audited and compared with
recruitment forms.
Study data collection
forms were completed for 99 of 226 (43.8%) eligible patients for whom ACC forms
were completed. The age
(χ2
= 1.48, p = 0.478) and gender (χ2 =
1.07, p = 0.302) profile of patients not recruited for the study was not
significantly different from recruited patients.
Table 2 shows the number of patients for whom the OAR were
positive, whether they received X-ray, and whether they had a
fracture.
Table 2. Comparison of the use of the OAR and usual
practice in identifying ankle fractures
* Excludes clinically insignificant fractures;
† Ordering X-rays was used as a measure
of usual practice
X-rays A total of
133 (67%) patients received X-rays, 74 (73%) in Christchurch and 59 (60%) in
Wellington.
Fractures Seventeen
fractures were diagnosed; nine (9%) in Christchurch and eight (8%) in
Wellington. Three fractures from Christchurch and one from Wellington were
clinically insignificant; three were flake avulsions and one was a bone
fragment.
Sensitivity and specificity
of the Ottawa ankle rules There were 11 cases of disagreement between the
reviewing practitioners when surveying the data sheets for Ottawa status that
were resolved by a third practitioner. In total, 113 patients were assessed as
Ottawa positive, and 16 of these patients had fractures. The four patients with
clinically insignificant fractures were excluded when the sensitivity and
specificity of the OAR was assessed. Of these four, three were Ottawa positive
and one was Ottawa negative.
The overall sensitivity of the Ottawa Rules was 100% (95%
CI: 75.3 – 100) and the specificity 47% (95% CI: 40.5 – 54.5). In
Christchurch the sensitivity was 100% (95% CI: 75.3 – 100) and the
specificity 39% (95% CI: 28.8 – 49.4); and in Wellington the sensitivity
was 100% (95% CI: 75.3 – 100) and the specificity 56% (95% CI: 44.7
– 66.0). The positive predictive value of the OAR was 12% overall (95% CI:
6.5 – 19.4); 10% (95% CI: 3.6 – 19.6) in Christchurch, and 15% (95%
CI: 6.2 – 28.3) in Wellington.
Sensitivity and specificity
of usual practice The sensitivity and specificity of usual practice was
determined by assessing whether an X-ray was ordered or not by the GP at the
time of the consultation. In total, 129 X-rays were ordered, of which 13
revealed a fracture. The overall sensitivity of usual practice was 100% (95% CI:
75.3 – 100) and the specificity 37% (95% CI: 30.2 – 44.2). In
Christchurch the sensitivity was 100% (95% CI: 75.3 – 100) and the
specificity 30% (95% CI: 21.03 – 40.5); and in Wellington the sensitivity
was 100% (95% CI: 75.3 – 100) and the specificity 43% (95% CI: 32.9
– 54.2). The positive predictive value of usual practice was 10% (95% CI:
5.3 – 16.1) overall; 9% (95% CI: 3.2 – 17.5) in Christchurch; and
12% (95% CI 5.0 – 23.3) in Wellington.
What difference would
implementing the Ottawa ankle rules make? General practitioners ordered
X-rays for 101 of the 110 (92%) Ottawa positive cases and for all four patients
with clinically insignificant fractures. X-rays were ordered for 28 of 86 (33%)
Ottawa negative patients. In the setting we studied, use of the Ottawa rules
would reduce the total number of X-rays ordered from 129 to 110. Implementing
the OAR would therefore reduce X-ray utilisation by 16% (95% CI: approx 10.8
– 21.3). However, strict application of the OAR would mean that one of the
four patients with clinically insignificant fractures would not have been
X-rayed.
DiscussionStrict application of the OAR to
the patients in this study would not have resulted in GPs missing any clinically
significant ankle fractures. The sensitivity and specificity of usual practice
was comparable to the sensitivity and specificity of the OAR. Strict application
of the OAR would have saved only 19 X-rays in the sample studied. It was
therefore decided that the benefits of further implementation of the OAR did not
warrant a specific implementation programme.
The study has validated the OAR in a New Zealand primary
care setting. The difference between this result and that found in the earlier
attempt to validate the OAR in New Zealand could be explained by the exclusion
criteria that were used in this study, but were absent from the earlier
research. If we had not applied exclusion criteria, the OAR would have appeared
to miss three fractures. Upon close examination, however, these were clinically
insignificant flake avulsions to which the OAR could not be applied. The
original validation of the refined OAR excluded fractures with bone fragments
less than 3 mm in breadth on the empirical grounds that such fractures were not
treated with plaster immobilisation.3 However,
in a setting in which the clinical norm is to treat avulsion fractures, the
appropriateness of the OAR for these events is open to debate. This emphasises
the point that while the OAR are valid in a New Zealand setting, they must be
applied appropriately.
Use of the OAR would have reduced the number of X-rays
ordered by 16% in the sample that we studied. Even though awareness of the OAR
among GPs was low, the outcome of usual practice of ordering X-rays is similar
to the outcome that would have been obtained using the OAR. This suggests that
clinically experienced practitioners may be able to apply the important elements
of guidelines without acknowledging that they are using a guideline in any
formal sense.
The potential for 16% reduction in X-rays was less than we
expected on the basis of previous
research,3,12–16 and it is therefore
questionable whether good results could be achieved with an education campaign.
Resources for an education campaign are probably justified only where there is
prima facie cause to suspect that an unduly high proportion of ankles are being
X-rayed.
We have validated the OAR in a New Zealand primary care
setting, however we have identified some limitations to them as a tool in
clinical practice. The Rules do not make as much of a difference to normal
practice in ordering X-rays as might be expected from previous studies, and the
definition of clinically insignificant fractures to which the OAR do not apply
is a grey area that deserves careful consideration in practice.
Author information:
Simon Wynn-Thomas, Senior Lecturer; Tom Love, Lecturer; Deborah McLeod,
Research Manager; Sue Vernall, Research Nurse; Marjan Kljakovic, Senior
Lecturer; Antony Dowell, Professor; John Durham, Senior Lecturer, General
Practice Department, Wellington School of Medicine and Health Sciences,
Wellington
Acknowledgements: We
thank the GPs, managers and receptionists at the Wellington and Christchurch
After Hours Medical Centres for participating in the study. The study was funded
by grants from Otago University and the Accident Compensation Corporation of New
Zealand.
Correspondence: Tom
Love, Department of General Practice, Wellington School of Medicine and Health
Sciences, PO Box 7343, Wellington South. Fax: (04) 385 5539; email: tom.love@wnmeds.ac.nz
References:
|
|
||||||||||||||||||||||||||||||||||||
| | Current
issue | Search journal |
Archived issues | Classifieds
| Hotline (free ads) Subscribe | Contribute | Advertise | Contact Us | Copyright | Other Journals |
|