28th April 2017, Volume 130 Number 1454

Kieran Holland, Graham McGeoch, Carolyn Gullery

Most health systems struggle to meet demand for care. However, not all services delivered are of value to patients,1,2 and some patients experience overdiagnosis and overtreatment.3 Strategies are needed to reduce unwarranted variation in diagnosis and treatment, and to focus resources on the care of most value to patients.4

Concern about overuse and rising costs in the US health system led to the Choosing Wisely campaign. Launched in 2012, nine leading US medical organisations each nominated five overused tests or treatments.5 More than half of the nominations were directly related to radiology.6 Radiology features similarly in Choosing Wisely campaigns started subsequently in other countries, including New Zealand.7 Low value imaging is particularly problematic as modern imaging techniques detect ‘incidentalomas’ in up to 40% of people, leading to patient anxiety and further follow-up investigations.8 Strategies for managing value are therefore particularly important in the area of radiology.

There is substantial opportunity to improve quality and value of care at the point of referral. Studies suggest that outpatient referral quality from general practice is variable to a degree that cannot be explained by differences in practice populations, and that a substantial number of referred cases could be managed in primary care.9 There is less evidence about the quality of radiology referrals, but wide variation has been observed in hospital settings.10,11

Referral management strategies are diverse, ranging from attempts to indirectly influence referral behaviour through to direct intervention in the referral process.12 Indirect referral management strategies include education, clinical guidelines, decision support systems, peer review with feedback and financial incentives. Direct referral management strategies include structured referral forms, clinical triage and referral redirection. There is some evidence supporting indirect referral management strategies,12 but previous reviews have found little evidence that direct strategies are cost-effective.12–14

Relatively little has been published about referral management in the area of primary care radiology, though there is evidence that guidelines and feedback reduce x-ray requests.15–17 A holistic approach may be needed because isolated interventions are likely to have complex effects.14

Over the last decade, the Canterbury health system, in the South Island of New Zealand, has worked towards a vision of integrated health and social care, delivered close to people’s homes.18–20 The Canterbury region has a population of 539,000.21 The region has 120 general practice teams and over 600 general practitioners. Community radiology is provided at no cost to patients within a constrained budget managed by the Canterbury District Health Board (DHB), which receives population-based government funding.

In 2009, early in the development of a broad suite of community-based clinical pathways,22 it became clear that primary care access to radiology was often a barrier to caring for patients in the community. Waiting times were growing and the budget for community radiology had been exceeded. There was limited ability for the system to adapt to changes in demand or provider capacity.

General practitioners in Canterbury were making almost 40,000 referrals annually for radiology, including plain film, ultrasound and CT imaging. Preliminary referral audits identified a range of issues, including: 1) complicated referral processes, 2) variable referral quality, including inflation of referral urgency in 30% of requests and questionable clinical indications in 35% of plain film requests, 3) referrals to specialist services for diagnostics that could potentially be avoided if the investigation were available in the community and 4) poor prioritisation of waiting lists with unacceptably long waits for some patients.

This paper describes the implementation of a multifaceted intervention to improve primary care radiology referral quality and value in Canterbury, and discusses the transferability of the outcomes to other health systems.


In 2009, a small team was resourced by the Canterbury DHB to develop a better model for community radiology. The team established a radiology workgroup, including general practitioners, radiologists and hospital specialists. Workgroup members were introduced to the issues and asked to brainstorm solutions.

Subsequently, between 2010 and 2014, the following changes were implemented:

  • Clinical access criteria to prioritise requests.
  • Clinical pathways to promote testing guidance, access criteria and patient information.
  • Electronic referral to improve referral accuracy and administration.
  • A shared health record to increase visibility of prior results.
  • Referral process improvements to streamline referral flow.
  • Referral quality improvement strategies to ensure appropriate testing.
  • Radiology reporting guidelines to improve utility of results.
  • Clinical pathway audits to ensure patient safety.

Clinical access criteria

To provide a framework for prioritising requests, the workgroup developed clinical access criteria for community radiology in consultation with hospital specialists. The access criteria were designed to balance evidence and available resources. Gaps between the access criteria and best practice were highlighted to encourage general practitioners to discuss patient-funded options. To reduce the demand for urgent bookings, the access criteria were subsequently refined to specify explicit maximum wait times (<24 hours, <1 week, <4 weeks) for each condition.

Relative to previous practice, the criteria were most restrictive in the following areas:

  • X-ray for suspected osteoarthritis without red flags.
  • Post-pneumonia chest x-ray without risk factors for underlying cancer.
  • Bone density scans when the estimated 10-year hip fracture risk was under 3%.
  • Thyroid ultrasound where locally agreed pathways recommend other first line investigations or specialist referral.
  • Musculoskeletal ultrasound where there was limited capacity for publicly funded treatment.

Clinical pathways

To disseminate the access criteria, these were published in an online clinical pathway tool for general practice teams called HealthPathways (http://www.healthpathwayscommunity.org).22 HealthPathways is developed collaboratively by general practitioners and hospital specialists, and provides evidence-informed clinical guidance for common conditions coupled with local service information and referral processes. It has a high level of adoption in Canterbury, with 94% of surveyed general practitioners reporting it is easy to use in clinical practice and 90% reporting it has improved the care they provide to patients.23

Where possible, clinical pathways were modified to reinforce the criteria by recommending alternatives, including watchful waiting, other diagnostics or specialist referral. Aligned patient information is also provided in a public website (http://www.healthinfo.org.nz).

Electronic referral

To support locally agreed pathways and access criteria, Canterbury developed the Electronic Request Management System (ERMS). ERMS is a centrally managed electronic referral system enabling referrals from general practice to any service in the health system, including publicly- and privately-funded hospital and community services. It is integrated with general practice software to enable forms to be automatically populated with accurate information. It provides guidance on required referral information and reduces the risk of misdirection. In 2010, all referrals to community radiology were sent via fax. By late 2015, over 95% of referrals were electronically generated.

Shared health record

To reduce the risk of duplicate testing, prior radiology results were made available to general practice and hospital teams via an electronic shared health record called HealthOne. HealthOne, developed in Canterbury, includes information about the majority of the Canterbury population and gives community and hospital clinicians a unified view of a patient’s clinical history.

Referral process improvement

To enable control of referral flow and to improve capture of referral data, a community radiology referral hub was established. Historically, general practices sent referrals directly to radiology providers, making it difficult to shift volumes between providers as demand and capacity changed. Using ERMS, the referral hub can efficiently coordinate referral triage and flexibly allocate requests to providers depending on demand and capacity. For example, when the Canterbury DHB’s primary community CT scanner had an extended outage in 2012, referrals were seamlessly re-routed to the hospital CT service. Prior to the referral hub it would have been necessary to instruct each of the local general practices to update their referral templates and fax numbers, which is an inaccurate and inefficient process. With a single administrator, the hub is able to process 40,000 referrals per year.

Referral triage and feedback

Local radiologists noted that variability in the quality of referrals from general practice was an obstacle to accurate radiology reporting. The importance of a succinct clinical question and relevant clinical background was promoted to general practitioners and reinforced in the ERMS referral system. For example, the pelvic ultrasound request form prompts for the date of the last menstrual period in pre-menopausal women so that the scan can be scheduled early in the menstrual cycle when imaging is most informative.

Referrals are triaged according to agreed access criteria by a small team of general practitioners. The electronic triage system makes the process flexible and efficient. A range of predefined reasons for returning requests have been developed, including references back to the relevant clinical pathway. Triagers can optionally modify these reasons to fit a specific case and are encouraged to superimpose their clinical judgement rather than applying the criteria rigidly. Referrers are advised to resubmit their request if they disagree with the decision of a triager. Referral triage decisions have been intermittently audited to ensure equity of access. Triagers can work from anywhere with an internet connection and most referrals are triaged within a few days. Overall, approximately 75% of referrals are triaged, with 13% of these being returned to the referrer with advice. Requests for urgent imaging bypass triage to avoid delays.

General practitioners are provided with feedback about their overall referral patterns relative to their peers. This data is discussed during practice liaison visits and at general practitioner education sessions.

Radiology reporting guidelines

Local general practitioners noted that radiology reports were often difficult to interpret and would sometimes recommend ordering investigations that were declined by the public health system. To ensure radiology reports were meaningful to general practitioners, guidelines were developed for radiologists outlining the features considered important:

  • A concise report identifying key points of clinical importance.
  • A clear distinction between normal and significantly abnormal.
  • Provision of a diagnosis or differential diagnosis.
  • Comparison with any previous images.
  • A phone call to practices about critical results (eg, new cancer).
  • Careful wording of follow-up recommendations, taking into account locally agreed clinical pathways.

A standard footer was added to all reports stating that “Recommendations made in this report do not necessarily indicate availability via publicly-funded clinical pathways. Please refer to HealthPathways for further information.” A radiology advice phone line was promoted to enable general practitioners to discuss any points of uncertainty with a radiologist.

Clinical pathway audit

A clinical audit programme was established to enable continuous quality improvement of clinical pathways. The focus has been on areas where new radiology pathways have been introduced or access has been restricted. Audits of patients declined imaging have provided reassurance that patients are being appropriately selected and followed-up, without compromising patient safety.


Improved referral quality

Triagers have observed a steady improvement in referral quality since the inception of the service with an increase in the median acceptance rate for individual referrers from 78% in 2011 to 88% in 2015 (Figure 1). Improvements have been most noticeable in areas such as gynaecological ultrasound where there has been a holistic approach, including well-developed clinical pathways, clear referral criteria, general practitioner education and detailed and constructive feedback provided to referrers from both gynaecology and radiology services.

Figure 1: Distribution of acceptance rates of radiology referrals for individual referrers making at least 15 referrals in the calendar year. 


The bottom and top of the box correspond to the first and third quartiles, respectively. The whiskers extend 1.5 times the interquartile range. 

Linking access criteria to explicit maximum wait times provided guidance and certainty for referrers. The proportion of referrals requested urgently (within one week) reduced from 59% in the 12-month period before the change to 22% in the subsequent 12 months (Table 1). The reduction in urgency of requests simplified the booking process and reduced pressure on administrative staff, eliminating delays for acute cases.

Table 1: Distribution of requested priorities in the 12-month periods before and after access criteria were linked to explicit target wait times. 

Requested priority

Before (%)

After (%)

Within 4 weeks



Within 1 week



Within 24 hours



Improved referral value

Following the establishment of the community radiology referral management service in 2009, there has been a sustained reduction in plain film volumes of 40% (Figure 2). Volumes of CT and ultrasound have continued to grow. This growth is partly explained by a shift in imaging from the hospital to the community, as new community-based pathways of care have been introduced. In addition, it is likely there is a degree of ‘pathway-driven demand’, as patients who were not previously investigated are now being investigated by general practitioners who have adopted pathway recommendations.

Figure 2: Volume of radiology referrals accepted by financial year. 


Whereas previously waiting times were unpredictable, imaging is now performed consistently within the target wait times. For patients at risk of hospital admission, general practitioners can reliably access same-day x-ray, ultrasound and CT urogram.

Resources conserved by a reduction in low-value imaging have been reallocated to enable new clinical pathways to support community-based investigation and management, including:

  • Breast imaging and biopsy for breast symptoms.
  • CT colonography for lower risk or frail patients with colorectal symptoms.
  • CT head and carotid ultrasound for transient ischaemic attack.
  • CT head for cognitive impairment.
  • CT urogram for renal colic.
  • Pelvic ultrasound for post-menopausal bleeding.

For an example of the benefits of resource reallocation, see the case study on the breast symptoms pathway in the supplementary materials.

Patient and clinician experience

In the first year of operation, the service received a number of complaints from clinicians and patients. Most complaints related to the reduction in access to lower value imaging. Each complaint was investigated and used to inform improvements to the service, including clarifications to clinical pathways, access criteria and processes. This feedback cycle has been critical to the ongoing development and improvement of the service. The frequency of complaints declined steadily and had become rare within three years of establishment.

Informal discussion with local clinicians suggests that by 2016 there was a high level of satisfaction with the service. Many find it useful to share access criteria with their patients during a consultation when they are pressured to order imaging they do not consider clinically necessary. The framework supports good clinical judgement, reducing pressures to practice defensively.


Clinical guidelines and associated access criteria are often ignored.24–26 This paper draws on real world referral and audit data, collected over a seven-year period, to demonstrate that a multifaceted intervention led to sustained improvement in primary care radiology referral quality and value. We are not able to evaluate the relative contribution of each improvement intervention.

Our implementation is distinct because the access criteria have been embedded into pathway and referral tools that are highly valued by local general practice teams. The pathways and criteria are closely aligned, and reflect local resources, so are directly relevant to local clinical practice. The criteria have been reinforced with a range of direct and indirect referral management strategies.

Current perspectives on referral management have inevitably been shaped by existing implementations.27,28 In the UK, only 29% of the primary care trusts interviewed by The King’s Fund viewed quality improvement as a primary driver for referral management.12 Our implementation has focused on a range of outcomes, including quality improvement, patient experience and choice and enabling new pathways. In exchange for constraints, local clinicians have seen benefits of reduced waiting times and increased access to advanced imaging.

Published evidence favours retrospective audit of referrals over prospective triage.12 Our observation is that retrospective approaches are often not sustained beyond the scope of research. Prospective triage, when implemented as an operationally critical step in patient care, is carried out with the same sense of clinical duty as other clinical activities. Canterbury’s community radiology service has sustained prospective triage for seven years, and communication between referrers and triagers has informed continual improvements to access criteria and clinical pathways.

Prospective triage can also deliver immediate benefit to clinicians and patients. In Canterbury, it has been a powerful tool to raise awareness of locally agreed clinical pathways. Declined requests usually refer back to a particular pathway. Clinicians have commented that while this is sometimes frustrating it can also be a useful way to learn about new pathways during the course of regular clinical practice.

Clinical risk from poor triage decisions is an argument against direct referral management.12 The ethical imperative, however, depends on the risk of the status quo. As we observed in Canterbury, pre-existing clinical risk may be considerable when referred services are poorly prioritised. Our experience suggests that the risks of direct referral management can be controlled if monitored carefully.

While operational triage data implies that referral quality and appropriateness has improved, a formal comparison of referral quality before and after the establishment of the service would be informative. It is possible that referral acceptance rates have increased partly due to ‘gaming’ of the system,29 although the sustained reduction in volumes suggests that the extent of any such behaviour is limited.

Clinician experience of the community radiology service reported here is anecdotal based on discussions with general practitioners and the low level of complaints to the service in recent years. A systematic evaluation of referrer and patient experience would be a useful adjunct.

Sustainability and transferability of these improvements are difficult to study systematically. Interventions occur in the context of an evolving health system, making it impossible to control for all variables. Outcomes are highly dependent on context and the quality of implementation, so may not be directly transferable between health systems.

We believe the following elements have been critical to sustaining the outcomes achieved by the community radiology service in Canterbury:

  • A philosophy of quality improvement and waste avoidance rather than cost saving.
  • Long-term continuity of clinical and administrative leadership.
  • A supportive funder.
  • Broad clinical engagement with hospital specialists and general practitioners.
  • A multifaceted intervention including direct and indirect referral management strategies.
  • A rich and tightly integrated set of health system tools to embed process change.
  • An agile and responsive implementation model.

In 2013, the New Zealand Ministry of Health established a working group to develop a national set of access criteria for community radiology to improve equity of access.30 The national criteria were based on those developed in a few DHBs including Canterbury. DHBs have been encouraged to implement these criteria. While the national criteria provide a helpful starting point, it is not yet clear that these have increased quality and value of referrals in other districts. Our belief is that a multifaceted approach is needed to deliver the full benefits of a set of access criteria.

The multifaceted approach described here is potentially relevant to other referred services. However, the Choosing Wisely nominations suggest that imaging may be a more overused and discretionary resource than specialist assessment and treatment,5 so the potential gains could be smaller in other clinical areas.


Case study: breast symptoms pathway

In 2011, waiting times for assessment of patients with breast symptoms at the Christchurch Hospital General Surgery Department exceeded treatment targets. A workgroup was established and identified that the majority of women referred had benign breast conditions (Figure 3). The group recommended that breast diagnostic imaging and biopsy be arranged in the community by general practitioners to free up surgeon capacity for clinics and surgery. The new pathway was documented on HealthPathways and a contract was negotiated between the community radiology service and a private breast radiology provider. The new pathway was launched in November 2011. After this date, referrals sent directly to the General Surgery Department were returned to the referrer with a reference to the new pathway. Of women referred by general practitioners for imaging, only about 5% required subsequent surgical assessment (Figure 4). Nearly all patients now receive a diagnosis within 1–4 weeks. Those with benign conditions can be reassured by their general practitioners and do not need to traverse the hospital system. Flows in the surgical service have improved so that women with serious pathology are seen and treated more quickly. The new pathway reduced the maximum wait for treatment from 30 weeks to 10 weeks, with urgent cases receiving treatment within a month.

Figure 3: Canterbury’s breast symptoms pathway prior to November 2011. 


Figure 4: Canterbury’s streamlined breast symptoms pathway after November 2011.  



This article describes changes made to Canterbury’s community radiology service over a seven-year period to improve the quality and value of care delivered to patients. Changes included clearly defining access criteria according to patient need on the HealthPathways website, improvements to processes and systems, and analysis and feedback of referral data to general practitioners. Conserved resources were reallocated to support new community-based pathways for care.



This article describes a seven-year multifaceted intervention leading to sustained improvement in primary care radiology referral quality and value in Canterbury, New Zealand, and discusses the transferability to other health systems.


Access criteria were developed with input from general practitioners and hospital-based specialists, and embedded in locally developed clinical pathways. A referral management service was created to streamline referral processes. Systems were developed to enable electronic referral and triage, and to provide visibility of prior imaging. A team of general practitioners was formed to continually review referrals relative to agreed criteria and to provide advice to referrers. Referring general practitioners were provided data and education about their referral patterns relative to their peers. A clinical audit programme was introduced to ensure quality and safety of care.


The service achieved sustained improvements in referral quality (referral acceptance rates increased from 78% to 88%, urgent referrals reduced from 59% to 22%) and value (plain film volumes reduced by 40%).


Sustained improvement to primary care radiology referral quality and value is achievable at scale using a multifaceted intervention. The transferability of this outcome is likely to be connected to supporting factors present in the Canterbury health system.

Author Information

Kieran Holland, Manager Community Referred Radiology, Canterbury Initiative, Christchurch; Graham McGeoch, General Practitioner, Canterbury Initiative, Christchurch; Carolyn Gullery, General Manager Planning, Funding & Decision Support, Canterbury District Health Board, Christchurch.


The authors wish to acknowledge the many administrators and clinicians across Canterbury who have contributed to the smooth operation of the community radiology service. Thanks to Gill Fowler for facilitating the relationship between public and private sectors, Sharyn MacDonald for her leadership and support of the concept of an integrated radiology service, and Nikki Elliot for her commitment and dedication to clinical audit. Our gratitude to Greg Hamilton, Tom Love and Brett Shand for their careful review of this paper. A special thanks to Bruce Penny and David Kerr for their leadership of the Canterbury Initiative team.


Dr Kieran Holland, Manager Community Referred Radiology, Canterbury Initiative, Christchurch.

Correspondence Email


Competing Interests

Dr Holland and Dr McGeoch report personal fees from Canterbury DHB and Streamliners outside the submitted work. Dr McGeoch has been clinical leader of the community-referred radiology programme for eight years. Carolyn Gullery is currently employed by Canterbury DHB which funds the Canterbury Initiative.


  1. Wennberg JE. Tracking medicine: a researcher’s quest to understand health care. Oxford University Press; 2010.
  2. Coulter A, Collins A. Making shared decision-making a reality: No decision about me, without me. London: The King’s Fund; 2011.
  3. Moynihan R, Doust J, Henry D. Preventing overdiagnosis: how to stop harming the healthy. Brit Med J. 2012; 344:e3502.
  4. Porter ME. What is value in health care? New Engl J Med. 2010; 363(26):2477–2481.
  5. Cassel CK, Guest JA. Choosing wisely: helping physicians and patients make smart decisions about their care. J Amer Med Assoc. 2012; 307(17):1801–1802.
  6. Rao VM, Levin DC. The overuse of diagnostic imaging and the Choosing Wisely initiative. Ann Intern Med. 2012; 157(8):574–576.
  7. Read D, Sherwood D, Ineson S. Choose Wisely—is more always better? N Z Med J. 2016; 129(1447):10–12.
  8. Orme NM, Fletcher JG, Siddiki HA, et al. Incidental findings in imaging research: evaluating incidence, benefit, and burden. Arch Intern Med. 2010; 170(17):1525–1532.
  9. Foot C, Naylor C, Imison C. The quality of GP diagnosis and referral. London: The King’s Fund. 2010.
  10. Triantopoulou C, Tsalafoutas I, Maniatis P, et al. Analysis of radiological examination request forms in conjunction with justification of X-ray exposures. Eur J Radiol. 2005; 53(2):306–311.
  11. Oswal D, Sapherson D, Rehman A. A study of adequacy of completion of radiology request forms. Radiography. 2009; 15(3):209–213.
  12. Imison C, Naylor C. Referral management: lessons for success. London: The King’s Fund; 2010.
  13. Akbari A, Mayhew A, Al-Alawi M, et al. Interventions to improve outpatient referrals from primary to secondary care. The Cochrane Collaboration; 2008.
  14. Blank L, Baxter S, Woods HB, et al. Referral interventions from primary to specialist care: a systematic review of international evidence. Br J Gen Pract. 2014; 64(629):e765–e774.
  15. Kerry S, Oakeshott P, Dundas D, et al. Influence of postal distribution of the Royal College of Radiologists’ guidelines, together with feedback on radiological referral rates, on X-ray referrals from general practice: a randomized controlled trial. Fam Pract. 2000; 17(1):46–52.
  16. Eccles M, Steen N, Grimshaw J, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. The Lancet. 2001; 357(9266):1406–1409.
  17. Glaves J. The use of radiological guidelines to achieve a sustained reduction in the number of radiographic examinations of the cervical spine, lumbar spine and knees performed for GPs. Clin Radiol. 2005; 60(8):914–920.
  18. Timmins N, Ham C. The quest for integrated health and social care: A case study in Canterbury, New Zealand. London: The King’s Fund; 2013.
  19. Gullery C, Hamilton G. Towards integrated person-centred healthcare: The Canterbury journey. Fut Hosp J. 2015; 2(2):111–116.
  20. Schluter PJ, Hamilton GJ, Deely JM, et al. Impact of integrated health system changes, accelerated due to an earthquake, on emergency department attendances and acute admissions: a Bayesian change-point analysis. BMJ Open. 2016; 6(5):e010709.
  21. New Zealand Census. Statistics New Zealand. http://www.stats.govt.nz/Census/2013-census/profile-and-summary-reports.aspx. Published October 2013. Accessed January 2016.
  22. McGeoch G, Anderson I, Gibson J, et al. Consensus pathways: evidence into practice. N Z Med J. 2014; 128(1408):86–96.
  23. McGeoch G, McGeoch P, Shand B. Is HealthPathways effective? An online survey of hospital clinicians, general practitioners and practice nurses. N Z Med J. 2015; 128(1408):2010–2019.
  24. Cranney M, Warren E, Barton S, et al. Why do GPs not implement evidence-based guidelines? A descriptive study. Fam Pract. 2001; 18(4):359–363.
  25. Gabbay J, le May A. Evidence based guidelines or collectively constructed “mindlines?” Ethnographic study of knowledge management in primary care. Brit Med J. 2004; 329(7473):1013.
  26. Lugtenberg M, Zegers-van Schaick JM, Westert GP, et al. Why don’t physicians adhere to guideline recommendations in practice? An analysis of barriers among Dutch general practitioners. Implement Sci. 2009; 4(54):5908–4.
  27. Davies M, Elwyn G. Referral management centres: promising innovations or Trojan horses? Brit Med J. 2006; 332(7545):844–846.
  28. Iacobucci G. Referral management schemes: good for whom? Brit Med J. 2017; 356(6856).
  29. Morreim EH. Gaming the system: dodging the rules, ruling the dodgers. Arch Intern Med. 1991; 151(3):443–447.
  30. National Criteria for Access to Community Radiology. Wellington, New Zealand: Ministry of Health; 2015.