26th October 2018, Volume 131 Number 1484

Jerome Ng, Penny Andrew, Paul Muir, Monique Greene, Sabitha Mohan, Jacqui Knight, Phil Hider, Peter Davis, Mary Seddon, Shane Scahill, Jeff Harrison, Lifeng Zhou, Vanessa Selak, Carlene Lawes, Geetha Galgali, Joanna Broad, Marilyn Crawley, Wynn Pevreal, Neil Houston, Tamzin Brott, David Ryan, Jocelyn Peach, Andrew Brant, Dale Bramley

Adverse drug events (ADEs) account for a significant proportion of all iatrogenic harm (up to 38%)1 and improvement initiatives have been introduced into New Zealand hospitals to prevent them.2 The routine identification and measurement of ADEs is important to determine whether initiatives are effective and for informing improvement.3,4

A number of ADE detection and measurement techniques have been described and some have been used in New Zealand hospital settings.4–7 Many of the methods used in these studies are too resource intensive to be sustained by most hospitals.3,4,6 In the absence of any surveillance system, most hospitals rely on incident reporting to monitor harm even though the limitations of these reports are well known. The current challenge is to develop a more efficient surveillance system that can continuously identify and measure ADEs over time.3,4

One promising ADE detection technique which does not require any additional data collection is clinical coding surveillance (CCS).8–10 Information from medical records at New Zealand and Australian public hospitals are routinely coded to generate standardised information about inpatient diagnoses and procedures using the International Classification of Disease 10th Edition, Australian Modification (ICD-10AM). In New Zealand, there is mandatory submission of ICD-10AM coded data to the Ministry of Health (MoH) for National Minimum Dataset (NMDS) collection.5,11–13 Overseas research and practice using CCS for routine monitoring show 0.7–4.5% of hospital admissions are associated with ADEs.8–10,14

Despite sharing the same ICD coding system as Australia, CCS is less established in New Zealand. Aside from some initial research by the Health Quality and Safety Commission (HQSC) and its primary use as a pharmacoepidemiological research tool,15–17 CCS has not been adopted in New Zealand for routine clinical use.18 It is not clear whether CCS is feasible or reliable in local settings for ADE monitoring. This study outlines the use of CCS for routine monitoring of ADEs and describes the characteristics of medication-related harm identified through this approach.


An ADE was defined as any medication-related incident, regardless of the cause, documented by the medical team as having harmed the patient and/or where medical care was necessary to monitor or manage the event(s). This broad definition of ADE includes harm occurring from both adverse drug reactions (ADR) (ie, noxious and unintended responses occurring at normally used doses) and poisonings (ie, intentional or accidental overdoses) (Table 1).8,19 Selection of ADE-related ICD-10AM codes were based on those previously described in the literature and in consultation with the local clinical coding team (Table 1).8

Table 1: ADE-related ICD-10AM codes.

ADE identified:

ADE-related ICD-10AM code types and brief description

Examples of ADE-related codes

By disease manifestation

Disease manifestation codes (DMC): classified by the disease and/or documented clinical symptoms of the ADE. NB: while there are some disease manifestation diagnosis codes specific to ADEs, many are non-specific (eg, nausea) and have to be used in conjunction with other codes or data sources for higher specificity. For example, linkage of ICD-10 codes with community pharmacy dispensing data is required to more reliably ascertain simvastatin associated rhabdomyolysis.19 

Drug specific disease manifestation codes:

N14.0: analgesic nephropathy (eg, non-steroidal anti-inflammatories (NSAIDs))

D52.1: Drug-induced (eg, methotrexate) folate deficiency anaemia

Non-drug specific disease manifestation codes:

R11: Nausea & vomiting

K59: Constipation

F05: Delirium

By drug cause

Chapter XX: External causes of morbidity and mortality

Y40-Y59: Drugs, medicaments and biological substances causing adverse effects in therapeutic use. Includes complications of care such as adverse drug reactions (ADR) which may occur despite appropriate care. NB: excludes accidents in the technique of administration of medicines

Y45 (group code): Analgesics, antipyretics and anti-inflammatory drugs comprising of Y45.0-Y45.9:

Y45.0: Opioids and related analgesics ADEs

Y45.3: Other NSAIDs

Y44.2: Anticoagulants ADEs

Y42.3: Insulin and oral hypoglycaemic ADEs


X40-49: Accidental poisoning by and exposure to noxious substances. Includes accidents in the use of drugs such as accidental overdose, wrong drug given/taken in error, drug taken inadvertently

X40: Accidental poisoning relating to non-opioid analgesics (eg, NSAIDs, paracetamol)

X42: Accidental poisoning relating to narcotics (eg, codeine)


Chapter XIX: Injury, poisoning and certain other consequences of external causes

T36-T50: Poisoning by drugs, medicaments and biological substances. Includes overdose or wrong substance given or taken in error

T42.4: poisoning by benzodiazepines (eg, triazolam)

T43.0: poisoning by tricyclic and tetracyclic antidepressants (eg, amitriptyline)

T40.7: poisoning by cannabis (derivatives)

By disease manifestation AND drug cause

Clustering of both disease manifestation AND external injury cause codes (eg, CHADx11,16,23). ADE is only counted when diagnosis code is immediately followed by one or more relevant external cause code.

Opioid-related nausea and vomiting (N&V) when:

R11: N&V AND immediately followed in sequence by Y45.0: opioids and related analgesic adverse effects

All admissions into Waitemata District Health Board (DHB), which provide secondary hospital services at North Shore (595 beds) and Waitakere (269 beds), from January 2015 to December 2016 with an ADE-related ICD-10AM (8th Edition) external cause code of Y40-Y59, X40-X49 or T36-T50 in any field were extracted using Microsoft Structured Query Language (SQL) Server Management Studio 2012 software (Figure 1). ADEs originating in or out of hospital were defined by the Condition Onset Flag (CoF) associated with diagnostic codes; providing insight into what conditions patients already have when admitted (CoF=2 (outside hospital)) and what arises during hospitalisation (CoF=1 (inside hospital)).12,20

Figure 1: Outline of research process and outputs.


Patient demographic details along with ADE-related diagnosis information associated with these admissions were described. Statistical process control U and Pareto charts were used to identify special cause variation over time and highlight the most common ADE-related ICD codes used respectively.22–24 Analyses were undertaken using Excel 2013 and Minitab® 2015.

To determine whether the coded ADEs and its CoF matched what was clinically documented, two reviewers (doctor and pharmacist) were provided a random list of patients with the ADE-related ICD-10AM code (n=140) (eg, Y44) and its corresponding code description (eg, anticoagulant-related ADE). Clinicians would then review the electronic discharge summary (EDS) to look for the documented ADE. If they find documentation which supports the ICD code they tick Correct. If they do not then they tick ‘unclear/not in EDS’. If incorrect (eg, documentation is for opioid ADE) then they tick ‘Incorrect’. Neither preventability nor the accuracy of what was clinically documented was assessed. Any disagreements were resolved by consensus with assistance from a senior physician. Based on previous research5 it was estimated that the sample size was adequate to provide a positive predictive value (PPV) estimate with reasonable accuracy and narrow confidence intervals (±5%).


In total, there were 244,992 admissions into Waitemata DHB hospitals during 2015–2016 (Table 2). 11,999 ADES were identified by ICD-10AM codes (4.9 ADEs per 100 admissions). Most (66%) ADEs were adverse drug reactions (Y codes). ADEs relating to poisoning (T codes) and accidental poisoning (X codes) were also relatively common, at 28% and 6% respectively. Most (82%) of the ADEs were classified as originating from outside of the hospital. Of all hospitalisations, 9,040 admissions had one or more ADE which gives an overall proportion of admissions with ADEs of 3.7%.

Table 2: Overall ADE numbers and annual average rates.





Total admissions




No. of ADEs identified via coding




    Average no. of ADEs per 100 admissions




Proportion of ADEs by ICD-10AM code type:*

    ADR (ie, Y40-Y59) related

4,029 (67%)

3,902 (66%)

7,931 (66%)

    Poisoning (ie, T36-T50) related

1,623 (27%)

1,749 (29%)

3,372 (28%)

    Accidental poisoning (ie, X40-X49)

381 (6%)

315 (5%)

696 (6%)

Proportion of ADEs by point of origin using CoF:

    1= originate in-hospital

1,079 (18%)

1,085 (18%)

2,164 (18%)

    2= originate outside-hospital

4,954 (82%)

4,881 (82%)

9,835 (82%)

No. of admissions with ≥1 ADE (proportion of admissions (%))

4,537 (3.8)

4,503 (3.6)

9,040 (3.7)

      Avg. no. of ADEs per admission with medication-related harm




    Range of ADEs per patient with medication-related harm




*NB: see Table 1 for descriptions of the ICD-10AM codes and illustrative examples. 

Some variation in the average number of ADEs per 100 admissions occurred over time and special cause variation was evident with higher (Jan 15, Dec 15 and Jan 16) and lower (Aug 15 and Sept 16) rates (Figure 2). ADEs originating outside hospital followed this pattern but no special cause variation was evident for inpatient ADE rates.

Figure 2: Statistical process control charts of average number of ADEs per 100 admissions over time (overall, originating in- and outside-hospital).  


In total, 9,040 admissions had ≥1 ADE. In patients with ADEs, the ALOS, median age and the proportion of female, ≥65 year olds and New Zealand/other Europeans were higher than those of all Waitemata admissions (Table 3).

Table 3: Patient characteristics of admissions with ADEs compared to overall inpatient demographics.

Patient demographics

Patient characteristics of admissions with ≥1 ADEs

n=9,040 admissions

Patient characteristics of all Waitemata admissions*

n=244,992 admissions

Median age (years)



Age groups (years)

No. (%)

No. (%)

0–14 years

481 (5.3%)

29,541 (12.1%)

15–44 years

2,274 (25.2%)

76,923 (31.4%)

45–64 years

1,852 (20.5%)

55,911 (22.8%)

65–74 years

1,367 (15.1%)

33,966 (13.9%)

75–84 years

1,780 (19.7%)

27,445 (11.2%)

85+ years

1,286 (14.2%)

21,206 (8.7%)



3,850 (42.6%)

105,957 (43.2%)


5,190 (57.4%)

139,034 (56.8%)


NZ European

5,121 (56.6%)

126,643 (51.6%)

Other European

1,509 (16.7%)

33,070 (13.5%)


860 (9.5%)

24,373 (9.9%)

Asian (Chinese and Indian)

482 (5.4%)

21,718 (8.8%)


253 (2.8%)

8,487 (3.5%)


815 (9%)

31,074 (12.7%)

Avg. Length of Stay (ALOS) (days)

6.2 days

2.7 days

*NB: obtained via business intelligence tools and provided for relative comparison. 

Adverse drug reactions (Y40-Y59 codes) predominated (n=7,931 ADEs (66%)) among ADEs originating both inside and outside of hospital (Table 4). Of all ADEs originating in hospitals, the majority (97.7%) were ADRs. ADRs were less frequent (59%) among ADEs originating from outside of hospital.

Table 4: ADE characteristics by ICD-10AM group codes.

ADE by cause

No. of ADEs originating in hospital

(n=2,164 ADEs) (%)

No. of ADEs originating outside hospital

(n=9,835 ADEs) (%)

Total no. of ADEs

(n=11,999 ADEs)


ADRs (ie, Y40-Y59)

2,115 (97.7%)

5,816 (59.1%)

7,931 (66.1%)

Poisonings (ie, T36-T50)

28 (1.3%)

3,344 (34%)

3,372 (28.1%)

Accidental poisoning (ie, X40-X49)

21 (1.0%)

675 (6.9%)

696 (5.8%)

The most common drug classes associated with ADEs originating from both inside and outside of hospitals were analgesics (Y45), antibiotics (Y40), anticoagulants (Y44), diuretics (Y54) and cardiovascular (Y52) (Figure 3). Poisoning by psychotropics (T43), anti-epileptics and anti-parkinsonism agents (T42) and non-opioid analgesics were common causes of ADEs originating outside of hospital.

Figure 3: Pareto chart of top 10 most frequently classified ADEs originating inside and outside of hospital (n=11,999 ADEs). 


ADE type

No. of ADEs classified as originating in-hospital

No. of ADEs classified as originating outside of hospital

Cumulative %- in hospital

Cumulative %- outside hospital

Analgesics, antipyretics and anti-inflammatory drugs (Y45)





Systemic antibiotics (Y40)





Agents affecting blood constituents (Y44)





Agents affecting water-balance and mineral and uric acid (Y54)





Agents primarily affecting cardiovascular system (Y52)





Hormones and synthetic substitutes and antagonists, not elsewhere classified (Y42) eg, insulin





Anaesthetics and therapeutic gases (Y48)





Other and unspecified drugs and medicaments (Y57)





Drugs primarily affecting autonomic nervous system (Y51) (eg, metaraminol)





Primarily systemic agents (Y43) (eg, antineoplastic, immunosuppressives)





Poisoning by anti-epileptic, sedative-hypnotic and anti-parkinsonism drugs (T42)





Poisoning by nonopioid analgesics, anti-pyretics and anti-rheumatics (T39)





Poisoning by psychotropic drugs (T43) (eg, antidepressants, antipsychotics)





NB: Arranged largest to smallest by top 10 occurring ADEs originating in-hospital then by those classified as outside hospital. 

A total of 140 ADEs were randomly selected to assess the accuracy of coded ADEs and its CoF against what was clinically documented (Table 5). For eight admissions, the coded ADE was not documented within the electronic discharge summary (EDS). Of the remaining admissions (n=132), the accuracy of the coded ADEs was high (91%). Four errors occurred where incorrect ICD-10AM codes were assigned. In one example, postural hypotension related to the bendrofluazide was wrongly classified using the glucocorticoid-related ADE code (Y42.0) rather than the benzothiadiazine derivative one (Y54.3). The accuracy of the assigned CoF was also high at 91%. In a small number of admissions (3% of the sample) the wrong CoF had been assigned to the diagnosis code.

Table 5: Accuracy of coded ADEs and its CoF with illustrative examples.

Overall summary

Total (%)

(n=140 randomly selected for review)

Accuracy of coded ADE#


128 (91.4%)


4 (2.9%)

Not clear or noted within EDS

8 (5.7%)

Accuracy of CoF*


127 (90.7%)


4 (2.9%)

Not clear or noted within EDS

9 (6.4%)

Accuracy of coded ADE

Illustrative examples (ICD-code, description and verbatim case notes relating to ADEs)


Y55.6: Antiasthmatics, not elsewhere classified: Hypokalaemia secondary to salbutamol nebs

Y40.9: Systemic antibiotic, unspecified: Started on trimethoprim 02/09, taken intermittently over five days having been on cefuroxime for two days before. Became unwell 10/09 with hypotension, rash, fevers, AKI [acute kidney injury] (Creatinine 220) and hyponatraemia (Na 125) with a CRP of 94. Seen by dermatology… - impression of drug reaction to antibiotics


Y42.0: Glucocorticoids and synthetic analogues: bendrofluazide potentially causing postural hypotension with risk of falling [wrong drug class]

T40.4: Other synthetic narcotics: Last night had a headache and difficulties with her family, so took some paracetamol, and then withdrew from the family in her room and continued to take paracetamol two tabs at a time, until she had taken 17 total tablets over the evening...She vomited once. This morning she felt unwell and did not go to school [paracetamol not a synthetic narcotic]

Not clear or noted within EDS

Y45.3: Other non-steroidal anti-inflammatory drugs [NSAID]: nothing noted within EDS

Y45.0: Opioids and related analgesics causing adverse effects in therapeutic use: nothing noted

Accuracy of CoF

Illustrative examples (ICD-code, description and verbatim case notes relating to ADEs)


Y40.1: Cephalosporins and other beta-lactam antibiotics (CoF=1, in hospital): Had episodes of diarrhoea postoperatively, as a result antibiotics were stopped

X44: Accidental poisoning by and exposure to other and unspecified drugs (CoF=2, outside hospital): accidentally given himself 44 unit of Apidra Solostar® [insulin formulation] at 11pm instead of his usual 44 units of Lantus®. He drank some sugary drinks and ate some bread in an attempt to keep his blood sugars high but BSL [blood sugar level] dropped to ~4 and he called for help


Y43.0 (CoF=2, outside hospital): Antiallergic and antiemetic drugs causing adverse effects in therapeutic use: Given domperidone [while in hospital] for nausea - developed itch + rash on arms

Y45.3 (CoF=1, inside hospital): Other non-steroidal anti-inflammatory drugs [NSAID]: Reviewed by renal registrar - AKI thought to be secondary to NSAID use for the last 3–4 months [prior to admission]- acute element to acute ATN [acute tubular necrosis] and cardiorenal failure

Not clear or noted within EDS

T50.9 (CoF=2, outside hospital): Other and unspecified drugs, medicaments and biological substances: nothing noted within EDS

Y45.0 (CoF=1, inside hospital): Opioids and related analgesics causing adverse effects: nothing noted


The aim of this study was to outline the use of CCS for routine monitoring of ADEs and describe the characteristics of medication-related harm identified through this approach. ADEs are relatively prevalent in patients admitted into Waitemata DHB hospitals with an average of 4.9 ADEs per 100 admissions. The average ADE rate observed in this study is consistent with other studies that have used clinical coding surveillance in New Zealand (0.9–7.9), Australia (0.7–4.5), UK (3.2%), Germany (4.8%) and the US (5.7%).5,8,25 Likewise the finding that ADEs are more prevalent in older adults, are mostly ADRs (66%) and associated with longer ALOS and medicines such as analgesics, antibiotics, anticoagulants and diuretics are consistent with previous research.4,7,26–28 By contrast the observation that most ADEs (82%) originated outside hospital was not consistent with other research findings where both the New Zealand Quality in Healthcare Study (NZQHS) and ADE Collaborative (ADEC) studies reported that only 40% or 16–29% of ADEs respectively originated in the community.4,7

Reviews undertaken by two clinicians of a random sample of ADEs showed the majority (90%) were accurately classified as per clinical documentation. For eight admissions, the coded ADE was not documented within the electronic discharge summary (EDS) but may have been identified had the paper medical record been obtained. While some ADEs were assigned the wrong ICD-10AM code and CoF, overall they did not affect the total number found; which provides a level of assurance to the quality and reliability of coded data at Waitemata DHB.

Based on research findings, coding data can be relied upon to reflect clinically diagnosed and documented ADEs. Research findings are generally consistent with those reported in key studies on ADEs, such as NZQHS, where iatrogenic harm has been robustly identified by interdisciplinary teams. The fact that the data obtained using CCS can be relied upon yet can be generated relatively quickly without the need for additional manual data collection in a large and busy DHB indicates CCS is a tool which can feasibly be used for routine ADE detection and measurement.


Current national coding standards mean that if a patient is transferred between general and rehabilitation wards, despite being part of the same hospital journey, this is counted as two admissions.20 In these scenarios, ADEs may be counted twice. Overall, however, the methodological limitation should not affect the average ADE rate because even though harm is counted more than once there is a corresponding increase in denominator size. The emergence of the above limitation prompted further investigation, which reassuringly identified that only a relatively small proportion (n=272 of 9,040 ADE positive admissions (3%)) were of the same hospital journey, so the effect of this is relatively minor. Nonetheless, further research to better understand the size and significance of this problem and adjusting where necessary will further improve data reliability.

Study findings suggest that if ADEs are coded, it is likely they are clinically documented, but it is unclear whether this is conversely true. That is, whether all clinically documented ADEs in medical records are coded. It is also unclear whether all ADEs that occur are correctly diagnosed and documented by the medical team. Research suggests that these processes are not always carried out as well as they could be, which has implications for coding and thus ADE rates.29,30 Future research involving in-depth medical record reviews of the same set of data as those obtained from coding and with a clinical coder will help to reveal the false negatives, false positives and true negatives rates.

The statistical process control charts (SPC) (Figure 2) showed the occurrence of results outside the control limits. Their occurrence seem to be quite variable with (largely ADEs originating outside of hospital) results fluctuating above and below the limits over relatively short periods of observation (six months). It is not clear whether the difference relates to changes in the patient casemix. For example, over the December to January period, one might expect to see a lower number of elective/waiting list admissions and a higher proportion of acutely admitted patients. Coupled with the fact that most ADEs identified through CCS originated from outside of the hospital, there is the potential for ADE proportions to be skewed during the Christmas period and this is a potential confounder. Future research into the reasons for variations through separating SPC charts by admission type (ie, acute or elective/waiting list) may be useful to better understand reasons for variation.

This study did not include outpatients, patient admissions where duration was ≤3 hours or use ADE-related Disease Manifestation Codes (DMC) either alone or in combination with drug-related external cause codes (Table 1) so ADEs may be under-counted. Future research using these codes may provide a more comprehensive insight into the magnitude of harm. Current coding classification systems also do not provide granular detail about ADEs such as its seriousness or preventability. Future and more sophisticated coding schemas such as SNOMED CT (System of Nomenclature of Medicine-Clinical Terms) and ICD-11 when implemented may provide additional information.31

Implications for policy

The limitations of using ADEs as the sole primary metric has previously been outlined but it remains an important component of a multi-dimensional approach to measuring medication safety.3,32,33 A major barrier to the routine collection and analysis of ADE data (eg, audits) for most New Zealand hospitals has been the capacity and resource required.

Because it is mandatory for all DHBs to submit coded data to the MoH as part of NMDS collection, all hospitals in New Zealand will already have the administrative data readily available and most, if not all, should be familiar with how to extract the data. Aggregation of ADE data from across DHBs can be used to identify areas of high risk to better prioritise national improvement initiatives and used to determine their effectiveness. Because standardised ICD-10AM codes are used across DHBs, CCS can be applied to NMDS data, which means that, for the first time, national ADE rates can be routinely and sustainably generated for monitoring over time. Furthermore, the coding of data using internationally agreed standards means that data across hospitals can be compared for the purpose of understanding whether unwarranted variation exists and if so, why; so lessons can be learnt from higher performing organisations.

Implications for system-wide improvement planning and monitoring

Even though hospitals have not focused on the use of the clinical coding dataset for the purpose of monitoring ADEs, it should be relatively straightforward to obtain and in the first instance, use descriptive statistics to identify high-risk areas in hospitals to inform and prioritise improvement. The fact that no additional data collection is required means that a minimal amount of resource, beyond that of an information analyst and someone with medication safety expertise, is required. Because health information is coded in a largely consistent and standardised manner over time within an organisation, significant variations in ADE rates is more likely to reflect system changes rather than variations due to inconsistent data collection. SPC charts can be used to identify special cause variation that signal the need for further investigation or the effects of system changes on ADEs over time.

Research suggests that certain ADE detection tools are more sensitive at detecting particular types of medication-related harm than others.6,34 The prevalence of 4.9 ADEs per 100 admissions identified in this study suggests that CCS detects more medication-related harm than incident reporting but not as many as those using other techniques such as ADE trigger tools. Ideally, CCS should be used in conjunction with other ADE identification tools to obtain a more complete and balanced overview of medication-related harm. In busy, resource-constrained DHB environments, the use of multiple detection tools for ongoing ADE monitoring is often not feasible. Despite lower rates of ADEs detected in this study, their characteristics are generally similar to previous research and because CCS does not require additional data collection, it is a practical solution for quickly identifying areas with highest rates of harm and for continuous ADE monitoring.

This study identified that a large proportion of ADEs (82%) were classified as originating outside of the hospital (n=9,853 ADEs), which is in contrast to rates reported elsewhere. One interpretation of this finding is that CCS is particularly sensitive at detecting ADEs originating from outside of the hospital. The majority of medicines use occurs in primary care35 and so another interpretation is that results indeed reflect the high numbers and proportions of ADEs originating from outside of the hospital. Their sheer number warrants further investigation and serves as a reminder of the importance of improving medicines use in community settings.

It is concerning that 41% of ADEs originating from outside of hospitals were classified as due to poisoning, both accidental and intentional. Such figures highlight the need for better preventative strategies and CCS may be a useful tool (especially using ICD-10AM X and T codes) for organisations such as Accidental Compensation Corporation (ACC) to inform improvement efforts and monitor the effectiveness of their initiatives. Significant proportions of ADEs identified using CCS were poisoning from psychotropics, anti-epileptics, anti-parkinsonism drugs and non-opioid analgesics; which indicates the need for improving medicines use beyond those typically focused on for improvement such as opioids, anticoagulants and insulin.

Implications for future research

This study has provided an overview of the numbers and characteristics of ADEs identified using CCS. More in-depth analyses of ADEs by drug class, location and point of origin and by patient population sub-groups can provide information by which to more precisely pinpoint improvement. Similarly, because coded data contains details of the discharge service and ward, ADE data can be analysed based on these variables for more specific intra-organisational improvement and monitoring.


This study has detailed, for the first time in New Zealand, the characteristics of hospital-wide ADEs identified using CCS and ICD-10AM data. By comparing and contrasting the findings from this study against existing knowledge on ADEs occurring in New Zealand, it is apparent that ADEs can be routinely measured in a reliable and sustainable manner on a local scale. By describing in detail the research process, this work serves to guide other hospitals who may be struggling to routinely measure and monitor ADEs.


The routine measurement of adverse drug events (ADE) is important for monitoring and informing improvement, but current detection tools are manual and too resource intensive. Our research, for the first time, shows ADEs can be reliably and sustainably measured using clinical coding surveillance (CCS). Using CCS, almost 12,000 ADEs over two years were detected in hospitalised patients. Most ADEs originated from the community setting.



To explore the feasibility and reliability of Clinical Coding Surveillance (CCS) for the routine monitoring of Adverse Drug Events (ADE) and describe the characteristics of harm identified through this approach in a large district health board (DHB).


All hospital admissions at Waitemata DHB from 2015 to 2016 with an ADE-related ICD10-AM code of Y40-Y59, X40-X49 or T36-T50 were extracted from clinical coded data. The data was analysed using descriptive statistics, statistical process control and Pareto charts. Two clinicians assessed a random sample of 140 ADEs for their accuracy against what was clinically documented in medical records.


A total of 11,999 ADEs were identified in 244,992 admissions (4.9 ADEs per 100 admissions). ADEs were more prevalent in older adults and associated with longer average length of stays and medicines such as analgesics, antibiotics, anticoagulants and diuretics. Only 2,164 (18%) of ADEs were classified as originating within hospital. Of ADEs originating outside of the hospital, the main causes were poisoning by psychotropics, anti-epileptics and anti-parkinsonism agents and non-opioid analgesics. Clinicians agreed that 91% of ADE positive admissions were accurately classified as per clinical documentation.


CCS is a feasible and reliable approach for the routine monitoring of ADEs in hospitals.

Author Information

Jerome Ng, Lead Advisor, Improvement, Research & Informatics, Institute for Innovation and Improvement (i3), Waitemata DHB, Auckland; Penny Andrew, Director, Institute for Innovation and Improvement (i3), Waitemata DHB, Auckland; Paul Muir, Medical Fellow, Planning, Funding and Outcomes, Waitemata DHB, Auckland; Monique Greene, Information Analyst, Institute for Innovation and Improvement (i3), Waitemata DHB, Auckland; Sabitha Mohan, Clinical Coding Auditor, Health Information Group, Waitemata DHB, Auckland; Jacqui Knight, Clinical Coding Team Leader, Health Information Group, Waitemata DHB, Auckland; Phil Hider, Senior Lecturer, Department of Population Health, University of Otago, Christchurch; Peter Davis, Professor, Centre of Methods and Policy Application in the Social Sciences (COMPASS), University of Auckland, Auckland;
Mary Seddon, Independent Consultant, Seddon Healthcare Quality, Auckland;
Shane Scahill, Senior Lecturer, School of Management, Massey University, Auckland;
Jeff Harrison, Associate Professor, School of Pharmacy, University of Auckland, Auckland;
Lifeng Zhou, Chief Advisor for Asian International Collaboration, Waitemata District Health Board, Auckland; Vanessa Selak, Senior Lecturer, School of Population Health, University of Auckland, Auckland; Carlene Lawes, Public Health Physician (Surgical), Institute for Innovation and Improvement (i3), Waitemata DHB, Auckland; Geetha Galgali, Public Health Physician (Maternity), Child, Women and Family, Waitemata DHB, Auckland; Joanna Broad, Senior Research Fellow, Department of Geriatric Medicine, University of Auckland, Auckland; Marilyn Crawley, Chief Pharmacist, Pharmacy Department, Waitemata DHB, Auckland; Wynn Pevreal, Medication Safety Pharmacist, Pharmacy Department, Waitemata DHB, Auckland (Died 24 April 2018); Neil Houston, Clinical Director for Safety and Quality in Primary Care, Waitemata DHB, Auckland; Tamzin Brott, Executive Director—Allied Health, Scientific & Technical Professions, Waitemata DHB, Auckland;
David Ryan, Information Systems Change Manager, Health Information Group, Waitemata DHB, Auckland; Jocelyn Peach, Director of Nursing and Midwifery, Waitemata DHB, Auckland;
Andrew Brant, Chief Medical Officer, Waitemata DHB, Auckland; Dale Bramley, Chief Executive Officer, Waitemata DHB, Auckland.


We gratefully acknowledge the following people for their support and advice: Ross S, Wallace I, Gilmour K, Farmer E, Wood P, Doogue M, Wilkinson J, Young M, Li Y, Hamblin R and Armstrong D.


Dr Jerome Ng, Institute for Innovation and Improvement, Waitemata DHB West Wing, Taharoto Building (Building 1), North Shore Hospital Site, Takapuna, Private Bag 93-503, Auckland 0740.

Correspondence Email


Competing Interests



  1. Council of Europe Expert Group on Safe Medication Practices. Creation of a better medication safety culture in Europe: building up safe medication practices 2006 [cited 2010 17/03/2010]. Available from: http://www.gs1health.net/downloads/medication.safety.report.2007.pdf
  2. SQM. Safe and Quality Use of Medicines 2005-2007 Report: Ministry of Health; 2008 [cited 2010 19/05/2010]. Available from: http://www.safeuseofmedicines.co.nz/Portals/0/About/S&QuseofMeds05to07.pdf
  3. Ng J, Scahill S, Harrison J. Getting the foundations right for the measurement of medication safety: the need for a meaningful conceptual frame N Z Med J. 2017; 130(1452):54–62.
  4. Robb G, Loe E, Maharaj A, Hamblin R, Seddon M. Medication-related patient harm in New Zealand hospitals. N Z Med J. 2017; 130(1460):21–32.
  5. Sapere Research Group. Report A: A framework for the measurement of medication-related harm. 2013.
  6. Briant R, Ali W, Lay-Yee R, Davis P. Representative case series from public hospital admissions 1998 I: drug and related therapeutic adverse events. N Z Med J. 2004; 117(1188).
  7. Davis P, Lay-Yee R, Briant R. Adverse events in New Zealand Public Hospitals: Principal Findings from a National Survey Wellington: Ministry of Health; 2001 [cited 2010 10/02/2010]. Available from: http://www.moh.govt.nz/publications/adverseevents
  8. Kunac D, Reith D. Preventable medication-related events in hospitalised children in New Zealand. N Z Med J. 2008; 121(1272):17–32.
  9. Seddon M, Jackson A, Cameron C, Young M, Escott L, Maharaj A, et al. The Adverse Drug Event Collaborative: a joint venture to measure medication-related patient harm. N Z Med J. 2013; 126(1368):9–20.
  10. Hohl C, Karpov A, Reddekopp L, Stausberg J. ICD-10 codes used to identify adverse drug events in administrative data: a systematic review. J Am Med Inform Assoc. 2014; 21:547–57.
  11. Parikh S, Christensen D, Stuchbery P, Peterson J, Hutchinson A, Jackson T. Exploring in-hospital adverse drug events using ICD-10 codes. Australian Health Review. 2014; 38(4):454–60.
  12. Hodgkinson M, Dirnbauer N, Larmour I. Identification of Adverse Drug Reactions Using the ICD-10 Australian Modification Clinical Coding Surveillance. Journal of Pharmacy Practice and Research Volume. 2009; 39(1):19–23.
  13. Ministry of Health. National Minimum Dataset (hospital events): Ministry of Health; 2017 [updated 17 August 2015; cited 2017 06/11/2017]. Available from: http://www.health.govt.nz/nz-health-statistics/national-collections-and-surveys/collections/national-minimum-dataset-hospital-events
  14. National Health Board. National Minimum Dataset (Hospital Events) Data Dictionary. Wellington: Ministry of Health, 2014.
  15. National Centre for Classification in Health. Volume. International statistical classification of diseases and related health problems 10th revision Australian modification (ICD-10-AM). 3rd ed. Sydney: National Centre for Classification in Health; 2002.
  16. Australian Commission on Safety and Quality in Health Care. Classification of Hospital Acquired Diagnoses (CHADx) Australian Commission on Safety and Quality in Health Care; 2014 [cited 2014 15/12/2014]. Available from: http://www.safetyandquality.gov.au/our-work/information-strategy/health-information-standards/classification-of-hospital-acquired-diagnoses-chadx/
  17. Kumpula E-K, Nada-Raja S, Norris P, Quigley P. A descriptive study of intentional self-poisoning from New Zealand national registry data. Australian and New Zealand Journal of Public Health. 2017; 41(5):535–40.
  18. Nishtala P, Ndukwe H, Chyou T, Salahudeen M, Narayan S. An overview of pharmacoepidemiology in New Zealand: medical databases, registries and research achievements. N Z Med J. 2017; 130(1449):52–8.
  19. Parkin L, Paul C, Herbison GP. Simvastatin dose and risk of rhabdomyolysis: Nested case–control study based on national health and drug dispensing data. International Journal of Cardiology. 2014; 174(1):83–9.
  20. Hider P, Parker K, von Randow M, Milne B, Lay-Yee R, Davis P. Can patient safety indicators monitor medical and surgical care at New Zealand public hospitals? N Z Med J. 2014; 127(1405):32–44.
  21. World Health Organization. Technical Report No 498: International Drug Monitoring, The Role of National Centres. Geneva: The Institute, 1972.
  22. Rains M, Thompson T. The New Zealand Casemix System: An Overview. Wellington, NZ: Casemix Working Group, 2015.
  23. HealthRoundTable. 3.1: Complications Rate & Analysis (CHADx): HealthRoundTable; 2016 [cited 2016 21.03.2016]. Available from: http://www.hed.nhs.uk/portalaustralia/Module.aspx?reportID=10
  24. Langley GJ, Moen RD, Nolan KM, Nolan TW, Norman CL, Provost LP. The improvement guide: a practical approach to enhancing organizational performance: John Wiley & Sons; 2009.
  25. Benneyan JC, Lloyd RC, Plsek PE. Statistical process control as a tool for research and healthcare improvement. Quality and Safety in Health Care. 2003; 12(6):458–64.
  26. Perla R, Provost L, Murray SK. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Quality & Safety. 2011; 20:46–51.
  27. Stausberg J. International prevalence of adverse drug events in hospitals: an analysis of routine data from England, Germany, and the USA. BMC Health Services Research. 2014; 14(1):125.
  28. Bates D, Cullen D, Laird N, et al. Incidence of adverse drug events and potential adverse drug events: implications for prevention. JAMA. 1995; 274:29–34.
  29. Abernethy AP, Herndon JE, Wheeler JL, Rowe K, Marcello J, Patwardhan M. Poor Documentation Prevents Adequate Assessment of Quality Metrics in Colorectal Cancer. Journal of Oncology Practice. 2009; 5(4):167–74.
  30. Graber ML, Wachter RM, Cassel CK. Bringing diagnosis into the quality and safety equations. JAMA. 2012; 308(12):1211–2.
  31. Cao F, Sun X, Wang X, Li B, Li J, Pan Y. Ontology-based knowledge management for personalized adverse drug events detection. Studies in Health Technology & Informatics. 2011; 169:699–703.
  32. Ng J, Scahill S, Harrison J. Stakeholder views do matter: a conceptual framework for medication safety measurement. Journal of Pharmaceutical Health Services Research. 2018; 9(1):21–31.
  33. Ng J, Andrew P, Crawley M, Pevreal W, Peach J. Assessing a hospital medication system for patient safety: findings and lessons learnt from trialling an Australian modified tool at Waitemata District Health Board. N Z Med J. 2016; 129(1430).
  34. Jha A, Kuperman G, Teich J, et al. Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc. 1998; 5:305–14.
  35. Sandars J, Esmail A. The frequency and nature of medical error in primary care: understanding the diversity across studies. Family Practice. 2003;20(3):231-6.