5th October 2018, Volume 131 Number 1483

Nina Scott, Helen Clark, Bridget Kool, Shanthi Ameratunga, Grant Christey, Donna Cormack

Injury, as a leading cause of morbidity and mortality, contributes to health inequity in New Zealand and is a major socio-economic burden.1,2 Health inequities are defined as “differences which are unnecessary and avoidable, but in addition are considered unfair and unjust”.3 Age-standardised rates of injury death and hospitalisation in New Zealand are consistently higher for Māori than non-Māori.4 There is evidence of inequities, particularly in access, timeliness and quality of care by ethnicity, across a range of healthcare pathways4–7 including access to rehabilitation services.8 Inequities along trauma and other care pathways are sensitive indicators of unwarranted variation in quality of care, and therefore reveal opportunities for quality improvement initiatives for improving outcomes for all as well as eliminating inequities.9 The resulting equity-focused interventions have potential for significant health gain and healthcare cost savings. However, availability of reliable, high-quality ethnicity data is essential if healthcare inequities are to be identified and the impact of interventions monitored. Reliable ethnicity data is also a prerequisite for fulfilling Māori rights to be counted and to monitor the government’s obligations to ensure the realisation of rights to health for all its citizens.3

Statistics New Zealand has defined the meaning of ethnicity for official statistics, acknowledging that it is self-identified and people can belong to more than one ethnic group.10 Achieving the long-standing goal of high-quality ethnicity data is a complex journey with much of the work to date being led by Māori researchers, Te Rōpū Rangahau Hauora a Eru Pōmare, and by (the recently disestablished) Te Kete Hauora, Māori Health Business Unit at New Zealand’s Ministry of Health.13 In most New Zealand healthcare settings, the official approach to ethnicity data collection should be self-identification using the New Zealand Census question (Figure 1), which allows respondents to self-identify with one or more ethnic groups. To support quality ethnicity data, the Ministry of Health’s Ethnicity Data Protocols11 describe standard procedures for collecting, recording and using ethnicity data. These protocols were published in 2004, updated in 2009 and revised in 2017.12,13

Figure 1: The New Zealand Census 2013 ethnicity question.

c

Source: Ministry of Health; Ethnicity Data Protocols.12 

Primary care and hospital data provide important information on the prevalence and incidence of disease, access to care and medications, as well as providing crucial information on timeliness and quality of care. Despite having both good-quality census denominator data and national mortality numerator data, challenges remain with the quality of morbidity numerator data from hospital and primary care sources.13,14 Multiple audits over the last 20 years have consistently shown that hospital ethnicity data are not accurate, resulting in unreliable information and underestimations of Māori hospitalisations.13 Similar problems have been identified with the quality of primary care ethnicity data.1,13,15

In 2013, Te Kete Hauora commissioned and published an ‘Ethnicity Data Audit Toolkit’.16 An adapted interim toolkit for use in hospital settings has since been developed, which provides tools to (1) assess the quality of ethnicity data and systems for data collection; (2) standardise recording and output within hospital settings; and (3) provide guidance on quality improvement activities.

The Waikato Trauma Registry was established in 2006, and contains individual level demographic, clinical and resource utilisation data for trauma patients from point of injury to definitive intervention.17 A Trauma Registry form is completed by trained trauma nurses and doctors for each patient at admission, or as soon as possible after admission. Data are collected from the patient as well as from clinical notes. The form captures information on: (1) demographics (age, gender and ethnicity); (2) how and where the injury occurred; (3) injuries sustained; and (4) processes of care.

Waikato Hospital is a provisional level 1 trauma centre, and one of five hospitals in the region that contribute data to the Registry. Waikato District Health Board (DHB) serves a population of almost 400,000 people, 84,900 (23%) of whom are Māori.18 Waikato Hospital provides tertiary services for the Midland region comprising five DHBs (Waikato, Bay of Plenty, Lakes, Tairawhiti and Taranaki).

Establishing processes to ensure that high-quality ethnicity data are captured by the Registry is an important foundational step as the service begins to analyse and report on the rich data resource, enabling a confident assessment of inequities in trauma incidence and care. This important work will serve as a foundation for the development, implementation and monitoring of a range of interventions to contribute towards improving Māori health and achieving equity in trauma care, as well as improving the trauma pathway for all.

The aim of this study was to assess the accuracy of ethnicity data in Trauma Registry records in Waikato Hospital using the interim Hospital Ethnicity Data Audit Toolkit. A secondary aim was to audit the quality of ethnicity data captured by the Waikato Hospital patient management system.

Method

The study used methods described in ‘Stage 4’ of the draft ‘Hospital Ethnicity Data Audit Toolkit 2014’. The audit team were granted permission by Dr Donna Cormack, the designer of the toolkit, to use the draft hospital Toolkit as part of their audit. During a three-month period in 2015, consecutive adult (16 years of age and older) trauma patients admitted to Waikato Hospital and recorded in the Waikato Trauma Registry were invited to take part in the study until a sample of 100 patients was achieved. A sample size of 100 is recommended by the Toolkit guideline as an initial step to assess major issues with ethnicity data quality. All participants participated while they were still in hospital and provided informed consent. A single researcher, of self-identified New Zealand European ethnicity, approached and interviewed all patients, provided information about the study and obtained written informed consent to take part in the study. The researcher was blind to the ethnicity coding for the patient in the Trauma Registry and hospital patient management database.

At the time of the study the registry had space for only one ethnicity to be recorded, however this was increased to two spaces in June 2015. The minimum requirements in the Ministry of Health’s current Ethnicity Data Protocols are for up to three ethnicities to be recorded for each person.12

Each participant was shown a questionnaire containing the New Zealand Census 2013 question “Which ethnic group do you belong to? Mark the space or spaces that apply to you.” (Figure 1). The participant then indicated their self-identified ethnicity by ticking the relevant option(s). If they selected the ‘Other’ category, they were prompted to specify the other ethnic group(s) they identified with. Information was also collected on age, gender and circumstances of injury. Information on the patient’s ethnic group(s) as recorded in the electronic hospital patient management database and Trauma Registry data collection paper form was subsequently collected and recorded by the research assistant.

Ethical approval for the study was obtained from the University of Auckland Human Participants Ethics Committee (Reference: 010706) and the Waikato Hospital Māori Research Review Committee (Reference: RD014118).

Analysis

Information on participants’ self-identified ethnic group(s) recorded on the audit form was compared with the ethnicity data recorded on the paper-based Trauma Registry forms as well as those recorded in the hospital patient management database. Most ethnicity data from the hospital patient management database was sourced from a computer database, with paper records only checked when accessible. All comparisons were made with the ethnicity recorded on the date of admission.

Using the method recommended in the interim Hospital Ethnicity Data Audit Toolkit, a match was determined when either the hospital patient management database or the Trauma Registry form listed the same ethnicity/ies as the self-identified ethnicity audit form. Partial matches occurred when some, but not all ethnicities matched, and a total mismatch was when none of the persons self-identified ethnicities or their only ethnicity was not recorded on the comparison form (Table 1). Using this method, matches, partial matches and total mismatches were calculated across the records for all 100 participants. Data were prioritised according to the Ministry of Health Ethnicity Data Protocols.12

Table 1: Example of classification of match, partial match and total mismatch data.

 

Self-identified ethnicity from audit form

Ethnicity on administrative record

 

Patient 1

Māori

Māori

Match

Patient 2

Māori/NZE

Māori

Partial match

Patient 3

Māori

NZE

Total mismatch

Data were analysed in Microsoft Excel and SPSS to generate descriptive statistics. Chi-square tests were used to detect any significant differences between cohorts in terms of concordance between self-identified ethnicity and either the hospital patient management database, or the trauma record. Effect sizes (the magnitude of the difference between the two groups) was calculated using Cramer’s V/phi.

Results

Of the 102 patients invited to take part in the study during the three-month period, 100 (98%) agreed to participate. More than half self-identified as New Zealand European (n=57/100), and 29% self-identified as Māori (Table 2). Most (n=18/29) self-identified Māori reported multiple ethnic groups compared to 4% of non-Māori (n=3/71).
Of the 29 people who self-identified as Māori, 23 were classified as Māori in the Trauma Registry and 24 in the hospital patient management database. Six of those who self-identified as Māori were recorded as New Zealand European in the Trauma Registry, and of these, five were also recorded as New Zealand European on the hospital patient management database. Hence the Māori undercount was 21% for the Trauma Registry and 17% for the hospital patient management database. Four of these individuals self-identified with multiple ethnic groups while two self-identified as Māori only. One patient who self-identified as New Zealand European was recorded as Māori in both the hospital patient management database and in the Trauma registry. Those who self-identified as Māori were more likely to be undercounted in the Trauma Registry than non-Māori participants. (Table 3).

Table 2: Proportion of trauma patients interviewed who self-identified with more than one ethnic group.

Self-identified ethnicity (coded to level 2# and not prioritised)

 Total

(n=120)

More than one ethnic group reported

New Zealand European/Pākehā

72

15/72

Māori

29

18/29

Samoan

4

3/4

Cook Island Māori

4

4/4

Chinese

1

0/1

Other European

7

1/7

Tokelauan

1

0/1

Other Pacific peoples

1

0/1

South East Asian

1

0/1

#Ethnic group code description for level 2; Māori, Tokelauan, Fijian, Niuean, Tongan, Cook Island Māori, Samoan, Other Pacific Island, Pacific Island NFD*, South East Asian, Indian, Chinese, Other Asian, Asian NFD, Latin American/Hispanic, African, Middle Eastern, Other, Other European, European NFD, NZ European. *Not Further Defined.
Note n= >100 because some people self-identified as having more than one ethnicity and are reported in each group they identified with.

Table 3: Total Māori (everyone who identified as Māori) and Non-Māori (coded to level 2), where self-identified ethnicity was different to recorded ethnicity (undercount).

 

Self-identified Māori (n=29)

Self-identified Non-Māori (n=71)

 p

Hospital patient management database

5 (17%)

10 (14%)

.414

Trauma Registry

6 (21%)

3 (4%)

.029

For those who self-identified as Māori (either as Māori only or as Māori plus at least one other ethnicity), the percentage concordance between the three records showed a 24% agreement between self-identified ethnicity versus the ethnicity recorded in the Trauma Registry form, and a 41% agreement with ethnicity recorded in the hospital patient management database (Figure 2). The equivalent for the New Zealand European cohort was 97% and 86%, respectively. Concordance was higher for those who self-identified as Māori only, as opposed to those who self-identified as Māori plus at least one other ethnicity (Table 4).

Table 4: Comparison of patients’ ethnicity across three sources.

 

Match

n

Partial match

n

Total mismatch

n

Self-identified ethnicity vs Trauma Registry form

Māori only

7/11

3/11

1/11

Māori and at least one other ethnicity

0/18

18/18

0/18

Non-Māori

65/71

4/71

2/71

Self-identified ethnicity vs hospital patient management database

Māori

7/11

2/11

2/11

Māori and at least one other ethnicity

5/18

12/18

1/18

Non-Māori

57/71

3/71

10/71

Trauma form vs hospital patient management database

Māori

9/11

1/11

1/11

Māori and at least one other ethnicity#

12/18

5/18

1/18

Non-Māori

62/71

2/71

7/71

#12 patients who were identified as Māori on the Trauma Registry form also had another ethnicity listed on the Trauma Registry form. 

When data were prioritised, partial matches of self-identified ethnicity (ie, where some but not all ethnicities matched) with the Trauma Registry form and the hospital patient management database for those identifying as Māori were 72% and 48%, respectively (Figure 2). The proportions for New Zealand European were 97% and 86%, respectively. In the majority of cases, partial matches were associated with multiple self-identified ethnicities not being recorded in either the hospital patient management database or the Trauma Registry form. Total mismatch rates were greater between self-identified ethnicity and the hospital patient management database (Māori 10%; New Zealand European 14%) than the Trauma Registry form for both groups (3% each) (Figure 2).

Figure 2: Comparison of self-reported ethnicity with the hospital patient management database and Trauma Registry form for prioritised patients.

c 

Significant differences for the prioritised data between the Māori and non-Māori cohorts were found for both the self-identified ethnicity compared with the hospital patient management database (χ2 (2, 100)=28.643, p<.001), with a large effect size (φ=0.54); as well as the self-identified ethnicity compared with the Trauma Registry form (χ2 (2, 100)=49.752, p<.001), also with a large effect size (φ=0.71).

The hospital patient management database compared with the Trauma Registry form recorded ethnicity showed significant differences between the cohorts (χ2 (2, 100)=8.974, p=.011), however the effect size was moderate (φ=0.30).

Discussion

To our knowledge this is the first published investigation of the accuracy of ethnicity data in both a trauma registry and a patient management database in New Zealand. By comparing patients’ self-identified ethnicity with their ethnicity recorded on administrative records, we found that one-fifth (21%), and one-quarter (24%) of trauma patients who self-identified as Māori were classified as New Zealand European in the Trauma Registry and hospital patient management database, respectively. We found a high degree of undercount and mismatch for Māori but not for non-Māori patients.

The significant undercount of people of Māori ethnicity found for the hospital patient management system could be reflective of ethnicity capture quality issues nationally. Ethnicity data quality issues at the DHB level also have flow-on impacts for national collections, as DHBs upload their local data to national databases such as the National Minimum Dataset (NMDS). As well as providing important information on inequities in access and quality of care, hospital data is the basis for calculating the incidence of major causes of illness such as cardiovascular disease, renal failure and mental illness. National prioritisation and planning for future health budgeting is based on consideration of such data, hence the need for high-quality ethnicity data. The analysis, reporting and interpretation of Trauma Registry and national morbidity data need to take into consideration the large undercounts of Māori found in this audit.

For the Trauma Registry and Waikato Hospital patient management database, the Māori undercount means that the Māori incidence of trauma is underestimated. In addition, inequities along the trauma care pathway and inequities in outcomes from trauma care are not able to be measured accurately for Māori. Interventions to eliminate inequities in care are less likely to be developed if inequities are not first identified—resulting in a loss of potential health gain from improvements to the trauma care pathway and in trauma prevention, especially for Māori.

Over 80% of those who were documented as Māori in the Trauma Registry were also documented as Māori on the hospital patient management database. However, as we have illustrated, ethnicity recorded on both databases had a very low concordance with patients’ self-identified ethnicity for those self-identifying as Māori. Partial matches occurred far more frequently in the Māori cohort.

There was 21% concordance with Trauma Registry ethnicity for those who self-identified as Māori—with much of this likely due to only one ethnicity being captured on the Trauma Registry form. The partial mismatch rate of over 70% for self-identified Māori against the Trauma Registry is likely to be due to the same issue. Total mismatch rates (for those who self-identified as Māori), were lower in the Trauma Registry (3%) than the hospital patient management database (10%). This could be because trauma staff ask patients the ethnicity question directly.

In an effort to adjust for undercounting of Māori data, methods have been developed to account for undercount and improve estimates. One approach is the ‘ever-Māori’ method where multiple data entry points are compared for an individual person who is then recorded as Māori if they had been classified as Māori in any of the matched databases.19,20 However, these methods require additional time and expertise and do not improve the quality of ethnicity data at source. It is more efficient and sustainable to improve the quality of data at the point of collection and recording than to retrospectively apply adjusters to improve estimates.21

This study lacked sufficient power to examine the three levels of ethnicity matching independently (match, partial match and total mismatch). The sample size was based on the toolkit guidelines and designed to identify major issues with quality as a first step, rather than aiming to get statistically significant results. This audit suggests the likelihood of substantial and systematic errors in ethnicity classification in the databases studied. The extent to which these issues are mirrored in other health databases deserves scrutiny. We recommend that the best use of resources would be to train staff, put in place standardised policies and practices around data collection and conducting future audits to see if quality has improved.

In the majority of calculations, non-Māori was used as a comparator with Māori instead of NZE. This is because of an interest in understanding the quality of data for Māori given the known patterns of health need and inequity. As Māori in this study often self-identified with more than one ethnic group in addition to Māori, non-Māori provided a non-overlapping comparator group. The small numbers of groups other than Māori and NZE constrained our ability to look in detail at undercounts for other ethnic groups. However, the identification of systemic issues with ethnicity data quality will likely have positive impacts for the quality of data for other ethnic groups as well.

The interim Hospital Ethnicity Data Toolkit includes a four-stage audit process; however, this study was limited to auditing the quality of ethnicity data and did not include an audit of the process and quality of systems for ethnicity data collection, recording and output, or of staff levels of knowledge and training. The audit team found the Audit Toolkit easy to understand and the method simple to follow. Data sources reviewed were complete and because many patients were very unwell, with many having suffered life threatening injuries, the 98% response rate was higher than expected.

This study has implications for the analysis of retrospective data and also for ongoing ethnicity data collection systems and future analyses. It also speaks to the need for regular ethnicity data audits and embedding of the Ministry of Health’s Ethnicity Data Protocol in policy and practice.12 Given the recent release of updated protocols, there is a current opportunity to improve future ethnicity data quality by fully implementing the revised protocols.

Conclusions

This audit confirms known problems with the use of non-standard ethnicity questions for data collection in the health sector and resulting poor-quality data. More particularly, the study also demonstrates the adverse differential impact of poor ethnicity data quality for Māori data.

A fundamental prerequisite for achieving health equity is the ability to first identify inequities in rates of death or disease, and the points of particular disease and care pathway where inequities occur. High-quality ethnicity data are also needed so that progress towards (or away from) equity goals can be closely monitored and reported.22 This study has demonstrated that there is an urgent need for quality collecting, monitoring and reporting of ethnicity data in order to hold health systems accountable for current health inequities and to guide efforts to implement the necessary steps for achieving equity.

Summary

Eliminating unfair differences in life expectancy between Māori and non-Māori is something the New Zealand government has signed up to. Part of this involves fixing unwarranted variations in healthcare that exist across a range of pathways. For this, we need good-quality ethnicity data to figure out where healthcare needs improving. We found that one-quarter (24%) of trauma patients who self-identified as Māori were misclassified as New Zealand European. We did this by asking patients to self-identify their ethnicity using the census ethnicity question. This is important because hospital data is used for national planning and budgeting and we need good-quality ethnicity data to guide efforts to implement the necessary steps for achieving equity.

Abstract

Māori are disproportionately impacted by injury in New Zealand, therefore reliable ethnicity data are essential for measuring and addressing inequities in trauma incidence, care and outcomes.

Aim

To audit the quality of ethnicity data captured by the Waikato Hospital Trauma Registry and Waikato Hospital patient management system against self-identified ethnicity.

Method

Self-identified ethnicity using the New Zealand Census ethnicity question was gathered from 100 consecutive trauma patients and compared with ethnicity recorded in their Trauma Registry record and in the hospital’s patient management database.

Results

Twenty-nine (29%) participants self-identified as Māori, of whom six were classified as New Zealand European (NZE) only in the Trauma Registry and five as NZE on the hospital patient management database. Over half of Māori (n=18/29) reported more than one ethnicity compared with 4% (n=3/71) of non-Māori. Self-identified ethnicity matched Trauma Registry ethnicity for one quarter (n=7/29) of Māori versus 9% of non-Māori.

Conclusion

The degree of misclassification of Māori ethnicity data among patients in the Waikato Trauma Registry and the Waikato Hospital patient management system highlights a need for improvements to how ethnicity data is captured within these databases and potentially many other similar entities collecting ethnicity data in New Zealand. The release of revised standardised protocols for the collection of ethnicity data is timely given the recent establishment of a national trauma registry. Without quality data, the opportunity to investigate and address ethnic inequities in trauma incidence and management is greatly compromised.

Author Information

Nina Scott, Public Health Physician, Waikato District Health Board Hamilton;
Helen Clark, Medical Education Officer, Clinical Education and Training Unit; Waikato District Health Board, Hamilton; Bridget Kool, Senior Lecturer (Epidemiology and Biostatics), School of Population Health, The University of Auckland, Auckland; Shanthi Ameratunga, Professor of Epidemiology, School of Population Health, The University of Auckland, Auckland; Grant Christey, Trauma Director, Waikato District Health Board Hamilton; Donna Cormack, Senior Lecturer, Te Kupenga Hauora Māori, Faculty of Medical and Health Sciences, The University of Auckland, Auckland. 

Correspondence

Nina Scott, Waikato Hospital, Pembroke Street, Private Bag 3200, Hamilton 3240.

Correspondence Email

nina.scott@waikatodhb.health.nz

Competing Interests

Dr Cormack reports grants from the Ministry of Health during the conduct of the study.

References

  1. Creamer G, Civil I, Ng A, Adams D, Cacala S, Koelmeyer T, et al. Ethnicity of severe trauma patients: results of a population based study, Auckland, New Zealand 2004. New Zealand Medical Journal 2010; 123(1316):1–7.
  2. Gulliver P, JC Simpson J. Fact Sheet 38: Injury as a leading cause of death and hospitalisation. Dunedin: Injury Prevention Research Unit, 2007.
  3. Whitehead M. The concepts and principles of equity and health. International Journal of Health Services 1992; 22(3):429–45.
  4. Robson B, Harris R. Hauora: Maori Standards of Health IV. A study of the years 2000–2005. Wellington: Te Ropu Rangahau a eru Pomare, 2007.
  5. Ameratunga S, Abel S, Tin Tin S, Asiasiga L, Milne S, Crengle S. Children admitted to hospital following unintentional injury: perspectives of health service providers in Aotearoa/New Zealand. BMC Health Services Research 2010; 10(333):1–9.
  6. Arlidge B, Abel S, Asiasiga L, Milne S, Crengle S, Ameratunga S. Experiences of whānau/families when injured children are admitted to hospital: a multi-ethnic qualitative study from Aotearoa/New Zealand. Ethnicity & Health. 2009; 14(2):169–83.
  7. Hosking J, Ameratunga S, Bramley D, Crengle S. Reducing ethnic disparities in the quality of trauma care: an important research gap. Annals of Surgery. 2011; 253(2):233–37.
  8. McNaughton H, McRae A, Green G, Abernethy G, Gommans J. Stroke rehabilitation services in New Zealand: a survey of service configuration, capacity and guideline adherence. New Zealand Medical Journal 2014; 127(1402).
  9. Scott N, Lawrenson R. Potential for health gain equity. New Zealand Medical Journal. 2015; 128(1415):10.
  10. Statistics New Zealand. Statistical Standard for Ethnicity 2005. Wellington: Statistics New Zealand, 2005.
  11. Ministry of Health. Ethnicity data protocols for the health and disability sector. Wellington: Ministry of Health, 2004.
  12. Ministry of Health. HISO 10001:2017 Ethnicity Data Protocols. Wellington, New Zealand: Ministry of Health, 2017.
  13. Cormack D, McLeod M. Improving and Maintaining Quality in Ethnicity Data Collection: Issues for the Health and Disability Sector. Wellington: Te Rōpū Rangahau Hauora a Eru Pōmare, 2010.
  14. Kilgour R, Keefe V. Kia piki te ora: the collection of Māori health statistics. Wellington: Department of Health, 1992.
  15. Sandiford P, Salvetto M, Bramley D, Wong S, Johnson L. The effect of Māori ethnicity misclassification on cervical screening coverage. New Zealand Medical Journal 2013; 126(1372):55–65.
  16. Ministry of Health. Primary Care Ethnicity Data Audit Toolkit: A toolkit for assessing ethnicity data quality. Wellington: Ministry of Health, 2013.
  17. Midland Regional Trauma System. Midland Regional Trauma System Annual Report. 2013–2014. ISBN. 1-877296-25-2. Waikato: Waikato District Health Board, 2014.
  18. Robson B, Purdie G, Simmonds S, Waa A, Brownlee G, Rameka R. Waikato District Health Board Māori Health Profile 2015. Wellington: Te Rōpū Rangahau Hauora a Eru Pōmare, 2015.
  19. Harris R, Curtis E, Robson B, et al. Disparities by Deprivation: understanding the relationship. A review of Maori:non-Maori disparities in caesarean sections. Wellington: Te Ropu Rangahau Hauora a Eru Pomare, 2003.
  20. Gulliver PJ, Cryer C, Langley JD, Davie DS. Identifying Māori ethnicity for estimating trends in fatal and serious non-fatal injury. Australian and New Zealand Journal of Public Health 2011; 35(4):352–56.
  21. Cormack D, Harris R. Issues in monitoring Māori health and ethnic disparities: an update. Wellington, New Zealand: Te Rōpū Rangahau Hauora a Eru Pōmare, 2009.
  22. Wyatt R, Laderman M, Botwinick L, Mate K, Whittington J. Achieving Health Equity: A Guide for Health Care Organizations. IHI White Paper. Cambridge, Massachusetts: Institute for Healthcare Improvement, 2016.