26th October 2012, Volume 125 Number 1364

Matire Harwood, Mark Weatherall, Api Talemaitoga, P Alan Barber, John Gommans, William Taylor, Kathryn McPherson, Harry McNaughton

New Zealand Maori have consistently been shown to have worse outcomes than European New Zealanders over a range of health conditions, including stroke, using various outcome measures which have generally been validated in European populations.1–3

Health outcomes research for Maori has been consistently hampered by the lack of outcome instruments that reflect issues important to Maori and conceived from a Maori perspective of health.4

The Hua Oranga (translated literally as 'the fruits of health') outcome tool was developed as a means of assessing outcomes after interventions for Maori people with mental illness.4 It is based on a holistic Maori conception of well-being Te Whare Tapa Wha, and considers each of the four 'pillars' of well-being; taha wairua ('spiritual'), taha hinengaro ('mental'), taha tinana ('physical'), taha whanau ('family').

Originally the tool was planned to be used by the patient/client, their whanau (family) and clinicians, with scores from each being accumulated into a single score for that person.4 There are few published studies using Hua Oranga as an outcome measure and all of these relate to Maori with mental illness.5–7 There is no particular reason why the tool, if psychometrically sound and valid, could not be used in other health conditions given the centrality of its four core components to health and well-being. Further, the tool might apply equally well to Pacific people. However, currently basic evaluation of the instrument’s psychometric properties is lacking.

We completed an intervention study designed to facilitate self-directed rehabilitation after stroke in Maori and Pacific people.8 Hua Oranga was used as one of the secondary outcome measures for participants and their whanau. This allowed a comparison of the performance of this instrument against other measures, and also to explore some of the psychometric properties of the instrument in a sizeable cohort of Maori and Pacific people.

The study interventions were based on previous qualitative work,9 and we hypothesised that improvement for Maori and possibly Pacific people following stroke would involve a strengthening in taha wairua (~ spiritual health) and taha whanau (~ family health and connections), something hard to capture with conventional instruments such as the Mental Component Summary (MCS) and Physical Component Summary (PCS) scores of the Short Form 36 (SF36).10 Consultation with Pacific people with stroke, their carers and health providers prior to the study confirmed a sense that the instrument fitted well with a Pacific view of health and well-being as much as it did for Maori.

We hoped that use of Hua Oranga would enable the exploration of these less conventional aspects of improvement for the participants, otherwise inaccessible in a quantitative study.

We present the results for the Hua Oranga outcome tool by intervention, compare this tool with the primary outcome measures (PCS and MCS) and use factor analysis to explore psychometric properties of the instrument.

Method

The overall design and methodology are presented in full elsewhere.8 Briefly, this was a randomised controlled study of two different interventions aimed at promoting self-directed rehabilitation for Maori and Pacific people, 15 years and older, within three months of stroke and living in the community. Participants were randomised in a 2x2 factorial design to receive one, both or neither of two interventions.
  1. ‘Inspirational’ DVD—80 minute professionally produced DVD about stroke and stroke recovery using the inspirational stories told by four Maori and Pacific people and their families. The dominant messages were the potential for good outcomes, overcoming adversity, personal and family roles and their contribution to recovery, encouraging meaningful activity and participation for the person with stroke, and where to access resources for people following stroke. The DVD was left with the person and they were encouraged to view it as many times as they wished.
  2. ‘Take charge’ session (TCS)—an 80 minuteindividualised assessment with a structured risk factor and activities of daily living assessment designed to engage the patient and their family in the process of recovery, facilitating a process where they identified for themselves areas where they could make progress and set personal goals i.e. self-directed rehabilitation. No direct therapy or formal goal-setting occurred.

Both interventions were delivered by research assistants of the same ethnicity as the participant. All research assistants had a minimum of 5 days training prior to starting the study and ongoing training days during the study. The control group received written material about stroke for people and their families delivered in person by a trained research assistant of the same ethnic group as the stroke person.

Primary outcome was self-rated health related quality of life (QoL) at 12 months following randomisation measured using the PCS and MCS on the SF36.10 Secondary outcomes were the Hua Oranga score for participants and carers measured at 12 months, activities of daily living (ADL) measured by the Barthel Index (BI),11 instrumental activities of daily living (IADL) measured by Frenchay Activities Index (FAI),12 Carer Strain Index (CSI),13 dependence (modified Rankin score14 [mRS]>2), and use of rehabilitation services.

Hua Oranga scores were not presented in the primary study report8 as it is a novel measure of uncertain validity in this context. To compute a score with the Hua Oranga instrument, the participant answers four questions from each of the four domains (taha wairua, taha hinengaro, taha tinana, taha whanau; respectively spiritual, mental, physical and family dimensions) with a general format of:

'As a result of the intervention do you feel ____ (eg 'healthier from a spiritual point of view')?'

The possible answers are scored 'much worse' (-2), 'worse' (-1), no change (0), better (+1), much better (+2), giving a summed score range for the 16 questions of -32 to +32. The questions for the carer have the general format:

'Has the intervention resulted in an improved ____(eg spiritual health) for your relative?'

Scoring is the same with a range of -32 to +32. Modification to the wording was made for Pacific people, such as substituting 'Pacific person' for 'Maori'. Subsequent to the present study, a four question version of Hua Oranga has also been studied15 (see 'Discussion' section).

Analysis of variance was used to compare the effects of the two treatments, DVD and TCS for continuous outcome variables. Plots and correlation coefficients were used to explore the association between the total Hua Oranga score and the MCS and PCS of the SF-36. Simple linear regression was used to estimate the change in Hua Oranga total score corresponding to a 10 unit change on the MCS and PCS.

Plots and correlation coefficients were used to explore the associations between the pre-nominated dimensions of Hua Oranga score, and between these dimensions and the eight dimensions of the SF-36. Principal components analysis with a scree plot was used to explore the structure of the four dimensional construct of the Hua Oranga tool and the 16 questions of the instrument.

To determine a possible number of underlying factors for the Hua Oranga tool, a scree plot of the eigenvalues of the principal components analysis was used. The number of factors is suggested by where the scree plot undergoes an abrupt change in slope, but also by the number of eigenvalues greater than one. If an eigenvalue is less than one this suggests that the particular linear combination of Hua Oranga instrument dimensions or questions explains less of the variance than one single dimension or question.

Results

172 participants, 94 Maori and 78 Pacific people, were randomised. The baseline characteristics of the participants are presented in the primary publication from the study.8 139 participants (80.8%) completed follow-up at 12 months after randomisation. Of these, 135 (97%) completed the Hua Oranga and 117 (84.2%) completed the SF36. Eighty-nine carers completed the Hua Oranga. See Table 1.

The Hua Oranga instrument was sensitive to change: Hua Oranga total scores were higher for the TCS (main effect 5.3 (95% CI 1.7 to 8.8), p=0.004) but not for the DVD (main effect 3.0 (95% CI -0.6 to 6.5), p=0.10). The TCS but not the DVD was associated with significant change in both PCS on the SF36 and dependence on the mRS. The Hua Oranga scores for carers were higher in both the TCS (main effect 5.1 (95% CI 1.3 to 9.0), p=0.01) and DVD (main effect 6.4 (95% CI 2.5 to 10.2), p=0.005) groups.

There was a moderate relationship between the total score of the Hua Oranga instrument and the PCS of the SF-36 (correlation coefficient 0.55, p<0.001; see Figure 1 for scatter plot), but only a weak relationship with the MCS (correlation coefficient 0.31, p<0.001). A 10 point change on the SF-36 PCS was associated with a 5.4 (95% CI 3.9 to 6.9) point change on the Hua Oranga total score.

Figure 1. Scree plot total Hua Oranga score vs physical component summary score (PCS) of the SF-36

Harwood-1

Table 2. Association between individual question sections of Hua Oranga and individual dimensions of the SF-36 at 12 months

HO: Wairua (Spiritual)
HO: Hinengaro
(Mental)
HO: Tinana
(Physical)
HO: Whanau
(Family)
SF-36 element
Correlation coefficient (N with both data)
Physical functioning
0.37 (133)
0.34 (136)
0.50 (135)
0.29 (136)
Role-physical
0.44 (129)
0.40 (132)
0.43 (131)
0.35 (132)
Bodily pain
0.46 (132)
0.28 (135)
0.42 (134)
0.38 (135)
General health
0.57 (128)
0.54 (131)
0.67 (130)
0.57 (131)
Vitality
0.55 (128)
0.50 (131)
0.41 (130)
0.39 (131)
Social functioning
0.16 (131)
0.11 (134)
0.08 (0.36)
0.20 (134)
Role-emotional
0.40 (131)
0.38 (133)
0.36 (132)
0.33 (133)
Mental health
0.44 (130)
0.42 (133)
0.30 (132)
0.37 (133)





PCS
0.52 (114)
0.42 (116)
0.64 (115)
0.44 (116)
MCS
0.33 (114)
0.34 (116)
0.13 (115)
0.30 (116)
HO=Hua Oranga, SF36=Short Form 36, PCS=Physical Component Summary of the SF36, MCS=Mental Component Summary of the SF36.

Individual dimensions of the Hua Oranga were most strongly associated with the General Health and Vitality dimensions of the SF 36 (Table 2). PCS score correlated most strongly with the physical (correlation coefficient 0.62) and spiritual (correlation coefficient 0.52) dimensions of the Hua Oranga. Individual dimensions of the Hua Oranga were strongly associated with each other (Table 3, correlation coefficients between 0.74 and 0.82).

Table 3. Association of dimension totals for Hua Oranga instrument with each other at 12 months
Variables
HO: Wairua (Spiritual)
HO: Hinengaro
(Mental)
HO: Tinana
(Physical)
HO: Whanau
(Family)
Correlation coefficient (N with both data)
HO: Wairua (Spiritual)
1 (135)
0.79 (135)
0.75 (135)
0.82 (135)
HO: Hinengaro
(Mental)

1 (138)
0.75 (137)
0.76 (138)
HO: Tinana
(Physical)


1 (137)
0.74 (137)
HO: Whanau
(Family)



1 (138)

Table 4 shows the principal components analysis of the four dimensions of the Hua Oranga. With only one of the eigenvalues greater than one, an underlying factor structure is not supported. However, Table 5 shows that a two factor structure is suggested by the factor analysis of the 16 component questions. These two factors have a physical-mental health component and a spiritual-family health component (Table 6). For this spiritual-family health component there was not a strong relationship with the equivalent SF-36 dimensions i.e. 'social functioning' and 'role-emotional' (Table 2). This suggests that the spiritual-family dimensions on the Hua Oranga instrument may be capturing quality of life issues not captured in the SF-36.

Table 4. Principal components values and scree plots for 4 dimensions of Hua Oranga
Eigenvalue number
Eigenvalue
Proportion of variance (%)
Cumulative proportion of variance (%)
1
2
3
4
3.3
0.28
0.24
0.17
82.6
7.0
6.1
4.3
82.6
89.6
95.7
100

Harwood-2

Table 5. Principal components values and scree plots for 16 questions of Hua Oranga
Eigenvalue number (first four only)
Eigenvalue
Proportion of variance (%)
Cumulative proportion of variance (%)
1
2
3
4
10.3
1.2
0.9
0.7
64.2
7.6
5.6
4.4
64.2
71.8
77.4
81.8

Harwood-3

Table 6. Factor loadings using maximum likelihood with varimax rotation with two factors specified
Hua Oranga Question
Factor 1 loading
Factor 2 loading
Communality




Wairua (Spiritual)



Content
0.65
0.51
0.69
Stronger
0.55
0.59
0.66
Spiritually healthier
0.44
0.71
0.69
Valued
0.58
0.57
0.67




Hinengaro (Mental)



Goals
0.77
0.17
0.62
Manage feelings
0.51
0.60
0.62
Positive
0.69
0.51
0.73
Understand Health
0.73
0.40
0.69




Tinana (Physical)



Healthier
0.71
0.44
0.69
Mental Wellbeing
0.68
0.32
0.57
Move
0.62
0.32
0.48
Physical Health
0.78
0.36
0.74




Whanau (Family)



Clearer
0.20
0.89
0.83
Communicate
0.35
0.82
0.79
Community
0.65
0.44
0.61
Confident
0.52
0.69
0.75

Discussion

This study provided an opportunity to study the psychometric properties of the Hua Oranga outcomes instrument in a novel, sizeable population of people following stroke. The nature of the questions and the framing in terms of the four dimensions of Te Whare Tapa Wha provide Hua Oranga with good face validity if one accepts this model as a valid representation of Maori (and Pacific people’s) health and well-being. Support was provided for its sensitivity to change, both for patients and carers. External validity was provided by correlation of total Hua Oranga scores with PCS scores and significant change occurring in Hua Oranga scores for the same intervention (TCS) that significant change in PCS and dependency scores were seen.

Factor analysis suggested that the Hua Oranga measures two rather than four separate factors, one physical-mental and one spiritual-family. However, this may have been influenced by the strong 'physical' nature of stroke recovery and may be different in a population with different health problems, such as mental health. There was some evidence that the spiritual-family factor was measuring something different to the equivalent dimensions of the SF-36. Utility of the measure was good with participants having little trouble completing the questions, generally in less than half the time it took to complete the SF-36 and with higher completion rates. Little cost is involved in use of the measure.

A significant issue with the measure relates to the wording of each question. The general question stem 'As a result of the intervention do you feel ...e.g. healthier from a spiritual point of view' asks the subject to both describe a change (choices of 'much more' to 'much less') and attribute this change to the intervention.

The measures against which we have compared Hua Oranga do not have this 'change' element and the validity of that comparison could be questioned. This has been addressed in a recent study15 which trialled the measure in 43 subjects with mental health problems as well as their clinicians and whanau. Two versions of the questionnaire were used with the first option being the one used in this study.

The second option reformulated the questions as statements to indicate how the person felt now and avoided mentioning an intervention, thus under the 'Wairua' category the options were from 'I feel that my spiritual health is extremely good at present' to 'I feel that my spiritual health is extremely bad at present'. This effectively condensed the instrument down to only four questions (one for each 'pillar') from the 16 questions used in the first option.

The second option was seen by participants as more acceptable and better correlations were seen between the responses of the subjects and their clinicians when the second option was used. It remains to be seen whether the 4-question version (option 2), although more acceptable, may be too limited to be useful.

Some caution is required in interpreting the results of our study. The study population comprised Maori and Pacific people with the instrument modified for Pacific participants. We chose not to analyse the Hua Oranga results separately in order to maximise the available information. Missing data mainly related to participants with communication difficulties who, if they had been able to respond, may have responded in a consistently different way to the questions than people with normal communication.

Overall, however, the Hua Oranga appears to have much to offer in Maori health outcomes research. Its simplicity, relative brevity, minimal cost, adequate psychometric properties should favour its use in future studies with both Maori and Pacific people. Its use in health conditions other than mental health and stroke could also be encouraged.

Our analysis suggests that results should be presented as total scores. If further subdivision is attempted, two scores - one summing physical and mental dimensions and one summing spiritual and family dimensions, but not each of the four pre-specified dimensions separately would be appropriate. It would be more conventional, and statistically more simple, to score each question 0–4 rather than -2 to +2 giving a total score between zero and 64.

A further study is required to test the instrument in a new population of subjects with items selected on the basis of the two-factor structure outlined here. Further work needs to be done on the questions themselves – perhaps transforming all 16 questions into statements and comparing this to the short version tested in option two of the McClintock study.15 In a further study, the 'spiritual-family' dimension of the Hua Oranga could be explored more fully, using complementary measures, to determine what is being measured by this part of the instrument.

Summary

The Hua Oranga instrument, developed for Maori people with mental illness, showed good responsiveness and adequate psychometric properties in Maori and Pacific people after stroke. Its simplicity, relative brevity, minimal cost and adequate psychometric properties should favour its use in future studies with both Maori and Pacific people. Suggestions are made for refinements to the measure. These should be tested in a new population before Hua Oranga is recommended for general use in a clinical setting.

Abstract

Aim

Health outcomes research for Maori has been hampered by the lack of adequately validated instruments that directly address outcomes of importance to Maori, framed by a Maori perspective of health. Hua Oranga is an outcome instrument developed for Maori with mental illness that uses a holistic view of Maori health to determine improvements in physical, mental, spiritual and family domains of health. Basic psychometric work for Hua Oranga is lacking. We sought to explore the psychometric properties of the instrument and compare its responsiveness alongside other, more established tools in an intervention study involving Maori and Pacific people following acute stroke.

Method

Randomised 2x2 controlled trial of Maori and Pacific people following acute stroke with two interventions aimed at facilitating self-directed rehabilitation, and with follow-up at 12 months after randomisation. Primary outcome measures were the Physical Component Summary (PCS) and Mental Component Summary (MCS) of the Short Form 36 (SF36) at 12 months. Hua Oranga was used as a secondary outcome measure for participants at 12 months and for carers and whanau (extended family). Psychometric properties of Hua Oranga were explored using plots and correlation coefficients, principal factors analysis and scree plots.

Results

172 participants were randomised, of whom 139 (80.8%) completed follow-up. Of these, 135 (97%) completed the Hua Oranga and 117 (84.2%) completed the PCS and MCS of the SF36. Eighty-nine carers completed the Hua Oranga. Total Hua Oranga scores and PCS improved significantly for one intervention group but not the other. Total Hua Oranga scores for carers improved significantly for both interventions. Total Hua Oranga score correlated moderately with the PCS (correlation coefficient 0.55, p<0.001). Factor analysis suggested that Hua Oranga measures two and not four factors; one 'physical-mental' and one 'spiritual-family'.

Conclusion

The Hua Oranga instrument, developed for Maori people with mental illness, showed good responsiveness and adequate psychometric properties in Maori and Pacific people after stroke. Its simplicity, relative brevity, minimal cost and adequate psychometric properties should favour its use in future studies with both Maori and Pacific people. Suggestions are made for refinements to the measure. These should be tested in a new population before Hua Oranga is recommended for general use in a clinical setting.

Author Information

Matire Harwood, Medical Research Institute of New Zealand, Wellington; Mark Weatherall, Professor, Wellington School of Medicine and Health Sciences, Wellington; Api Talemaitoga, Clinical Director, Pacific Health, Ministry of Health, Wellington; P Alan Barber, Professor, Centre for Brain Research, University of Auckland; John Gommans, Physician, Hawke’s Bay District Health Board, Hastings; William Taylor, Associate Professor, Wellington School of Medicine and Health Sciences, Wellington; Kathryn McPherson, Professor, Health and Rehabilitation Research Institute Auckland University of Technology, Auckland; Harry McNaughton, Medical Research Institute of New Zealand, Wellington

Acknowledgements

The study was funded by the Health Research Council of New Zealand and the B Basham Medical Charitable Trust. We also thank the Maori and Pacific Stroke Study participants and their families, team members, and all the staff at participating centres.

Correspondence

Dr Harry McNaughton, Medical Research Institute of New Zealand, Private Bag 7902, Wellington 6242, New Zealand. Fax: +64 (0)4 3895707

Correspondence Email

harry.mcnaughton@mrinz.ac.nz

Competing Interests

None known.

References

  1. Curtis E, Harwood M, Riddell T, et al. Access and society as determinants of ischaemic heart disease in indigenous populations. Heart Lung Circ. 2010;19:316–24.
  2. Hill S, Sarfati D, Blakely T, et al. Survival disparities in Indigenous and non-Indigenous New Zealanders with colon cancer: the role of patient comorbidity, treatment and health service factors. J Epidemiol Community Health. 2010;64:117–23.
  3. McNaughton H, Feigin V, Kerse N, et al. Ethnicity and functional outcome after stroke. Stroke. 2011;42:960–4.
  4. Durie M, Kingi TK. Hua Oranga: A Maori measure of mental health outcomes. Palmerston North, NZ: Massey University; 1999.
  5. Adamson S, Deering D, Moana-o-Hinerangi, et al. An Evaluation of the Moana House Residential Therapeutic Community. Alcohol Advisory Council of New Zealand; 2010.
  6. Kingi TK. Hua Oranga: Best outcomes for Maori. PhD thesis. Massey University; 2002.
  7. Bennett ST. Te Huanga o te ao Maori. Cognitive behavioural therapy for Maori clients with depression – development and evaluation of a culturally adapted treatment programme. Ph D thesis. Massey University; 2009.
  8. Harwood M, Weatherall M, Talemaitoga A, et al. Taking charge after stroke: promoting self-directed rehabilitation to improve quality of life – a randomized controlled trial. Clin Rehabil. 2011 Nov 15. [Epub ahead of print]
  9. Harwood M. Understanding and Improving Stroke Recovery for Maori and Their Whānau. PhD thesis University of Otago (submitted).
  10. Ware JE, Kosinski M. SF-36® Physical and Mental Health Summary Scales: A Manual for Users of Version 1. 2nd Ed. 2001 Lincoln, RI. QualityMetric, Inc.
  11. Mahoney FI, Barthel DW. Functional evaluation; the Barthel Index. Md State Med J. 1965;14:61–65.
  12. Wade DT, Legh-Smith J, Langton Hewer R. Social activities after stroke: measurement and natural history using the Frenchay Activities Index. Int Rehabil Med. 1985;7:176–181.
  13. Robinson BC. Validation of a caregiver strain index. J Gerontol 1983;38:344–48.
  14. van Swieten JC, Koudstaal PJ, Visser MC, et al. Interobserver agreement for the assessment of handicap in stroke patients Stroke. 1988;19:604–607.
  15. McClintock KK, Mellsop GW, Kingi TK. Development of a culturally attuned psychiatric outcome measure for an indigenous population. International Journal of Culture and Mental Health DOI:10.1080/17542863.2010.537484.