New Zealand Maori have consistently been shown to have worse outcomes than European New Zealanders over a range of health conditions, including stroke, using various outcome measures which have generally been validated in European populations.1–3
Health outcomes research for Maori has been consistently hampered by the lack of outcome instruments that reflect issues important to Maori and conceived from a Maori perspective of health.4
The Hua Oranga (translated literally as 'the fruits of health') outcome tool was developed as a means of assessing outcomes after interventions for Maori people with mental illness.4 It is based on a holistic Maori conception of well-being Te Whare Tapa Wha, and considers each of the four 'pillars' of well-being; taha wairua ('spiritual'), taha hinengaro ('mental'), taha tinana ('physical'), taha whanau ('family').
Originally the tool was planned to be used by the patient/client, their whanau (family) and clinicians, with scores from each being accumulated into a single score for that person.4 There are few published studies using Hua Oranga as an outcome measure and all of these relate to Maori with mental illness.5–7 There is no particular reason why the tool, if psychometrically sound and valid, could not be used in other health conditions given the centrality of its four core components to health and well-being. Further, the tool might apply equally well to Pacific people. However, currently basic evaluation of the instrument’s psychometric properties is lacking.
We completed an intervention study designed to facilitate self-directed rehabilitation after stroke in Maori and Pacific people.8 Hua Oranga was used as one of the secondary outcome measures for participants and their whanau. This allowed a comparison of the performance of this instrument against other measures, and also to explore some of the psychometric properties of the instrument in a sizeable cohort of Maori and Pacific people.
The study interventions were based on previous qualitative work,9 and we hypothesised that improvement for Maori and possibly Pacific people following stroke would involve a strengthening in taha wairua (~ spiritual health) and taha whanau (~ family health and connections), something hard to capture with conventional instruments such as the Mental Component Summary (MCS) and Physical Component Summary (PCS) scores of the Short Form 36 (SF36).10 Consultation with Pacific people with stroke, their carers and health providers prior to the study confirmed a sense that the instrument fitted well with a Pacific view of health and well-being as much as it did for Maori.
We hoped that use of Hua Oranga would enable the exploration of these less conventional aspects of improvement for the participants, otherwise inaccessible in a quantitative study.
We present the results for the Hua Oranga outcome tool by intervention, compare this tool with the primary outcome measures (PCS and MCS) and use factor analysis to explore psychometric properties of the instrument.
Method
- ‘Inspirational’ DVD—80 minute professionally produced DVD about stroke and stroke recovery using the inspirational stories told by four Maori and Pacific people and their families. The dominant messages were the potential for good outcomes, overcoming adversity, personal and family roles and their contribution to recovery, encouraging meaningful activity and participation for the person with stroke, and where to access resources for people following stroke. The DVD was left with the person and they were encouraged to view it as many times as they wished.
- ‘Take charge’ session (TCS)—an 80 minuteindividualised assessment with a structured risk factor and activities of daily living assessment designed to engage the patient and their family in the process of recovery, facilitating a process where they identified for themselves areas where they could make progress and set personal goals i.e. self-directed rehabilitation. No direct therapy or formal goal-setting occurred.
Both interventions were delivered by research assistants of the same ethnicity as the participant. All research assistants had a minimum of 5 days training prior to starting the study and ongoing training days during the study. The control group received written material about stroke for people and their families delivered in person by a trained research assistant of the same ethnic group as the stroke person.
Primary outcome was self-rated health related quality of life (QoL) at 12 months following randomisation measured using the PCS and MCS on the SF36.10 Secondary outcomes were the Hua Oranga score for participants and carers measured at 12 months, activities of daily living (ADL) measured by the Barthel Index (BI),11 instrumental activities of daily living (IADL) measured by Frenchay Activities Index (FAI),12 Carer Strain Index (CSI),13 dependence (modified Rankin score14 [mRS]>2), and use of rehabilitation services.
Hua Oranga scores were not presented in the primary study report8 as it is a novel measure of uncertain validity in this context. To compute a score with the Hua Oranga instrument, the participant answers four questions from each of the four domains (taha wairua, taha hinengaro, taha tinana, taha whanau; respectively spiritual, mental, physical and family dimensions) with a general format of:
'As a result of the intervention do you feel ____ (eg 'healthier from a spiritual point of view')?'
The possible answers are scored 'much worse' (-2), 'worse' (-1), no change (0), better (+1), much better (+2), giving a summed score range for the 16 questions of -32 to +32. The questions for the carer have the general format:
'Has the intervention resulted in an improved ____(eg spiritual health) for your relative?'
Scoring is the same with a range of -32 to +32. Modification to the wording was made for Pacific people, such as substituting 'Pacific person' for 'Maori'. Subsequent to the present study, a four question version of Hua Oranga has also been studied15 (see 'Discussion' section).
Analysis of variance was used to compare the effects of the two treatments, DVD and TCS for continuous outcome variables. Plots and correlation coefficients were used to explore the association between the total Hua Oranga score and the MCS and PCS of the SF-36. Simple linear regression was used to estimate the change in Hua Oranga total score corresponding to a 10 unit change on the MCS and PCS.
Plots and correlation coefficients were used to explore the associations between the pre-nominated dimensions of Hua Oranga score, and between these dimensions and the eight dimensions of the SF-36. Principal components analysis with a scree plot was used to explore the structure of the four dimensional construct of the Hua Oranga tool and the 16 questions of the instrument.
To determine a possible number of underlying factors for the Hua Oranga tool, a scree plot of the eigenvalues of the principal components analysis was used. The number of factors is suggested by where the scree plot undergoes an abrupt change in slope, but also by the number of eigenvalues greater than one. If an eigenvalue is less than one this suggests that the particular linear combination of Hua Oranga instrument dimensions or questions explains less of the variance than one single dimension or question.
Results
172 participants, 94 Maori and 78 Pacific people, were randomised. The baseline characteristics of the participants are presented in the primary publication from the study.8 139 participants (80.8%) completed follow-up at 12 months after randomisation. Of these, 135 (97%) completed the Hua Oranga and 117 (84.2%) completed the SF36. Eighty-nine carers completed the Hua Oranga. See Table 1.
The Hua Oranga instrument was sensitive to change: Hua Oranga total scores were higher for the TCS (main effect 5.3 (95% CI 1.7 to 8.8), p=0.004) but not for the DVD (main effect 3.0 (95% CI -0.6 to 6.5), p=0.10). The TCS but not the DVD was associated with significant change in both PCS on the SF36 and dependence on the mRS. The Hua Oranga scores for carers were higher in both the TCS (main effect 5.1 (95% CI 1.3 to 9.0), p=0.01) and DVD (main effect 6.4 (95% CI 2.5 to 10.2), p=0.005) groups.
There was a moderate relationship between the total score of the Hua Oranga instrument and the PCS of the SF-36 (correlation coefficient 0.55, p<0.001; see Figure 1 for scatter plot), but only a weak relationship with the MCS (correlation coefficient 0.31, p<0.001). A 10 point change on the SF-36 PCS was associated with a 5.4 (95% CI 3.9 to 6.9) point change on the Hua Oranga total score.
HO: Wairua (Spiritual)
|
HO: Hinengaro
(Mental) |
HO: Tinana
(Physical) |
HO: Whanau
(Family) |
|
SF-36 element
|
Correlation coefficient (N with both data)
|
|||
Physical functioning
|
0.37 (133)
|
0.34 (136)
|
0.50 (135)
|
0.29 (136)
|
Role-physical
|
0.44 (129)
|
0.40 (132)
|
0.43 (131)
|
0.35 (132)
|
Bodily pain
|
0.46 (132)
|
0.28 (135)
|
0.42 (134)
|
0.38 (135)
|
General health
|
0.57 (128)
|
0.54 (131)
|
0.67 (130)
|
0.57 (131)
|
Vitality
|
0.55 (128)
|
0.50 (131)
|
0.41 (130)
|
0.39 (131)
|
Social functioning
|
0.16 (131)
|
0.11 (134)
|
0.08 (0.36)
|
0.20 (134)
|
Role-emotional
|
0.40 (131)
|
0.38 (133)
|
0.36 (132)
|
0.33 (133)
|
Mental health
|
0.44 (130)
|
0.42 (133)
|
0.30 (132)
|
0.37 (133)
|
PCS
|
0.52 (114)
|
0.42 (116)
|
0.64 (115)
|
0.44 (116)
|
MCS
|
0.33 (114)
|
0.34 (116)
|
0.13 (115)
|
0.30 (116)
|
Individual dimensions of the Hua Oranga were most strongly associated with the General Health and Vitality dimensions of the SF 36 (Table 2). PCS score correlated most strongly with the physical (correlation coefficient 0.62) and spiritual (correlation coefficient 0.52) dimensions of the Hua Oranga. Individual dimensions of the Hua Oranga were strongly associated with each other (Table 3, correlation coefficients between 0.74 and 0.82).
Variables
|
HO: Wairua (Spiritual)
|
HO: Hinengaro
(Mental) |
HO: Tinana
(Physical) |
HO: Whanau
(Family) |
Correlation coefficient (N with both data)
|
||||
HO: Wairua (Spiritual)
|
1 (135)
|
0.79 (135)
|
0.75 (135)
|
0.82 (135)
|
HO: Hinengaro
(Mental) |
1 (138)
|
0.75 (137)
|
0.76 (138)
|
|
HO: Tinana
(Physical) |
1 (137)
|
0.74 (137)
|
||
HO: Whanau
(Family) |
1 (138)
|
Table 4 shows the principal components analysis of the four dimensions of the Hua Oranga. With only one of the eigenvalues greater than one, an underlying factor structure is not supported. However, Table 5 shows that a two factor structure is suggested by the factor analysis of the 16 component questions. These two factors have a physical-mental health component and a spiritual-family health component (Table 6). For this spiritual-family health component there was not a strong relationship with the equivalent SF-36 dimensions i.e. 'social functioning' and 'role-emotional' (Table 2). This suggests that the spiritual-family dimensions on the Hua Oranga instrument may be capturing quality of life issues not captured in the SF-36.
Eigenvalue number
|
Eigenvalue
|
Proportion of variance (%)
|
Cumulative proportion of variance (%)
|
1
2 3 4 |
3.3
0.28 0.24 0.17 |
82.6
7.0 6.1 4.3 |
82.6
89.6 95.7 100 |
Eigenvalue number (first four only)
|
Eigenvalue
|
Proportion of variance (%)
|
Cumulative proportion of variance (%)
|
1
2 3 4 |
10.3
1.2 0.9 0.7 |
64.2
7.6 5.6 4.4 |
64.2
71.8 77.4 81.8 |
Hua Oranga Question
|
Factor 1 loading
|
Factor 2 loading
|
Communality
|
Wairua (Spiritual)
|
|||
Content
|
0.65
|
0.51
|
0.69
|
Stronger
|
0.55
|
0.59
|
0.66
|
Spiritually healthier
|
0.44
|
0.71
|
0.69
|
Valued
|
0.58
|
0.57
|
0.67
|
Hinengaro (Mental)
|
|||
Goals
|
0.77
|
0.17
|
0.62
|
Manage feelings
|
0.51
|
0.60
|
0.62
|
Positive
|
0.69
|
0.51
|
0.73
|
Understand Health
|
0.73
|
0.40
|
0.69
|
Tinana (Physical)
|
|||
Healthier
|
0.71
|
0.44
|
0.69
|
Mental Wellbeing
|
0.68
|
0.32
|
0.57
|
Move
|
0.62
|
0.32
|
0.48
|
Physical Health
|
0.78
|
0.36
|
0.74
|
Whanau (Family)
|
|||
Clearer
|
0.20
|
0.89
|
0.83
|
Communicate
|
0.35
|
0.82
|
0.79
|
Community
|
0.65
|
0.44
|
0.61
|
Confident
|
0.52
|
0.69
|
0.75
|
Discussion
This study provided an opportunity to study the psychometric properties of the Hua Oranga outcomes instrument in a novel, sizeable population of people following stroke. The nature of the questions and the framing in terms of the four dimensions of Te Whare Tapa Wha provide Hua Oranga with good face validity if one accepts this model as a valid representation of Maori (and Pacific people’s) health and well-being. Support was provided for its sensitivity to change, both for patients and carers. External validity was provided by correlation of total Hua Oranga scores with PCS scores and significant change occurring in Hua Oranga scores for the same intervention (TCS) that significant change in PCS and dependency scores were seen.
Factor analysis suggested that the Hua Oranga measures two rather than four separate factors, one physical-mental and one spiritual-family. However, this may have been influenced by the strong 'physical' nature of stroke recovery and may be different in a population with different health problems, such as mental health. There was some evidence that the spiritual-family factor was measuring something different to the equivalent dimensions of the SF-36. Utility of the measure was good with participants having little trouble completing the questions, generally in less than half the time it took to complete the SF-36 and with higher completion rates. Little cost is involved in use of the measure.
A significant issue with the measure relates to the wording of each question. The general question stem 'As a result of the intervention do you feel ...e.g. healthier from a spiritual point of view' asks the subject to both describe a change (choices of 'much more' to 'much less') and attribute this change to the intervention.
The measures against which we have compared Hua Oranga do not have this 'change' element and the validity of that comparison could be questioned. This has been addressed in a recent study15 which trialled the measure in 43 subjects with mental health problems as well as their clinicians and whanau. Two versions of the questionnaire were used with the first option being the one used in this study.
The second option reformulated the questions as statements to indicate how the person felt now and avoided mentioning an intervention, thus under the 'Wairua' category the options were from 'I feel that my spiritual health is extremely good at present' to 'I feel that my spiritual health is extremely bad at present'. This effectively condensed the instrument down to only four questions (one for each 'pillar') from the 16 questions used in the first option.
The second option was seen by participants as more acceptable and better correlations were seen between the responses of the subjects and their clinicians when the second option was used. It remains to be seen whether the 4-question version (option 2), although more acceptable, may be too limited to be useful.
Some caution is required in interpreting the results of our study. The study population comprised Maori and Pacific people with the instrument modified for Pacific participants. We chose not to analyse the Hua Oranga results separately in order to maximise the available information. Missing data mainly related to participants with communication difficulties who, if they had been able to respond, may have responded in a consistently different way to the questions than people with normal communication.
Overall, however, the Hua Oranga appears to have much to offer in Maori health outcomes research. Its simplicity, relative brevity, minimal cost, adequate psychometric properties should favour its use in future studies with both Maori and Pacific people. Its use in health conditions other than mental health and stroke could also be encouraged.
Our analysis suggests that results should be presented as total scores. If further subdivision is attempted, two scores - one summing physical and mental dimensions and one summing spiritual and family dimensions, but not each of the four pre-specified dimensions separately would be appropriate. It would be more conventional, and statistically more simple, to score each question 0–4 rather than -2 to +2 giving a total score between zero and 64.
A further study is required to test the instrument in a new population of subjects with items selected on the basis of the two-factor structure outlined here. Further work needs to be done on the questions themselves – perhaps transforming all 16 questions into statements and comparing this to the short version tested in option two of the McClintock study.15 In a further study, the 'spiritual-family' dimension of the Hua Oranga could be explored more fully, using complementary measures, to determine what is being measured by this part of the instrument.