HealthPathways is a password-protected website that provides easy-to-follow localised “best practice” guidance for general practice teams.1 The website was developed in the Canterbury region of New Zealand and is now used routinely by the majority of general practitioners in the area.
A report on the development and characteristics of HealthPathways and its role in contributing to healthcare integration is available on the Canterbury Initiative website.2 That report shows HealthPathways has continued to grow steadily with an increase in the number of visits and pages viewed each month. Localised versions of the website are currently used in 7 other district health boards in New Zealand and 14 local health districts in Australia.
Given this widespread and increasing use of HealthPathways we decided to obtain information on perceptions of the website from healthcare professionals. This paper describes the findings of an online survey of general practitioners, hospital clinicians and practice nurses in Canterbury on the clinical usefulness and ease of use of HealthPathways.
Survey details—The survey named, ‘Looking to the Future. A Survey of HealthPathways Users’ was carried out between 9 December 2013 and 9 January 2014 as part of ongoing assessment and development of the website by the Canterbury Initiative.2 Ethical approval for the survey was therefore not sought. The questionnaire was formatted and distributed using the online tool, Survey Monkey® (Survey Monkey, CA, USA). A repeat invitation was sent to non-responders midway through the survey period. The questionnaire was not anonymous, while an incentive for participation was offered in the form of a draw for a gift voucher. A copy of the questionnaire is available on the Canterbury Initiative website.2
The survey questionnaire was based on the technology acceptance model which suggests that when an individual is presented with new technology, a number of factors influence how and when they will use it.3
Two of the most important factors are perceived usefulness and ease-of-use.4 Other surveys on acceptance of technology by medical professionals were used as a guide when constructing the questionnaire.5–9
General practitioners and practice nurses were sent a questionnaire that included 8 questions on demographics, computer literacy and use of other online clinical websites, 7 questions on the use of HealthPathways, 9 on the clinical usefulness and effectiveness of the website (7 for practice nurses), 3 on ease-of-use of the website, 2 on personal experiences, and 2 on preferences for online clinical guidance.
The questionnaire included 3 graded questions on the influence HealthPathways has had on professional working relationships and 3 open-ended questions (qualified or unqualified comment to a colleague from another region regarding the website; concerns regarding the development and maintenance of the website; and any other general comment).
Hospital clinicians were sent a different questionnaire that contained 10 questions on their experience and opinions of HealthPathways in regard to referral quality and the triage process and the same 3 open-ended questions described above. The responses to the questions were graded used a seven-point scale: 7 (totally agree), 6 (agree), 5 (slightly agree), 4 (neutral), 3 (slightly disagree), 2 (disagree), and 1 (totally disagree).
Data analysis—Responses to the questions were expressed as percentage frequency for each grade and mean score (±SD). The nonparametric Mann-Whitney test was used to compare differences in the distribution of the graded responses to questions between the general practitioners and practice nurses and between subgroups of general practitioners stratified according to gender, age (<40 yr vs. >40 yr), and location of practice (urban vs. rural), and 2) differences in demographic categorical variables between the study groups. P values <0.05 were considered statistically significant. The internal consistency of the graded responses of each professional group to theoretically related constructs in the questionnaire was assessed by calculating Cronbach’s alpha. For qualitative analysis of the open-ended questions, the comments were grouped into broad themes and the percentage frequency of each theme calculated.
Response rates—An invitation to participate in the survey was sent to the majority of general practitioners in Canterbury (n=480).. A total of 249 questionnaires were returned (response rate, 52%).
Demographics, computer literacy and use of HealthPathways—About two-thirds of the general practitioners were female (Table 1).Approximately three-quarters of the respondents had practiced for longer than 10 years. Nearly all of the general practitioners (93%) were confident users of computer technology and considered they had better than basic skills.
The website was visited regularly (6–15 times/wk) by approximately 50% of the general practitioners, with 33% accessing the website on a more frequent basis. The respondents visited the site mainly to obtain guidance on the assessment and management of clinical conditions. Since the introduction of HealthPathways, the use of other online medical resources had decreased in 69% of respondents.
Graded responses on the effectiveness and ease of use of HealthPathways—The responses to the graded questions in the questionnaire are summarised in Table 2. With the exception of the two questions on preferences for online guidance the Cronbach alpha values for related constructs were between 0.70–0.80, indicating the responses had good internal consistency.
The role of the website in contributing to the increase in community-based health care services was acknowledged by the majority of general practitioners (88%). Overall, the response to the questions showed that the website was regarded favourably as it provided locally relevant clinical and referral information that was easily accessible during a patient consultation. 90% of respondents thought the website had improved the care they provided to their patients. However, 53% of the general practitioners considered using the website increased the duration of a patient consultation.
The majority of general practitioners considered the website made the referral process more transparent by identifying patients who required specialist care from those who could be managed in general practice. One-half of the respondents disagreed with the question on whether or not the website should be altered to provide structured decision support, with only 17% considering this would be a positive change. In this regard, 58% of respondents preferred to make decisions based on their knowledge of patients rather than obtaining advice from structured decision support. Ninety percent of respondents thought the website had improved the care they had provided to their patients.
Table 1. Demographics and use of HealthPathways and other clinical websites of the general practitioners (n=249)
The questions on personal experiences showed that approximately one-third of general practitioners considered their relationship with patients had improved since using HealthPathways. Similarly, more than one-half of the respondents considered their working relationship with hospital clinicians had improved since the introduction of HealthPathways.
Subgroup analysis of the general practitioner data showed that female respondents had a more positive response in 16 of the 20 questions than male respondents. In particular, female doctors considered the website increased their ability to provide information resources for patients [mean score: female, 5.2 (1.3) vs. male, 4.7 (1.5); P<0.01], and disagreed to a greater extent than male doctors that the website should be altered to provide structured decision support (mean score: female, 3.1 (1.4) vs. male, 3.6 (1.5); P<0.01).
Table 2. Response of the general practitioners to the graded questions
Analysis of the data stratified according to age (<40 yr vs. >40 yr) showed younger doctors had a more positive opinion of HealthPathways than older doctors with a higher mean score in 18 of the 20 questions. Not surprisingly, younger doctors found the website easier to use than older doctors [mean score: young, 6.4 (0.6) vs. old, 6.0 (1.1); P<0.01], with a smaller proportion considering the website increased the duration of a patient consultation [mean score: young, 4.1 (1.4) vs. old, 4.6 (1.5); P<0.01].
Comparison of rural and urban general practitioners showed that of the 20 questions, rural doctors had more positive responses in 17, with the information on private referral options being regarded as particularly useful [mean score: rural, 4.4 (1.1) vs. urban, 3.6 (1.4); P<0.001].
Written comments on experience with HealthPathways—The majority of general practitioners (77%) considered HealthPathways was a service worthy of an unqualified recommendation to a colleague in another area, with only 2% expressing an overtly negative comment.
Regarding concerns on the development and maintenance of the website, approximately one-quarter of the respondents identified minor issues, mainly the increasing size of the website, the inflexibility of the clinical pathways, and the requirement to adequately reference and continually update the information. A small proportion of respondents (4%) considered that since the introduction of the website that more services were being carried out in the primary sector without a compensatory rise in funding. A similar proportion (3%) considered HealthPathways had removed clinical judgment from general practice and represented “tick-box medicine”.
Response rates—A limitation of the survey for this group was the unavailability of a complete list of email addresses for practice nurses in the Canterbury region. Of the 82 invitations to participate in the survey 38 were returned (response rate, 46%). An additional 34 questionnaires were returned from practice nurses who heard about the survey from a colleague in the same practice during the survey period, making a total of 72 completed forms.
Demographics, computer literacy and use of HealthPathways—As shown in Table 3, all but one of the practice nurses were female, with two-thirds having practiced for longer than 10-yr and all considering they were confident users of computer technology. The website was visited regularly (6-15 times/wk) by the majority of respondents (94%). Practice nurses used the site in a more general manner than general practitioners and accessed the site mainly to obtain patient information or linkage to other clinical resources. Since the introduction of HealthPathways, the use of other online medical resources had decreased in 65% of practice nurses, although about one-fifth (19%) stated that it had increased their access of other websites.
Graded responses on the effectiveness and ease of use of HealthPathways—The responses to the graded questions in the questionnaire are summarised in Table 4. With the exception of the two questions on preferences for online guidance, the Cronbach alpha values for related constructs ranged between 0.61–0.76, indicating the responses had acceptable internal consistency.
The majority of practice nurses considered the website was easy to use, had contributed to an increase in community-based health care, and had improved the care they were able to provide to their patients by distinguishing between those they could manage themselves from those who required to be seen by a general practitioner.
Similar to the responses of the general practitioners, about one-half (40%) of the practice nurses considered using the website had increased the duration of a patient consultation. The questions on working relationships showed that a significantly greater proportion of practice nurses than general practitioners considered their relationship with patients had improved since using HealthPathways (nurses 60% vs. general practitioners 31%; p<0.001), while a similar proportion stated their working relationship with hospital clinicians had improved since introduction of the website (nurses, 51% vs. general practitioners, 57%; p=0.12).
In contrast to the response of the general practitioners, about one-third of the practice nurses stated a preference for the website to be altered to provide structured decision support (nurses, 35% vs. general practitioners, 17%; p<0.001). This was reflected by a greater proportion of general practitioners preferring to make clinical decisions based on their knowledge of patients rather than obtaining advice from structured decision support (nurses, 43% vs. general practitioners, 58%; p<0.01).
Table 3. Demographics and use of HealthPathways and other clinical websites of the practice nurses (n=72).
Written comments on experience with HealthPathways—The majority of practice nurses (82%) considered HealthPathways was a service worthy of an unqualified recommendation to a colleague in another area, with no respondent expressing an overtly negative comment. Only a small number of respondents had minor concerns regarding the development and maintenance of the website, with the majority of these being navigational issues in specific clinical pathways.
Table 4. Response of the practice nurses to the graded questions
Response rates and demographics—The hospital clinicians sent the survey questionnaire were either the clinical director of their department or had been involved in the work groups that contributed to the development of the pathways. A summary of the demographics of the hospital clinicians is shown in Table 5. The response rate to the survey was 65% (43 returned vs. 66 sent).
Graded responses on the effectiveness of HealthPathways—The Cronbach alpha values of the two related constructs showed the responses had relatively high internal consistency. As shown in Table 5, the majority of hospital clinicians (87%) considered the website had contributed to better patient management in primary care and had improved all stages of referral and follow-up of patients. A common response was that referrals now contained more explicit and relevant information in a standardized format, with 36% of respondents considering that the acuity of the referred patients had increased. These improvements were seen as assisting in the triage process and reducing the rejection rate of referrals. Approximately 60% of the clinicians agreed that involvement with development and maintenance of the website had improved their department’s working relationships with general practices. However, a greater proportion (75%) indicated that this involvement required significant time and effort by staff members.
Written comments on HealthPathways—More than half (58%) of hospital clinicians gave an unqualified recommendation to HealthPathways with common themes being the value of the website as a resource for general practice and its positive effect on the quality of referrals. The time required to develop and maintain the clinical pathways was repeated by 16% of the respondents. Other mildly expressed concerns included the increasing size and overly prescriptive nature of the website.
Table 5. Demographics and response to the graded questions of the hospital clinicians
The respondents in this survey generally expressed positive opinions on the role of HealthPathways in their day-to-day practices. The clinical information on the website and the local criteria for secondary care referral was considered useful by most general practitioners and practice nurses. This information appeared to make the referral process more transparent and help general practitioners identify patients who required referral from those who they could manage themselves, or in the case of practice nurses, patients who needed to be seen by a doctor.
The majority of hospital clinicians responded that the referral and triage process for secondary care had improved since the introduction of HealthPathways. A further encouraging finding of the survey was that the involvement of healthcare professionals in the development and continuing review of the clinical pathways had contributed to better working relationships in the region. The small proportion of negative comments or opinions of the website included concerns on its increasing size and overly prescriptive nature and its effect of increasing the workload of both primary and secondary care clinicians.
At the time HealthPathways was developed it was generally acknowledged that clinical guidelines10-12 and structured decision support systems13were not achieving their goals of improving treatment and clinical outcomes of patients. Numerous barriers to the effective use of guidelines were identified that included the prior knowledge and beliefs of health professionals,14 and the prescriptive nature of guidelines often making it difficult and time consuming to fit them easily into a patient consultation.15,16Another common barrier was the perception that guidelines were developed by experts who did not understand general practice.17
To overcome these barriers it was recognised that guidelines needed to be applicable to usual practice11,12 and be presented in a consistent and brief format that was easy to follow.14,16 The need for unbiased sources of evidence,11 local opinion leaders, educational seminars,18 and structured audit and feedback processes15 were other important factors for successful integration of online guidelines.
HealthPathways was designed to incorporate all these requirements by providing relevant localised information written by local clinicians, that conformed to ‘best practice’ guidelines and was backed-up by continual feedback, two-yearly reviews, and where necessary clinical audit. Our survey indicates that HealthPathways has overcome many of the barriers encountered by other online clinical resources, with the collaboration between local healthcare professionals in its development being a major factor in breaking down historical barriers. The relevance of the information to the local health community was another important factor.
The main barrier to use of online clinical information systems is time.11,15,19 Despite being easy to use about 50% of the general practitioners and practice nurses in our survey thought HealthPathways had increased the duration of a patient consultation. This could be due to either time spent on the site or empowerment of nurses and doctors to do more for their patients.
Another relatively common comment was that the website was becoming too big. To avoid this impacting adversely on a patient consultation represents a challenge facing HealthPathways in the future.
The lack of interest for decision support to be more integrated with the patient record and based on computer algorithms has relevance to the current investment in such systems in other parts of New Zealand. Further study is warranted to determine whether or not simpler systems including HealthPathways are sufficient and preferred by clinical teams.
Subgroup analysis of the general practitioner data provided some interesting findings. Female general practitioners, younger doctors and rural practitioners had an overall more positive opinion of the website.
This tendency for female general practitioners to use online clinical guidance to a greater extent than their male counterparts has been described by several earlier studies.11,21 Our finding that younger doctors found the website easier to use than older doctors was not unexpected and has been reported previously.11,20–22 The more positive general opinion of rural practitioners on online clinical resources was also not unexpected given their remote locations and relative isolation, with the survey showing information on private referral options was especially useful.
This survey had several limitations. The response rate of the general practitioners and practice nurses did not reach 60%, the rate considered to signify that a representative proportion of the study population has been surveyed. The possibility that individuals with a more positive perception of HealthPathways responded to the survey may therefore be a potential source of bias. The survey was carried out over a relatively short period, immediately preceding the summer holiday period, which was likely to have influenced the response rate. The practice nurses were a select group with known email addresses, while the hospital clinicians were those who were willing to assist in the development of the clinical pathways.
In conclusion, this survey shows that HealthPathways has achieved its objectives of providing easily accessible best practice guidance for general practice teams that is relevant to local services and resources.
The survey shows that HealthPathways has contributed to fostering better working relationships between the primary and secondary healthcare sectors in Canterbury and as such has acted both as an online clinical resource and a tool for promoting integration of health care delivery.