30th January 2015, Volume 128 Number 1408

Graham McGeoch, Peter McGeoch, Brett Shand

HealthPathways is a password-protected website that provides easy-to-follow localised “best practice” guidance for general practice teams.1 The website was developed in the Canterbury region of New Zealand and is now used routinely by the majority of general practitioners in the area.

A report on the development and characteristics of HealthPathways and its role in contributing to healthcare integration is available on the Canterbury Initiative website.2 That report shows HealthPathways has continued to grow steadily with an increase in the number of visits and pages viewed each month. Localised versions of the website are currently used in 7 other district health boards in New Zealand and 14 local health districts in Australia.

Given this widespread and increasing use of HealthPathways we decided to obtain information on perceptions of the website from healthcare professionals. This paper describes the findings of an online survey of general practitioners, hospital clinicians and practice nurses in Canterbury on the clinical usefulness and ease of use of HealthPathways.

Methods

Survey details—The survey named, ‘Looking to the Future. A Survey of HealthPathways Users’ was carried out between 9 December 2013 and 9 January 2014 as part of ongoing assessment and development of the website by the Canterbury Initiative.2 Ethical approval for the survey was therefore not sought. The questionnaire was formatted and distributed using the online tool, Survey Monkey® (Survey Monkey, CA, USA). A repeat invitation was sent to non-responders midway through the survey period. The questionnaire was not anonymous, while an incentive for participation was offered in the form of a draw for a gift voucher. A copy of the questionnaire is available on the Canterbury Initiative website.2

The survey questionnaire was based on the technology acceptance model which suggests that when an individual is presented with new technology, a number of factors influence how and when they will use it.3

Two of the most important factors are perceived usefulness and ease-of-use.4 Other surveys on acceptance of technology by medical professionals were used as a guide when constructing the questionnaire.5–9

General practitioners and practice nurses were sent a questionnaire that included 8 questions on demographics, computer literacy and use of other online clinical websites, 7 questions on the use of HealthPathways, 9 on the clinical usefulness and effectiveness of the website (7 for practice nurses), 3 on ease-of-use of the website, 2 on personal experiences, and 2 on preferences for online clinical guidance.

The questionnaire included 3 graded questions on the influence HealthPathways has had on professional working relationships and 3 open-ended questions (qualified or unqualified comment to a colleague from another region regarding the website; concerns regarding the development and maintenance of the website; and any other general comment).

Hospital clinicians were sent a different questionnaire that contained 10 questions on their experience and opinions of HealthPathways in regard to referral quality and the triage process and the same 3 open-ended questions described above. The responses to the questions were graded used a seven-point scale: 7 (totally agree), 6 (agree), 5 (slightly agree), 4 (neutral), 3 (slightly disagree), 2 (disagree), and 1 (totally disagree).

Data analysis—Responses to the questions were expressed as percentage frequency for each grade and mean score (±SD). The nonparametric Mann-Whitney test was used to compare differences in the distribution of the graded responses to questions between the general practitioners and practice nurses and between subgroups of general practitioners stratified according to gender, age (<40 yr vs. >40 yr), and location of practice (urban vs. rural), and 2) differences in demographic categorical variables between the study groups. P values <0.05 were considered statistically significant. The internal consistency of the graded responses of each professional group to theoretically related constructs in the questionnaire was assessed by calculating Cronbach’s alpha. For qualitative analysis of the open-ended questions, the comments were grouped into broad themes and the percentage frequency of each theme calculated.

Results

General practitioners

Response rates—An invitation to participate in the survey was sent to the majority of general practitioners in Canterbury (n=480).. A total of 249 questionnaires were returned (response rate, 52%).

Demographics, computer literacy and use of HealthPathways—About two-thirds of the general practitioners were female (Table 1).Approximately three-quarters of the respondents had practiced for longer than 10 years. Nearly all of the general practitioners (93%) were confident users of computer technology and considered they had better than basic skills.

The website was visited regularly (6–15 times/wk) by approximately 50% of the general practitioners, with 33% accessing the website on a more frequent basis. The respondents visited the site mainly to obtain guidance on the assessment and management of clinical conditions. Since the introduction of HealthPathways, the use of other online medical resources had decreased in 69% of respondents.

Graded responses on the effectiveness and ease of use of HealthPathways—The responses to the graded questions in the questionnaire are summarised in Table 2. With the exception of the two questions on preferences for online guidance the Cronbach alpha values for related constructs were between 0.70–0.80, indicating the responses had good internal consistency.

The role of the website in contributing to the increase in community-based health care services was acknowledged by the majority of general practitioners (88%). Overall, the response to the questions showed that the website was regarded favourably as it provided locally relevant clinical and referral information that was easily accessible during a patient consultation. 90% of respondents thought the website had improved the care they provided to their patients. However, 53% of the general practitioners considered using the website increased the duration of a patient consultation.

The majority of general practitioners considered the website made the referral process more transparent by identifying patients who required specialist care from those who could be managed in general practice. One-half of the respondents disagreed with the question on whether or not the website should be altered to provide structured decision support, with only 17% considering this would be a positive change. In this regard, 58% of respondents preferred to make decisions based on their knowledge of patients rather than obtaining advice from structured decision support. Ninety percent of respondents thought the website had improved the care they had provided to their patients.

Table 1. Demographics and use of HealthPathways and other clinical websites of the general practitioners (n=249)

FIG1

The questions on personal experiences showed that approximately one-third of general practitioners considered their relationship with patients had improved since using HealthPathways. Similarly, more than one-half of the respondents considered their working relationship with hospital clinicians had improved since the introduction of HealthPathways.

Subgroup analysis of the general practitioner data showed that female respondents had a more positive response in 16 of the 20 questions than male respondents. In particular, female doctors considered the website increased their ability to provide information resources for patients [mean score: female, 5.2 (1.3) vs. male, 4.7 (1.5); P<0.01], and disagreed to a greater extent than male doctors that the website should be altered to provide structured decision support (mean score: female, 3.1 (1.4) vs. male, 3.6 (1.5); P<0.01).

Table 2. Response of the general practitioners to the graded questions

 FIG2-2

Analysis of the data stratified according to age (<40 yr vs. >40 yr) showed younger doctors had a more positive opinion of HealthPathways than older doctors with a higher mean score in 18 of the 20 questions. Not surprisingly, younger doctors found the website easier to use than older doctors [mean score: young, 6.4 (0.6) vs. old, 6.0 (1.1); P<0.01], with a smaller proportion considering the website increased the duration of a patient consultation [mean score: young, 4.1 (1.4) vs. old, 4.6 (1.5); P<0.01].

Comparison of rural and urban general practitioners showed that of the 20 questions, rural doctors had more positive responses in 17, with the information on private referral options being regarded as particularly useful [mean score: rural, 4.4 (1.1) vs. urban, 3.6 (1.4); P<0.001].

Written comments on experience with HealthPathways—The majority of general practitioners (77%) considered HealthPathways was a service worthy of an unqualified recommendation to a colleague in another area, with only 2% expressing an overtly negative comment.

Regarding concerns on the development and maintenance of the website, approximately one-quarter of the respondents identified minor issues, mainly the increasing size of the website, the inflexibility of the clinical pathways, and the requirement to adequately reference and continually update the information. A small proportion of respondents (4%) considered that since the introduction of the website that more services were being carried out in the primary sector without a compensatory rise in funding. A similar proportion (3%) considered HealthPathways had removed clinical judgment from general practice and represented “tick-box medicine”.

Practice nurses

Response rates—A limitation of the survey for this group was the unavailability of a complete list of email addresses for practice nurses in the Canterbury region. Of the 82 invitations to participate in the survey 38 were returned (response rate, 46%). An additional 34 questionnaires were returned from practice nurses who heard about the survey from a colleague in the same practice during the survey period, making a total of 72 completed forms.

Demographics, computer literacy and use of HealthPathways—As shown in Table 3, all but one of the practice nurses were female, with two-thirds having practiced for longer than 10-yr and all considering they were confident users of computer technology. The website was visited regularly (6-15 times/wk) by the majority of respondents (94%). Practice nurses used the site in a more general manner than general practitioners and accessed the site mainly to obtain patient information or linkage to other clinical resources. Since the introduction of HealthPathways, the use of other online medical resources had decreased in 65% of practice nurses, although about one-fifth (19%) stated that it had increased their access of other websites.

Graded responses on the effectiveness and ease of use of HealthPathways—The responses to the graded questions in the questionnaire are summarised in Table 4. With the exception of the two questions on preferences for online guidance, the Cronbach alpha values for related constructs ranged between 0.61–0.76, indicating the responses had acceptable internal consistency.

The majority of practice nurses considered the website was easy to use, had contributed to an increase in community-based health care, and had improved the care they were able to provide to their patients by distinguishing between those they could manage themselves from those who required to be seen by a general practitioner.

Similar to the responses of the general practitioners, about one-half (40%) of the practice nurses considered using the website had increased the duration of a patient consultation. The questions on working relationships showed that a significantly greater proportion of practice nurses than general practitioners considered their relationship with patients had improved since using HealthPathways (nurses 60% vs. general practitioners 31%; p<0.001), while a similar proportion stated their working relationship with hospital clinicians had improved since introduction of the website (nurses, 51% vs. general practitioners, 57%; p=0.12).

In contrast to the response of the general practitioners, about one-third of the practice nurses stated a preference for the website to be altered to provide structured decision support (nurses, 35% vs. general practitioners, 17%; p<0.001). This was reflected by a greater proportion of general practitioners preferring to make clinical decisions based on their knowledge of patients rather than obtaining advice from structured decision support (nurses, 43% vs. general practitioners, 58%; p<0.01).


Table 3. Demographics and use of HealthPathways and other clinical websites of the practice nurses (n=72).

FIG3

Written comments on experience with HealthPathways—The majority of practice nurses (82%) considered HealthPathways was a service worthy of an unqualified recommendation to a colleague in another area, with no respondent expressing an overtly negative comment. Only a small number of respondents had minor concerns regarding the development and maintenance of the website, with the majority of these being navigational issues in specific clinical pathways.


Table 4. Response of the practice nurses to the graded questions

FIG4 

Hospital clinicians

Response rates and demographics—The hospital clinicians sent the survey questionnaire were either the clinical director of their department or had been involved in the work groups that contributed to the development of the pathways. A summary of the demographics of the hospital clinicians is shown in Table 5. The response rate to the survey was 65% (43 returned vs. 66 sent).

Graded responses on the effectiveness of HealthPathways—The Cronbach alpha values of the two related constructs showed the responses had relatively high internal consistency. As shown in Table 5, the majority of hospital clinicians (87%) considered the website had contributed to better patient management in primary care and had improved all stages of referral and follow-up of patients. A common response was that referrals now contained more explicit and relevant information in a standardized format, with 36% of respondents considering that the acuity of the referred patients had increased. These improvements were seen as assisting in the triage process and reducing the rejection rate of referrals. Approximately 60% of the clinicians agreed that involvement with development and maintenance of the website had improved their department’s working relationships with general practices. However, a greater proportion (75%) indicated that this involvement required significant time and effort by staff members.

Written comments on HealthPathways—More than half (58%) of hospital clinicians gave an unqualified recommendation to HealthPathways with common themes being the value of the website as a resource for general practice and its positive effect on the quality of referrals. The time required to develop and maintain the clinical pathways was repeated by 16% of the respondents. Other mildly expressed concerns included the increasing size and overly prescriptive nature of the website.

Table 5. Demographics and response to the graded questions of the hospital clinicians

FIG5 

Discussion

The respondents in this survey generally expressed positive opinions on the role of HealthPathways in their day-to-day practices. The clinical information on the website and the local criteria for secondary care referral was considered useful by most general practitioners and practice nurses. This information appeared to make the referral process more transparent and help general practitioners identify patients who required referral from those who they could manage themselves, or in the case of practice nurses, patients who needed to be seen by a doctor.

The majority of hospital clinicians responded that the referral and triage process for secondary care had improved since the introduction of HealthPathways. A further encouraging finding of the survey was that the involvement of healthcare professionals in the development and continuing review of the clinical pathways had contributed to better working relationships in the region. The small proportion of negative comments or opinions of the website included concerns on its increasing size and overly prescriptive nature and its effect of increasing the workload of both primary and secondary care clinicians.

At the time HealthPathways was developed it was generally acknowledged that clinical guidelines10-12 and structured decision support systems13were not achieving their goals of improving treatment and clinical outcomes of patients. Numerous barriers to the effective use of guidelines were identified that included the prior knowledge and beliefs of health professionals,14 and the prescriptive nature of guidelines often making it difficult and time consuming to fit them easily into a patient consultation.15,16Another common barrier was the perception that guidelines were developed by experts who did not understand general practice.17

To overcome these barriers it was recognised that guidelines needed to be applicable to usual practice11,12 and be presented in a consistent and brief format that was easy to follow.14,16 The need for unbiased sources of evidence,11 local opinion leaders, educational seminars,18 and structured audit and feedback processes15 were other important factors for successful integration of online guidelines.

HealthPathways was designed to incorporate all these requirements by providing relevant localised information written by local clinicians, that conformed to ‘best practice’ guidelines and was backed-up by continual feedback, two-yearly reviews, and where necessary clinical audit. Our survey indicates that HealthPathways has overcome many of the barriers encountered by other online clinical resources, with the collaboration between local healthcare professionals in its development being a major factor in breaking down historical barriers. The relevance of the information to the local health community was another important factor.

The main barrier to use of online clinical information systems is time.11,15,19 Despite being easy to use about 50% of the general practitioners and practice nurses in our survey thought HealthPathways had increased the duration of a patient consultation. This could be due to either time spent on the site or empowerment of nurses and doctors to do more for their patients.

Another relatively common comment was that the website was becoming too big. To avoid this impacting adversely on a patient consultation represents a challenge facing HealthPathways in the future.

The lack of interest for decision support to be more integrated with the patient record and based on computer algorithms has relevance to the current investment in such systems in other parts of New Zealand. Further study is warranted to determine whether or not simpler systems including HealthPathways are sufficient and preferred by clinical teams.

Subgroup analysis of the general practitioner data provided some interesting findings. Female general practitioners, younger doctors and rural practitioners had an overall more positive opinion of the website.

This tendency for female general practitioners to use online clinical guidance to a greater extent than their male counterparts has been described by several earlier studies.11,21 Our finding that younger doctors found the website easier to use than older doctors was not unexpected and has been reported previously.11,20–22 The more positive general opinion of rural practitioners on online clinical resources was also not unexpected given their remote locations and relative isolation, with the survey showing information on private referral options was especially useful.

This survey had several limitations. The response rate of the general practitioners and practice nurses did not reach 60%, the rate considered to signify that a representative proportion of the study population has been surveyed. The possibility that individuals with a more positive perception of HealthPathways responded to the survey may therefore be a potential source of bias. The survey was carried out over a relatively short period, immediately preceding the summer holiday period, which was likely to have influenced the response rate. The practice nurses were a select group with known email addresses, while the hospital clinicians were those who were willing to assist in the development of the clinical pathways.

In conclusion, this survey shows that HealthPathways has achieved its objectives of providing easily accessible best practice guidance for general practice teams that is relevant to local services and resources.

The survey shows that HealthPathways has contributed to fostering better working relationships between the primary and secondary healthcare sectors in Canterbury and as such has acted both as an online clinical resource and a tool for promoting integration of health care delivery. 

Summary

HealthPathways is a website that was developed in Canterbury to provide general practitioners and practice nurses with locally relevant clinical information required during a patient appointment. An online survey was carried out in 2013 to determine the opinions of general practitioners, practice nurses and hospital doctors on HealthPathways. The survey showed a very high level of acceptance of HealthPathways in the three groups, with the website considered easy-to-use, and as having contributed to increased healthcare in the community and an improved referral and triage process. More than half of the respondents considered HealthPathways had improved their working relationships with other healthcare professionals. Analysis of the data showed some small but interesting differences in the responses of female, younger or rural general practitioners.

Abstract

Aim

An online survey was used to determine the perceptions of healthcare professionals in Canterbury on HealthPathways, a website that provides clinical and referral information for general practice teams, relevant to locally available health services and resources.

Method

The survey questionnaire included questions on the effectiveness and ease-of-use of the website, and computer literacy and use of online clinical guidance systems. Differences in the distribution of responses between work groups were analysed using the Mann-Whitney test.

Results

249/480 (52%) of general practitioners, 72/156 (46%) of practice nurses, and 43/66 (65%) of hospital clinicians completed the questionnaire. Approximately 90–95% of general practice teams considered the website was easy to use and had contributed to both an increase and improvement of care in the community, with about 50% stating that it had improved their relationships with patients and hospital clinicians. Minor concerns included the website’s increasing size and prescriptive nature and that it increased the duration of a patient consultation. Approximately 60% of hospital clinicians reported improvements in referral quality and triage and working relationships with general practices since the introduction of HealthPathways.

Conclusion

HealthPathways has achieved a high level of acceptance in both primary and secondary care, and has therefore acted as a valuable change management tool increasing healthcare integration in Canterbury.

Author Information

Graham McGeoch, General Practitioner; Peter McGeoch, Medical Student; Brett Shand, Clinical Writer and Analyst. The Canterbury Initiative, Canterbury District Health Board, Christchurch

Acknowledgements

We thank Emma Harding, Streamliners Ltd for her assistance with the survey and help with preliminary data analysis.

Correspondence

Dr Graham McGeoch, The Canterbury Initiative, 4th floor, The Princes Margaret Hospital, Christchurch, New Zealand.

Correspondence Email

mcgeochg@gmail.com

Competing Interests

Nil.

References

1. Timmins N, Ham C. The quest for integrated health and social care: A case study in Canterbury, New Zealand: The King’s Fund. 2013 Available at: http://www.kingsfund.org.uk/publications/quest-integrated-health-and-social-care

2. The Canterbury Initiative. http://www.canterburyinitiative.org.nz/

3. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly. 1989;13:319–40.

4. Bagozzi RP, Davis FD, Warshaw PR. Development and test of theory of technological learning and usage. Human Relations. 1992;45:660–86.

5. Zanaboni P, Scalvini S, Bernocchi P, et al. Teleconsultation service to improve healthcare in rural areas: acceptance, organizational impact and appropriateness BMC Health Serv Res. 2009;9:238. doi: 10.1186/1472-6963-9-238.

6. Parra C, Jódar-Sánchez F, Jiménez-Hernández MD, et al. Development, Implementation, and Evaluation of a Telemedicine Service for the Treatment of Acute Stroke Patients: TeleStroke. Interact J Med Res. 2012;1:e15. doi: 10.2196/ijmr.2163.

7. Patterson V, Wootton R. A web-based telemedicine system for low-resource settings 13 years on: insights from referrers and specialists. Glob Health Action. 2013;6:21465. doi: 10.3402/gha.v6i0.21465.

8. Lee SI, Park H, Kim JW, et al. Physicians' perceptions and use of a health information exchange: a pilot program in South Korea. Telemed J E Health. 2012;18:604–12. doi: 10.1089/tmj.2011.0261. Epub 2012.

9. van der Wees PJ, Zagers CA, de Die SE, et al. Developing a questionnaire to identify perceived barriers for implementing the Dutch physical therapy COPD clinical practice guideline. BMC Health Serv Res. 2013;13:159. doi: 10.1186/1472-6963-13-159.

10. Zielstorff RD. Online practice guidelines: issues, obstacles, and future prospects. J Am Med Inform Assoc. 1998;5:227–36.

11. Watkins C, Harvey I, Langley C, et al. General practitioners' use of guidelines in the consultation and their attitudes to them. Br J Gen Pract. 1999;49:11–5.

12. Burgers JS, Grol RP, Zaat JO, et al. Characteristics of effective clinical guidelines for general practice. Br J Gen Pract. 2003;53:15–9.

13. de Jong JD, Groenewegen PP, Spreeuwenberg P, et al. Do decision support systems influence variation in prescription? BMC Health Serv Res. 2009;9:20. doi: 10.1186/1472–6963-9-20.

14. Langley C, Faulkner A, Watkins C, et al. Use of guidelines in primary care--practitioners' perspectives. Fam Pract. 1998;15:105–11.

15. Cranney M, Warren E, Barton S, et al. Why do GPs not implement evidence-based guidelines? A descriptive study. Fam Pract. 2001;18:359–63.

16. Veldhuijzen W, Ram PM, van der Weijden T, et al. Characteristics of communication guidelines that facilitate or impede guideline use: a focus group study. BMC Fam Pract. 2007;8:31.

17. Gupta L, Ward JE, Hayward RS. Clinical practice guidelines in general practice: a national survey of recall, attitudes and impact. Med J Aust. 1997;20:166:69–72.

18. Lugtenberg M(1), Burgers JS, Han D, et al. [Epub ahead of print] General practitioners' preferences for interventions to improve guideline adherence. J Eval Clin Pract. 2014. doi: 10.1111/jep.12209.

19. Davies K. The information-seeking behaviour of doctors: a review of the evidence. Health Inform and Libraries J. 2007 24;78–94.

20. Ward R, Stevens C, Brentnall P, et al. The attitudes of health staff to information technology: a comprehensive review of the research literature. Health Inform and Libraries J. 2008;25:81–97. doi: 10.1111/j.1471-1842.2008.00777.x.

21. Fantini MP, Compagni A, Rucci P, et al. General practitioners’ adherence to evidence-based guidelines: a multilevel analysis. Health Care Manage Rev. 2012;37:67–76. doi: 10.1097/HMR.Ob013e31822241cf.

22. Ammenwerth E, Mansmann U, Iller C, et al. Factors affecting and affected by user acceptance of computer-based nursing documentation: results of a two-year study. J Am Med Informatics Assoc 2003 10;69–84.doi:10.1197/jamia.M1118.