Public narrative feedback: A journey of growth
A case study of online, transparent feedback in a small rural health service.
Download this case study as a PDF
Executive Summary
Care Opinion Australia is an independent, not-for-profit, award-winning website that was founded in the UK in 2005. In Australia, Care Opinion was founded and established in July 2012, with an aim of harnessing the power of the web to carry the voices of consumers and carers into the heart of health and care services.
This report explores the power of public, narrative feedback on care services and the barriers to its use experienced by consumers, individual staff and at the organisational level. Findings within this report have also been highlighted by academic research and anecdotal evidence.
Using a case study, this report addresses these barriers and answers the question of how an organisation can embrace online, public feedback whilst ensuring:
- Consumers have a safe space to share their experience via a mechanism of their choosing
- Staff can safely invite, use, respond to and benefit from consumer feedback
- Members of the public can hear about other consumers’ stories that form compliments, complaints and suggestions through their stories
- Organisations can use the one mechanism to have their voices heard and important messages shared to both specific consumers and the broader public via story responses.
This case study analysed quantitative and qualitative data generated about Kerang District Health (Kerang) based on its experience with a pilot program. The pilot, commissioned and funded by Safer Care Victoria, gave seven health services access to trial the use of the Care Opinion platform for an 18-month period.
Data included stories published on the platform throughout the pilot and questionnaires completed by key staff after its conclusion (see Appendix A).
Staff questionnaires revealed that prior to the pilot, the take-up of feedback forms and annual patient surveys was reasonably low as consumers felt they would be identifiable based on the small size of the community. The specifics of the feedback received was not transparently shared internally and staff on the front line felt feedback was typically complaints-based. There was a sense of feedback-fatigue, thus it was not actively sought or welcomed by staff.
At the pilot’s commencement, staff reportedly expected negative feedback, however, there was anticipation of having transparency around feedback content.
The case study found that, despite initial expectations, the feedback (stories) was submitted consistently and 110 stories were published throughout the pilot period. Consumers reportedly appreciated having an avenue to anonymously to:
- express the full gamut of their feelings and experiences
- have input to service improvements
- know they were being heard from staff on the frontline as well as management and acted on.
79 per cent of published stories were assessed as purely positive, many of which personally and publicly commended staff, thus motivating and encouraging staff to continue to provide good care and/or to improve their provision of care.
Of the 110 stories received a total of 32,124 public views were gained, enabling Kerang to:
- publicly showcase the organisation’s high-quality staff and delivery of care to the broader community
- educate the storyteller and the broader community on the processes staff follow and restraints the organisation works within, particularly within the rural context
- demonstrate the value it places on the consumer's voice
- highlight what the organisation does with the feedback received.
Key to the success of the use of Care Opinion in this case study was that the system was implemented with the full support of Kerang’s Board of Management, Chief Executive Officer and Executive Leadership team. This top-down approach of seeking transparency and opportunities to learn from the consumer's voice enabled staff to feel supported and encouraged by stories, whether they were a compliment, complaint or somewhere in between.
It is recommended that care organisations using or considering using Care Opinion Australia, emulate the steps Kerang’s management took throughout the pilot, including:
- 1. Communicating the Chief Executive Officer and senior executive team endorsement to all staff
- 2. Actively fostering staff engagement with the system
- 3. Keeping the platform topical among the workforce
- 4. Engaging with consumers about their feedback
- 5. Using consumer feedback to drive care change and improvements.
This case study found that public, online feedback in narrative, dialogical form radically transformed the culture of feedback fatigue to one where feedback was welcomed, sought and perceived as a gift.
Background
If we have learned anything from COVID-19, it is that organisations need to, and can, quickly adapt to operating in an online world. We’ve seen that staff can work from home, adjust to changed working conditions when supported, and manage consumer engagement online when in-person contact is no longer best practice.
With social media so popular and accessible, it is inevitable that people turn to these platforms to share their experiences, learn about others’ experiences and openly converse about them. Thus, it is common for many consumers to choose to publish their opinions and reviews of their care online, whether it is solicited or not, with many others joining the conversation.
However, as few online platforms offer little protection from ‘trolling’ and unsolicited contributions to conversations in the public sphere, apprehension about public, online feedback is well understood.
It is commonly said that when a person has a good experience they will tell one person, but when it is a poor experience they will tell ten people. However, people are fallible and health care services cannot always "get it right" and there are usually two sides to a story.
Consumer voices have been found to have significant power to inform and drive service improvements and change, particularly when delivered in narrative, dialogical form, called ‘stories’ on the Care Opinion platform. However, both academic research and anecdotal evidence highlight several barriers to the efficacy and safety of online, transparent feedback in health and care services, including:
- consumer reluctance to provide feedback for fear of being identified and/or care becoming compromised
- staff reluctance to seek consumer feedback based on perceptions of it being unsafe from it being used as a performance management tool and/or being publicly ‘named and shamed’
- organisational culture that does not empower staff ‘on the ground’ to take ownership of, and act on consumer feedback
- organisational fear of reputational damage.
Understanding the patient experience has long been an important element of informing and improving the quality of healthcare. However, a significant barrier to the provision of feedback in care settings is the fear it will negatively impact the healthcare relationship (1).
When patient feedback is received, all too frequently the data is not used and applied in the healthcare setting to influence service improvement (2). This can be exacerbated as some front line staff do not take ownership of feedback if it is provided about their hospital and not specifically directed to their ward (3).
Complicating the issue is that without a coordinated and streamlined approach to processing patient feedback, the patient experience of giving feedback, and staff process of using feedback can be variable and suboptimal (4). These findings are not universal, however, and this case study examines how Kerang used CareOpinion Australia to overcome consumer fears of being identified when providing feedback, empower staff to take ownership of it, streamline its feedback process and use the data to improve service delivery and organisational culture.
Kerang District Health
Kerang is a 20-bed small rural health service housed in a state-of-the-art redeveloped facility opened in 2016. The organisation has provided health care to residents of the Gannawarra Shire Council area for over 60 years, offering a broad range of acute medical, oncology and surgical services, which include general surgery, gynaecology and urology services.
With an annual turnover of more than $14 million and employing approximately 180 staff, Kerangprovides a wide range of community services, a GP clinic and residential aged care within a brand new 30-bed facility connected to the acute hospital. Separations recorded for the 2018 year from acute services are 1273. The service has an Urgent Care Centre operating 24 hours a day and in the2018 year, there were 2488 presentations.
In 2018, following a recent change of Board of Management (BOM) members, Kerang was looking for an online-based feedback system that was mediated and that supported its staff and consumers on the journey; a transparent system for all of its stakeholders. It successfully applied to participate in the Care Opinion Subscription Pilot to engage with its online-based feedback system that was mediated and that supported its staff and consumers on the journey.
At the time, it also coincided with a newly rolled out an initiative to educate staff on its culture which included the consumer voice, how feedback was perceived and what was done with it.
The organisation felt that stories told on the Care Opinion platform would complement existing feedback forums as it would be real-time data and could be actioned in a timelier manner. It was also envisaged that the BOM would be informed in real-time, enabling them to be part of the complaints/feedback system rather than only being informed after resolutions had been reached.
The application was approved and Kerang commenced the pilot on 1 July 2018.
The Care Opinion system
Care Opinion moderates every story to ensure that stories meet its moderation principles. The policy is based on the premise of keeping everyone who uses our platform and those referred to in stories, safe in the public domain. More information can be found on the Care Opinion website about the moderation principles and how the company’s moderators keep everyone safe.
The moderation process encompasses three broad steps:
- 1. Removing the names and identifying information of staff from negative stories and authors in all stories, and removing surnames from staff in positive stories.
- 2. Assigning and polarising (negative, positive or neutral) tags related to aspects of care mentioned in the story and the emotions the storyteller experienced.
- 3. Assessing a criticality rating to the story (purely positive to severely negative) against a set of guidelines (see Appendix B: Moderation Criticality Table) based on how critical the story is of the service or care experience.
Implementation of Care Opinion
In order to encourage staff engagement with the platform, the pilot was launched at a staff afternoon tea where staff:
- were shown how to access and review stories
- educated on how to encourage their consumers to give feedback
- encouraged to see feedback as a gift, whether positive or negative.
Staff reported feeling anxious about consumers being provided the opportunity to publicly criticise staff. However, staff also reported feeling optimistic in anticipation of having direct access to the feedback through the platform.
For the duration of the pilot, existing hard copy feedback mechanisms were taken out of circulation but remained available for download on the Kerang’s website. Email feedback or posted letters continued to be accepted, lodged and responded to via VHIMS Central, the organisation’s incident reporting system developed by the Department of Health and Human Services.
From the onset, the system was presented as a new way to manage feedback that needed staff to be actively participating in the system and encouraging consumers to tell their stories. Steps taken to achieve this included:
- meeting with staff and explaining why the organisation is using the platform
- training staff in the use of the site
- giving staff access to read stories
- recognising staff when they were commended in stories
- empowering staff to take ownership of the feedback and help formulate story responses
- discussing and using stories in staff meetings as learning opportunities
- sharing management reports on the pilot with front-line staff
- encouraging staff to ask consumers to tell stories
One strategy was to place a laptop in the staffroom with the Care Opinion stories widget on the home page. This displayed the latest stories told about Kerang. It was reported that the easy access to stories started many impromptu conversations between staff about the feedback. Another was to set light-hearted challenges to see which departments could generate the most feedback.
Kerang used much of Care Opinion’s functionality to engage with consumers about their feedback and get the most from the system, including:
- responding to stories promptly and honestly
- taking ownership in story responses when the care experience was not optimal, explaining what could and/or should have happened
- showing the organisation’s ‘human side’ through a conversational tone where appropriate
- making changes and service improvements based on stories at every opportunity
- posting follow-up responses to planned changes with updates to the plans
- providing responses from a member of staff closer to the care interaction and following up with a response from a more senior staff member, often the Chief Executive Officer.
Keeping the use of Care Opinion topical among staff, using positive and creative strategies for staff engagement and the senior executive team’s open endorsement of the system resulted in the system being understood, accepted, used and promoted across the organisation.
The Pilot Goals
As a small community with a population of approximately 10,550 people, there is a strong sense that ‘everybody knows everybody’. As a result, within Kerang, consumers were reluctant to complain out of fear of being identified and there was not a strong take-up of feedback forms or annual patient surveys.
Prior to using Care Opinion, positive feedback was not conveyed to staff. Negative feedback made it back to the frontline, where the complaint was reviewed by only the staff involved. Lessons learnt and changes of process in response to a complaint were not shared organisationally, resulting in staff not being enabled to learn from negative patient experiences. Further, staff reportedly considered feedback to be wearying and time-consuming to respond to, although this did vary somewhat across different departments.
What the data showed
Figures referred to here are drawn from reports generated from the Care Opinion platform on story data about KerangDistrict Health from 1 July 2018 to 31 December 2019.
While the overall response to the concept of receiving public, online feedback was generally positive, staff were described in survey results as feeling very nervous and daunted at the idea of the 'opportunity to receive negative feedback freely that anyone could see’, fearing they ‘would be criticised’.
However, the feedback that came through was overwhelmingly positive with 79 per cent of stories being assessed by moderators as purely positive. Of the 110 stories received, only one was assessed as being strongly critical and no stories were rated as severely critical (see Figure 1).
In total, 110 stories were published which have been viewed on Care Opinion Australia 32,124times in total.
Despite being an online platform, consumers can submit their stories by writing them down on a self-sealing, reply-paid Care Opinion leaflet. This proved a popular option with 78 per cent of consumers submitting their story through this mechanism (See Figure 1).
Figure 1: Methods of story submission
In questionnaires (refer to Appendix A: Staff questionnaires) respondents were asked if they had received critical feedback and how it had been managed. Despite receiving only one criticality 4 story and three criticality 3 stories (see Figure 2), respondents did feel they had received critical feedback, indicating that any negative content was considered criticism.
However, rather than feeling weary of feedback, the Quality Coordinator for Acute and Aged Care Services advised that it was seen as an “opportunity to explain our processes, to address the concerns of the consumer and have a real-time response to the critical feedback, share this not only with our staff but the community accessing Care Opinion.”
Figure 2: Breakdown of story criticality
This approach was clearly demonstrated in Kerang’s response to its criticality 4 story entitled Chest Pain. See Breakout Box 1 for details on how this story was managed.
Other survey respondents reported the critical feedback received was looked at objectively and reviewed for potential changes informing service improvement. The stories stimulated a formal response which front-line staff were involved in formulating and the development of a plan for future management of the situation/circumstances.
Responsiveness was a theme across all services within Kerang. Figure 3 demonstrates that of the 110 stories told, many storytellers received more than one response to their story. Figure 4 shows that 78 of the stories received a response within 24 hours of its publication.
Figure 3: Number of responses stories received
Figure 4: Number of days from publication until first story response
The data also reveals that the quality of responses was very high when assessed against the Plymouth Listen, Learn and Respond framework criteria (see Figure 5) which assesses:
- Responsiveness (if there is a response)
- Timeliness (mean time taken to respond)
- Identifiable authorship (responder profile picture and personal signature included)
- Helpfulness (responses marked as helpful by the storyteller)
- Specificity (responses flagged as the service planning to make a change or having made a change).
Figure 5: Response quality against the Plymouth University Listen, Learn and Respond Framework
Figure 6: Mean hours taken to respond to stories
The four most commonly used tags can be seen in Figure 7 (what aspects of care were good) and Figure 8 (what aspects of care could be improved). It is notable that the most common positive tag, being ‘care’ was used in 70 stories, whereas the most common negative tag, being ‘communication’ was used in only eight stories.
Figure 7: Top 4 positively polarised tags
Figure 8: Top 4 negatively polarised tags
The comparison is also demonstrated in Figure 9; a visualisation report that shows the most popular tags for your stories as bubbles. The bubbles are split according to how often the tag is used to say "what was good" (coloured green), or "what could be improved" (coloured pink).
They demonstrate that the positive tags were used significantly more often than negative tags. The proportion of positive tags, in conjunction with the 79 per cent of stories being purely positive likely informed the cultural change from one that tolerated feedback to one that embraced it.
Figure 9: Kerang District Health’s Tag bubbles for stories told during Care Opinion pilot
Case in Point - Kerang Medical Clinic
Stories told about one particular service, Kerang Medical Clinic, opened up strong opportunities to use the real-time data effectively, improve service delivery and explain its processes to the broader public.
Throughout the pilot, 11 stories were told about Kerang Medical Clinic, 55 per cent of which were rated a criticality 1 and raised issues regarding the phone system, staff workload and turnover, waiting times and appointment availability.
Figure 10 visually represents the data as to how stories told about the Clinic were heard or responded to. It shows that 42 per cent of stories told resulted in a change being made, a figure significantly higher than is commonly seen across services using Care Opinion Australia.
A common challenge to rural health organisations is the successful recruitment and retainment of suitably qualified clinicians; a challenge faced by Kerang. The Care Opinion platform enabled the organisation to communicate these challenges with the broader public, thus validating the frustrations consumers had been experiencing.
See Breakout Boxes 1 and 3 for example of how Kerang showed storytellers that their voices were being heard and were valued. This data can also be used as examples of evidence on how the service is partnering with consumers in the planning, design, delivery, measurement and evaluation of systems and services (Standard 2 of the National Safety and Quality Health Service (NSHQS Standards).
Figure 10: Visual representation of story responses
Discussion
This report sought to answer the questions of how organisations can actively invite online, public feedback within three set parameters. To achieve this, it assessed Kerang District Health’s experience with a pilot program of online, public feedback using the Care Opinion Australia platform.
To assess the success of the pilot, both qualitative and quantitative information from before and after the pilot was analysed, including story and response data drawn from Care Opinion Australia and respondents’ answers to survey questions.
When Kerang applied to participate in the pilot:
- patients were known to be reticent to provide feedback out of fear of identification
- feedback forms and surveys were not well utilised
- feedback was endured and perceived as a discouragement by staff
- patients did not know if their feedback made a difference
- there were few opportunities to evidence consumer engagement influencing organisational change.
Data revealed in reports generated through Care Opinion’s reporting system evidenced growth towards patients providing feedback, staff embracing feedback and feedback influencing organisational change through:
- the consistency of story submissions
- the take-up of reply-paid Care Opinion feedback leaflets
- responses to patients publicly commending staff and the care they provide
- patients valuing being told of the impact their feedback had by marking responses helpful
- stories activating change and improvements in service delivery.
Story and response content demonstrated staff taking full advantage of the opportunity to educate the broader public on its processes. It also showed that the telling of stories highlighted gaps in staff knowledge and the patient experience which enabled staff learning and training and service delivery changes and improvements.
To learn more about the impact of Kerang’s use of the Care Opinion Australia platform against each pilot goal, this case study refers to survey responses. Table 2 cites these responses that from anecdotal evidence of the goals having been met.
Table 1: Anecdotal evidence of pilot goals being met
Goal | Survey responses |
Increasing the volume and consistency of patient feedback |
|
Increasing transparency of feedback internally to enable opportunities for learning and service improvement |
|
Changing staff perceptions of, and engagement with, patient feedback. |
|
Survey respondents were also asked to identify the top ways they believe consumers have benefited from Kerang’s participation in the Care Opinion pilot. Table 2 cites answers to these questions according to different work areas in the service.
Table 2: Consumer benefits of Kerang pilot with Care Opinion Australia
Work Area | Consumer benefits of pilot |
Acute Ward |
|
Quality Office |
|
District Services |
|
Operating Theatre |
|
Finally, Kerang’s experience using online, public feedback is assessed against the three parameters defined in this case study.
Parameter 1: Consumers have a space to publicly share their experience without staff being "named and shamed".
The primary premises from which Care Opinion operates is facilitating feedback and showing that it can be given and responded to in a safe environment with conversations remaining constructive.
Despite initial apprehensions about using online, public feedback, staff grew to understand that no details pertaining to their identity would be revealed on the public platform when the story was negative. Through the moderation process, negative statements were attributed to being the storyteller's opinion rather than statements of fact, derogatory content was removed prior to publication and other members of the public could not enter the conversation.
This controlled environment enabled consumers to have a space to publicly share their experiences without staff being "named and shamed".
Parameter 2: Members of the public hear about positive care experiences, not only negative ones.
While people may lean towards telling others about negative interactions more than positive ones, the public nature of the platform ensured that the public had equal access to read about positive interactions.
The 110 stories published about Kerang District Health during the pilot received 32, 124 total views. With 79 per cent of these stories being purely positive, this pilot publicly showcased the organisation's high-quality staff and delivery of care to the broader community.
Parameter 3: Organisations can get tailored messages out to the community, such as how they react to feedback and what they do about it.
It is impossible for organisations and their staff to always get everything right. However, when service was not delivered as it should or could have been, Kerang crafted their responses to educate not only the storyteller but the broader public too. Processes were explained, planned action outlined and patients invited to reach out to relevant individuals within the organisation so they could take steps to set things right.
The responses published on Care Opinion were honest and very transparent and showcased the human side of the organisation. Staff were revealed to be individuals who sometimes make mistakes, have bad days and do not always know everything they need to know. They were also revealed to be caring, kind and dedicated people, for whom a "well done" by those they work so hard to deliver quality care to, goes a very long way and means a great deal.
Conclusion
While feedback forms and surveys have their place, they are typically confined to standard, predetermined questions, limit the amount of free-form text provided, categorise feedback into a compliment or a complaint and include rating systems that can be filled in without providing any contextual information.
Qualitative feedback through stories enables staff from all hierarchies and all departments to learn about patient experiences without the confines naturally built into surveys and feedback forms. Care Opinion’s criticality rating recognises that patient experiences are often multi-faceted; storytellers can give a compliment and complaint via the same mechanism, or simply tell of their experience being neither a compliment nor complaint.
Throughout the pilot, consumer feedback increased in volume and transparency, within the safety of the moderation process, ensuring all parties remained safe in the online sphere.
The pilot also resulted in staff being made aware that their dedication and care was recognised and valued by the community, thus improving staff morale. This also benefited the organisation as members of the public were also made aware of the high-quality care Kerang provides, and the efforts extended to improve that quality on a continual basis.
Inviting feedback through an online, public platform ensured that the public also heard Kerang’scarefully crafted messages, tailored to demonstrate how it both welcomes and responds to consumer feedback.
The report concludes that organisations can actively invite online, public feedback when it is managed through a controlled environment with system functionality that enables the public to hear two sides of a story, rather than just one party’s perspective.
It shows that frontline staff need to be made aware of all feedback, positive and negative to be able to learn, grow and reap the rewards of the care they delivery. Such transparency not only enables internal growth and maturation but enables the community to better understand and perceive health organisations as groups of people who listen, care, respond and are willing to partner with them on their journey of continual improvement.
Recommendations
The report recommends that organisations looking to implement Care Opinion emulate a similar approach to Kerang’s implementation of the platform sing five primary approaches:
- 1. Communicate the Senior Executive Team’s team endorsement of the platform to all staff
- 2. Actively foster staff engagement with the system
- a. Educate and train staff about the platform
- i. how to use it
- ii. why it is being used
- iii. what the goals of its use are
- iv. how they will remain safe in the public domain
- v. how they will benefit from the feedback
- a. Educate and train staff about the platform
- 3. Keep the platform topical among the workforce
- a. grant staff of all levels access to read stories
- b. involve staff in the formulation of responses
- c. recognise staff and departments when they have been personally commended in stories
- d. motivate staff to encourage consumers to tell their stories
- e. treat feedback as learning opportunities
- 4. Engage with consumers about their feedback
- a. invite them to tell their stories whether they be a complaint, compliment or both
- b. respond promptly and honestly
- c. acknowledge when the care could have been better
- d. explain what will be done with their feedback
- e. have a person close to the care experience respond to the story (e.g. Nurse UnitManager)
- f. follow up on planned changes
- 5. Use consumer feedback to drive care change and improvements
- a. use stories to identify training/learning gaps
- b. embed stories in staff training materials
- c. discuss stories in front-line staff meetings and senior executive meetings
- d. look for opportunities to review processes, train staff, improve facilities and improve internal and external communications.
Appendices
Appendix A: Staff Questionnaire after Pilot’s Conclusion
General Questions
What is your role within KDH?
What is your typical patient/consumer demographic?
Joining the Pilot
How would you describe your feedback culture prior to joining the Pilot?
Had KDH actively sought patient feedback prior to joining the pilot? If so, how?
What has been involved in using Patient Opinion? Has it involved a lot of work?
Have other staff members been involved in this?
Your Experience with Patient Opinion
How did staff initially react to the concept of patient feedback being published online and how do they view it now?
How are you currently using Patient Opinion?
How do you make sure patients at the clinics know they can leave feedback?
Have you had any critical feedback? If so, how have you handled it?
What are the top three changes you have noticed within the staff culture since you have been using Patient Opinion?
What are the top three benefits you believe patients have experienced from KDH using Patient Opinion?
What are the most common comments you hear from staff about Patient Opinion and/or the stories received?
What have you learnt from using Patient Opinion and what would you say to other health services considering subscribing?
Appendix B: Moderation Criticality Table
Appendix C: Staff learnings and their recommendations for health services considering online, public feedback
1 Atmiller, G. The role of constructive feedback in patient safety and continuous quality improvements. NursingClinics of North America. 47, 3:365-374. (2012). doi: 10.1016/j.cnur.2012.05.002
2 Robert G, Donetto S. Whose data is it anyway? Patient experience and service improvement. Journal of Health Services Research & Policy. 25, 3:139-141. (2020). doi:10.1177/1355819620921423
3 Reeves, R, West, E & Barron, D. Facilitated patient experience feedback can improve nursing care: A pilot study for a phase III cluster randomized controlled trial. BMC Health Services Research. 13, 259 (2013).https://doi.org/10.1186/1472-6963-13-259
4 Radmore S, Eljiz, K & Greenfield, D. Patient feedback: Listening and responding to patient voices. Patient Experience Journal. 7, 1:13-19. (2020). doi: 10.35680/2372-0247.1370