By Guest Blogger:  Darren Frizzell

How can we use BC PSLS data to drive organizational change and help healthcare providers who may be drowning in mountains of work? This is my focus and the main question I have been striving to answer during my time with BC PSLS Central Office.

Here are some interesting facts about BC PSLS and patient safety:

• Over 100,000 healthcare providers across the province have access to BC PSLS to report patient safety events and concerns
• Healthcare providers report more than 100,000 safety events and hazards to BC PSLS every year
• 10,000 healthcare leaders are “Handlers” who use BC PSLS to follow-up on events at over 1,500 facilities across BC

Figure 1: Event Reporting Volume since 2008/2009

As shown, the number of events reported to BC PSLS has steadily increased since our first Go Live in 2008. Some of this volume reflects implementation efforts, but the trend has continued since full provincial use was achieved in mid-2011. With this growth, BC PSLS Handlers are increasingly in need of quick and easy access to information specific to their clinical areas. Without access to accurate, aggregate BC PSLS data, they may end up relying on personal opinions or even “luck” to detect and fix problems rather than following the “data”.

Detecting a “Black Hole”

One of the drawbacks of paper-based incident reporting systems is the tendency for reports to end up in a “black hole”, where it is unclear to reporters whether anyone has read the reports, followed up on them, or acted on them. In the early days of planning BC PSLS, using an electronic reporting and learning system to eliminate the black hole by making all parts of the reporting and investigation process visible was a key objective, and the software helps support this goal by assigning different statuses to event reports as they move through the follow-up process. But…

…as I worked on analysis of the BC PSLS database, I started to notice an increasing gap between the number of events reported and the number with “Final Approval” status, meaning that event review has been completed and the comprehensive data is available for analysis. Then one day, I found a safety event that had not yet been reviewed – 575 days after it was reported! I then searched to see if there were other events that had not been reviewed for over one year. I found thousands of records still waiting to be reviewed. I had discovered a “black hole”! It was beginning to look like we had just shifted from a paper-based black hole to an electronic one, with many event reports piling up in the database instead of on desks.

Timely follow-up is important for a number of reasons. Healthcare providers who report safety events to BC PSLS trust that leaders will respond quickly, so Handlers across the province are encouraged to promptly follow-up on events and to “close the loop” with Reporters. Investigating an event as soon as possible after it has happened also offers the best opportunities for learning while it is still “fresh” in the minds of those involved. Boards and the public expect that dangers and harm identified through reporting will be assessed and actions taken to mitigate the risk of recurrence. 

In order for data to be included in HA analyses, event review must be completed and the report assigned “Final Approval” status, otherwise those reports are excluded and our picture is incomplete. When we talked to our Handlers about why event reports were piling up awaiting review, they told us they sometimes felt overwhelmed with the volume of events reported, were unsure what proper follow-up entailed, and were unclear about timelines for completing investigations. None of them had a clear picture of where they stood with their review timelines and processes. So, we wanted to help!

Bubbles Close the Black Hole!

In 2011, we launched our BC PSLS Analytics website, which included dashboard reports and a new tool called “Explorer.” Right away, we could see that Explorer was going to be popular. Users were amazed at how fast and easily they could answer questions about BC PSLS events. In the background, we determined a model to track how events went through various stages of approval. This model allowed us to peer into and examine the black hole. A simple bubble chart emerged as a powerful yet easy-to-use method to visually show complex calculations about safety event workflow and let Handlers “see” how they were doing with their review processes and timelines.

How to Read Our Bubble Charts (correlation charts):

• Each bubble represents an organization
• The horizontal X-axis  = Average Days to Final Approval
• The vertical Y-axis = Average Days to First Review
• Bubble size = Number of events reported

The goal is to see your organization’s “bubble” in the bottom left corner. This placement means that events undergo first review quickly and are then followed up and given “Final Approval” status within a reasonable timeframe.

Figure 2: 2011/12 Bubble Chart

As shown in Figure 2, in 2011/12 some organizations had an average of over 40 days to first review event reports and close to 100 days to complete review and give Final Approval. The largest bubble on this chart represents approximately 25,000 records and the smallest bubble represents approximately 5,000. What pops out very quickly in this chart is that the number of events reported does not directly correlate with average days to approve.

We launched a push across the province to make this problem visible: to draw attention to the fact that many events were not being reviewed within the established timeframes. Our dedicated team of BC PSLS Coordinators within the Health Authorities engaged people in their organizations – from CEOs and executive leaders to nurse managers and educators on the front lines – to raise awareness. We wanted to utilize both  top-down and bottom-up efforts so everyone was working on this problem together.

One approach that was effective in helping streamline the follow-up process for Handlers involved events that are reported as having caused “no harm” to patients, which represent about 70% of the events in the BC PSLS database. These events require little individual follow-up but can be powerful when viewed in the aggregate and used to identify opportunities for quality and process improvement. Encouraging Handlers to promptly and minimally review these event reports and assign “Final Approval” status, with a plan to track and trend them using analytics, freed up time for them to focus on the more time-consuming work associated with less frequent complex safety events and those involving harm.

One-on-one support for Handlers who had amassed a significant backlog was also offered in some cases, and changes were made to the follow-up form in BC PSLS to help clarify processes. As the focus on the importance of responding to reports in a timely manner increased, progress began to be made.

Figure 3: 2012/13 Bubble Chart

One year later, we saw a huge move towards the bottom left corner! Notice that the X-axis maximum is half of 2011/12 and that no organization is over 60 days, where last year only 2 organizations were below 60 days. Again, we see the size of the bubble (number of events reported) does not directly relate to days to Final Approval. In fact, the two organizations with the highest reporting volume had the shortest average time to completion.

Figure 4: 2013/14 First Quarter Bubble Chart

In the first quarter of this fiscal year, all organizations have bubbles under 30 days! This is remarkable progress since we know that reporting volume has continued to increase. What’s more, the bubble with the fastest approval time in 2011/12 has also cut their average time to approve in half.

Timely completion of review processes means that more comprehensive data is available in the BC PSLS database for analysis. Now, here is the hard part: We are approaching 500,000 records in BC PSLS. How do we avoid another potential black hole, where data goes in but isn’t easily retrieved in a meaningful form? How do we best learn from these events and use the information to improve patient care?

Stay tuned for Part 2!

Darren Frizzell is Business Development & Information Systems Analyst at BC PSLS Central Office. His work focuses on using analytics for process management and quality improvement. To learn more, please contact Darren Frizzell dfrizzell@phsa.ca and follow him on twitter @BCPSLS_Darren