How do we know if we are providing high-quality care? The answer to this question is sought by a multitude of parties: patients, clinicians, educators, legislators, and insurance companies. Unfortunately, it’s not easy to determine. There is no single score or report that provides a definitive benchmark of quality, but various measures can help paint the picture. One of the measures garnering a lot of attention is outcome tools.
Outcome tools provide value in many areas. We can use them to:
Outcome tools have practice-wide benefits as well. At PT Solutions, for example, we use our outcomes data to:
Imagine how much influence we could have if the whole profession joined together and used outcomes data with a unified approach?
While outcome tools are valuable and definitely represent a piece of the care quality puzzle, they are often met with resistance by clinicians. Outcome tools can elicit negative emotional reactions, as clinicians may feel attacked and scrutinized. As Danielle Ofri wrote in What Doctors Feel, “the desired practical outcome is smothered by the emotion cost.” While this is certainly true if the outcomes are treated as the only indicator of clinical quality, writing them off completely can remove objective data and accountability from the care equation.
Unfortunately, we are bound to be heavily biased in our assessments of care quality. I experienced this challenge at PT Solutions, where I serve as the National Director of Quality and Research. In 2015, we rolled out Focus on Therapeutic Outcomes (FOTO), a system of outcome measures, to track clinical outcomes and patient satisfaction. Unfortunately, it was not a smooth undertaking.
I immediately sought to discover and correct the issue, as I knew we gave high-quality care. While recognizing we always have room for improvement, the results we were obtaining were not consistent with our other quality metrics, such as cancellation rates, online reviews, and patient-generated referrals.
What I found is that the rapid rollout of FOTO resulted in a poor understanding of how to use the tool. Clinics were failing to complete surveys at regular intervals, especially on or near discharge. This resulted in a large “days between status and discharge” value. We treated patients for 10 visits, but the final score obtained was on visit six. Essentially, we were treating patients and improving them without getting proper credit in our outcome scores. I spent months beating the drum that the tool was being used incorrectly. Enter the law of unintended consequences.
By placing a large emphasis on our incorrect use of the tool, I created a narrative that quickly became a convenient excuse when results plateaued—any poor outcomes or satisfaction values from that day forward were attributed to an issue with the surveys themselves. At this point, I started seeking alternative explanations and strategies to improve our outcomes processes.
I started researching and reading about decision-making and critical thinking. Soon, I stumbled across one of the most influential books I have ever read. Thinking, Fast and Slow by Daniel Kahneman opened my eyes to the world of biases and cognitive fallacies. The pieces fell into place, and I started to understand many of the issues impeding the use of quality data.
Now, I’ll walk through the key cognitive fallacies our practice has encountered in our attempt to improve our use of outcomes and satisfaction data. I have fallen victim to each of these and can attest to their power. The goal is not to abolish these biases—after all, it’s impossible to rid ourselves of them completely—but rather to generate awareness so we can more effectively recognize and navigate these cognitive traps.
The first bias to address is arguably the most common. Confirmation bias is when people seek data that are likely to support the beliefs they currently hold. We focus on what we know and neglect what we do not—which makes us overly confident in our beliefs. In the clinical world, this commonly occurs when supporting our treatment decisions. A quick PubMed search often reveals a host of studies supporting my viewpoint. So, I just conveniently ignore the ones refuting it. We see similar approaches regarding quality metrics.
As clinicians, we will naturally defend our treatment decisions and clinical quality. We don’t go into work thinking, “Today my aim is to be exceptionally mediocre at treating.” Therefore, when looking at quality data, we are likely to focus on the information that supports our current beliefs: “I don’t care what FOTO may say; look at how low my cancellation rate is!” This is exacerbated when adding new measures—like our practice did with the rollout of FOTO—as it invites additional cognitive fallacies.
Theory-induced blindness, a term coined by Daniel Kahneman, essentially means that once you have accepted a theory and used it as a tool in your thinking, it is extraordinarily difficult to notice its flaws. Our theoretical beliefs are robust, and it takes much more than one embarrassing finding for established theories to be seriously questioned. If I have accepted certain quality measures (like the cancellation rate or the number of patient-generated referrals), I am more likely to cling to those. This remains true even if new data comes to light that calls into question previous beliefs. We see this when clinicians fail to adopt new treatments or assessments because they have had success with previously accepted approaches. Changing how we conduct and assess our job is uncomfortable and difficult. This is exacerbated if we are working with limited information.
Compounding the issues of confirmation bias and theory-induced blindness is a failure to procure all relevant information. It’s very easy for us to cherry-pick information, both intentionally and unintentionally. The availability bias is the process of passing judgment by the ease with which instances come to mind. If we constantly assess our cancellation rate or our visits per referral, then those numbers will more easily come to mind. If practice leaders frequently refer to those same metrics, then their perceived importance will increase. Another piece of the availability bias is knowing when to draw conclusions.
I employed an availability bias when I was hyper-focused on the FOTO data for one of our flagship clinics. Following months of training and no discernible improvement in the outcomes data, I was certain there was an issue in the quality of care. However, once I looked at the entire picture and recognized our great cancellation rate, visits per referral, self-discharge rate, patient-generated referrals, Net Promoter Score® (NPS®), online reviews, and patient satisfaction numbers, I had to concede that clinical quality was not a concern. As it turned out, a single clinician was artificially impacting the scores by prematurely closing cases in FOTO to avoid the hassle of collecting and monitoring outcomes data. So, we must always ask ourselves, “Do I have all the information necessary to draw a conclusion?”
The second issue we faced regarding available information was the halo effect. The halo effect is when we increase the weight of first impressions, sometimes to the point that subsequent information is mostly wasted. It can affect subsequent performance assessments. How does this pertain to outcomes?
An example of the halo effect is assuming all clinicians in your practice—simply because you hired them—provide high-quality care, even if you have not seen them treat or reviewed their metrics. Even as we gain more information through co-treating and training, the initial conclusion drawn during the hiring process heavily influences our perception. Assuming we are great and the tool is wrong will stunt clinical development.
As you will notice, this progression is akin to the five stages of grief. Let’s now explore three biases that comprise the search for excuses. The first is the sunk-cost fallacy. This occurs when individuals continue a behavior or endeavor as a result of previously invested resources (time, money, or effort), regardless of the outcome. In rehab therapy, we often see this when clinicians cling to a particular treatment approach—regardless of the efficacy and literature support—because of previously invested resources. It can blind our acceptance of outcomes, as we assume the quantity of resources invested should be proportional to the quality of care.
The second fallacy is the outcome bias. The outcome bias occurs when we blame decision-makers for good decisions that worked out poorly and give them too little credit for successful moves that appear obvious only after the fact. For example: “My cancellation rate is great because I have implemented more open and frequent communication with patients. The poor satisfaction score is outside my control. I have had a large influx of Medicaid patients and my new clinician is struggling.” Or perhaps: “Of course Sarah has great outcomes; her clinic is in an affluent area, and all of her patients are young, athletic, and seeking care during the acute stage. Meanwhile, my clinic is in a different part of town, and all my patients are in chronic pain and have been using opioids.” While these facts may be true, there is a lack of ownership, and no actionable tasks are being developed to address the outlined issues.
Lastly, we have the loss aversion fallacy. Loss aversion is a powerful conservative force that favors minimal departure from the status quo. We experience stronger regret if a poor outcome results from action rather than inaction. For example: “I knew I shouldn’t have discussed recent cancellations with my patient. Now they are uncomfortable and won’t come back. I should have stayed quiet.” We are more prone to seek excuses and reasons to prevent action than to seek it. When presented with poor outcomes or satisfaction data, we are less likely to implement strategies to improve the results because of loss aversion.
At this point in my journey, buy-in started to take hold, but the results were inconsistent and region-specific. Ultimately, we found that culture had a substantial influence on results. The regions that were more stable, more open and honest with feedback, and more driven by internal competition consistently outperformed the others. Now that we had sufficient training in FOTO usage and cognitive bias awareness, we needed to determine which tool to use for promoting improved performance: carrots or sticks.
The research is clear that reward for improved performance works better than punishment for mistakes. Note that I say reward and not financial incentive. The evidence is mixed regarding the effectiveness of financial incentives compared to other types of rewards, which could include:
However, the question is not simply a matter of how large a bonus the high-performing clinics should receive; it is whether the focus should be on the high-performing clinics or low-performing ones.
Some of you may disagree with focusing on reward, citing your experiences with the opposite effect, but that is often a result of the regression to the mean phenomenon. If someone exceeds expectations, future results are likely to be relatively worse, despite the initial praise. The reverse is seen with unexpected dips in performance and subsequent punishment. We need to assess trends before drawing conclusions on the effectiveness of our interventions.
Once the biases have been addressed and the method of feedback has been established, it’s time to improve our outcomes assessment process and plan for the future. We need to be both reactionary and proactive. An interesting consideration is the difference in motivation between meeting goals and exceeding them.
We work a lot harder to meet goals than to exceed them. One is the difference between success and failure while the other is the difference between success and, well, more success. We saw this with our quality data. If we meet the goal of 98% patient satisfaction, it is no longer discussed. We no longer develop plans to improve it further—despite 2% of patients remaining unhappy—and we instead focus on other issues. If the goal is not a stretch goal—something difficult but still attainable—we often fall short of our potential. Additionally, we will have a handful of clinics simply meeting the goal and a handful missing the mark. So, on average, the group is still below the goal.
“Using distributional information from other ventures similar to that being forecasted is called taking an ‘outside view’ and is the cure to the planning fallacy.” – Bert Flyvbjerg
The planning fallacy was the last major hurdle we needed to clear while reshaping our culture on clinical outcomes. We needed to shift from strictly reacting to data from past treatments and start looking forward to proactively address potential concerns. The planning fallacy is the tendency to underestimate potential complications and time needed for future projects. This often occurs in patient care. We may not consider the impacts of a patient’s diet or sleep hygiene. We may not account for the influence of understaffed front offices or the time it will take a new graduate to assume a full caseload. To combat this fallacy, we need to aggregate multiple perspectives.
Aggregating assessments adds to the collective pool of information. This allows the decision-maker to take “the wisdom of the crowd” and formulate a more well-informed opinion. This can be accomplished at the treatment level and at the overall practice assessment level. Ask your colleagues to review a recent evaluation to look for any potential missed data, incorrect assessments, or likely future pitfalls in the prognosis and plan of care. Work as a leadership team to assess all relevant data and bring your individual perspectives to determine why a clinic or region is struggling with clinical outcomes. Regardless of the level of the issues—be it individual patient care or clinic-wide—look at the big picture. Looking at cases in isolation leads to more quick, emotionally-driven reactions.
As Orfi pointed out in her book, outcome measures and similar quality metrics don’t provide a full understanding or measure of quality. They provide valuable information, but we cannot determine if quality care was provided solely from the outcomes data. The same can be said for patient satisfaction measures. Emotion and bias can influence a patient just as it does a clinician. For example, even if a patient’s primary issue is a long wait time prior to the appointment, the patient is likely to give a 1-star rating across the board, when the wait time has no influence on the care provided.
In my role, quality assessment is a key responsibility. Previously, I homed in on problems and expected excellent results. I also heavily weighed outcomes data and assumed that if the numbers were poor, then the care was poor. Through reflection, feedback, and assessment, I now have a more refined understanding of the value outcomes and satisfaction data provide. I also realize I was previously mistaken in my weighing of their value, and that I caused undue stress, frustration, and potentially even shame among my colleagues. We must hold one another accountable to high-quality care, and we have a variety of tools to make assessments of said quality. But we must also ensure we do not pull each other down when making those assessments.
Outcome tools can provide substantial value for your practice and the profession. While we may not solely rely on outcomes to tell a story, we can maximize their value by using the tools correctly and obtaining employee buy-in. The biases I covered are not all cognitive fallacies you will run across in clinical care, but they are highly prevalent. As stated earlier, the goal is not to eliminate them, but to have an awareness and to navigate potential cognitive traps. It will take practice. There will be struggles and resistance, but it will be well worth the effort.
Dr. Zach Walston, PT, DPT, OCS, serves as the National Director of Quality and Research at PT Solutions. He currently oversees research efforts at PT Solutions, develops and teaches continuing education courses, and creates quality improvement initiatives throughout the practice. Zach also serves as the Program Coordinator of the PT Solutions Orthopedic Residency Program and the director of the Clinical Mentorship Program. Connect with Zach on Twitter (@zachwalston) and LinkedIn.
The post Why Cognitive Bias is Ruining Your Clinic’s Outcomes Data Quality—and How to Fix it appeared first on WebPT.
Did you know that SEED SPOT’s alumni network encompasses more than 1,000 members throughout the country? Moreover, 81% come from underrepresented communities and 51% are female entrepreneurs, and these founders have created 2,385 jobs and raised over $53.3 million of capital.
Every SEED SPOT alum is provided with an abundance of resources to support you through every step of your social impact startup journey. Whether you’re fresh out of the 2-Day Launch Camp where ventures start or explore a new impact business venture, or through the eight-week Impact Accelerator, where ventures are prepared and ready for traction for scale and funding, SEED SPOT has your back!
So, You Want to Be A SEED SPOT Alumni?
As an alum, you are instantly emerged in the private SEED SPOT Community Slack Group where you gain access to the entirety of the alumni network and resources. Additionally, alumni receive lifetime benefits that include membership to a national mentorship database full of industry and subject matter experts, invitations to weekly founder scrums, access to pro bono legal services, up to $100,000 in Amazon Web Service credits for secure cloud computing, and so much more! Feel free to email firstname.lastname@example.org for additional alumni benefits and check out a few members of the alumni network directory here.
Within a vast variety of sectors, these diverse, visionary, and social-driven entrepreneurs are persevering through this pandemic, highlighting immense grit and a whole lot of pivoting. As these programs continue to support them through the variety of benefits offered as a SEED SPOT alumni, this series will highlight a few incredible alumni entrepreneurs through a multi-part series where we discuss key business wins, their social impact, and ways our community can continue to support their journey.
Calling All Entrepreneurs
As SEED SPOT continues to support diverse entrepreneurs through these difficult times, follow along in the next couple of months to learn more and celebrate their accomplishments and successes in a vast array of industries and services.
The post Shining Light on SEED SPOT’s Diverse Entrepreneurs appeared first on SEED SPOT.
With your updated sick leave policy—and the current hypervigilance around preventing the spread of illness—there’s a good chance that you may see more employees using their sick days over the next several months. And that’s a good thing. Whether you send home an employee who has the sniffles, one of your therapists calls out because he or she is keeping a child home from daycare, or someone on your staff needs to self-quarantine for the full 14 days, everyone should be able to care for themselves and their family members. But that doesn’t necessarily mean that, operationally speaking, the process for covering patient appointments is easy—which is why you’ll want to have a plan in place ahead of time. With that in mind, here are four strategies to consider when deciding how to handle a staff therapist calling in sick:
Depending on your region, you may find that it’s easier than usual to tap temporary or per diem PTs, OTs, and SLPs to fill in for staff therapists on a moment’s notice. And that can prove incredibly useful, especially if you’re already operating at capacity with your current staff members. That said, unless you meet the limited requirements for what was previously referred to as “locum tenens billing,” then you’ll need to be sure the fill-in therapists are credentialed with all of the insurances you accept—or, at the very least, those for the patients he or she will be seeing.
As WebPT Co-Founder and Chief Clinical Officer Heidi Jennenga, PT, DPT, ATC, notes here, the best way to ensure all temp staff credentials are on the up and up is to hire temporary staff through a “qualified agency with vetted insurance credentials.” While “this may be a slightly more costly way of doing things compared to simply hiring a friend or associate,” Jannenga believes this is still “your best choice.” Otherwise, you risk not being able to receive payment for your services.
Having your other current—and healthy—staff members pick up the slack for your absent therapist might be more feasible if your practice is just beginning to reopen, considering that some members of your team may not have full caseloads yet. If that’s the case, then spreading the work around your team is likely the easiest path forward. As a first step, consider shifting regular patient visits to therapy assistants and evaluative appointments to other therapists. While all supervision requirements remain in effect—and said supervision must be provided by a licensed therapist—it doesn’t necessarily mean that the supervising therapist must be the one who conducted that patient’s initial evaluation (or even the one who supervised the patient’s last appointment). Instead, this is a perfect opportunity for everyone to pitch in.
If the therapist who called out did so as a precaution—as opposed to because he or she is actually sick or caring for a sick family member—then you could consider having that person work from home and schedule telehealth visits with his or her patients. This becomes even easier if you’ve already communicated the benefits of telehealth to existing patients—and established processes and tools to conduct therapy sessions virtually. That way, the groundwork has already been laid and doesn’t need to be handled on the fly. This option will likely work best in conjunction with one of the other options above, as some patients may not be ideal candidates for telehealth—and some may simply prefer to reschedule.
While not ideal by any means, rescheduling patient visits may be the only realistic way to ensure your team can provide excellent care to your patients—while keeping workloads at a manageable level. For longer absences, you’ll hopefully only need to do this for the first day or two, max—just until you can implement a longer-term strategy. If you have a solid plan in place ahead of time, you may not need to reschedule at all. Depending on your clinic setup, however, you may need to let patients know that their scheduled appointment will be with a different therapist or therapy assistant than they’re used to. Otherwise, you risk negatively impacting the patient experience. This might be a good reason to encourage patients to move between staff members regularly, so these types of last-minute transitions don’t feel jarring. Also, as a reminder from Veda Collmer, WebPT’s in-house counsel: You’re not permitted to share any health-related information about an employee, as that is a violation of workplace privacy laws.
Giving staff therapists the flexibility to stay home when they—or their family members—are sick is crucial to employee satisfaction and, particularly in the era of COVID-19, the health and safety of everyone in your clinic. And having a backup plan in place can help make absences less stressful for everyone.
The post What to Do When a Staff Physical Therapist Calls in Sick appeared first on WebPT.