Print
E-mail
Reprint

alt

 

Patient satisfaction has become the latest catchphrase throughout hospital emergency departments. Many hospital administrators are under pressure from hospital boards to improve patient satisfaction scores and CMS has indicated that patient satisfaction scores will impact reimbursement to hospitals. Given that patient satisfaction is poised to become an integral part of health care delivery in this country, we decided to look at some of the potential drawbacks to relying on patient satisfaction scores.

We chose to review the data collection and reporting methods of Press Ganey Associates, Inc. for this article. Press Ganey partners with roughly 40% of hospitals in the United States - including more than 10,000 health care facilities - to measure and improve quality of care. Part of Press Ganey’s business model includes sending surveys to patients who have visited a hospital asking them about their impressions of the facilities, the staff, and the physicians. This data is then analyzed and forwarded to participating hospitals. Hospital use the Press Ganey data to judge not only the quality of care being provided in different hospital departments, but also to compare their hospital to other hospitals within the Press Ganey database. In some cases, hospitals even attempt to compare survey data for specific physicians. Even though the surveys are purported to improve the quality of patient care, there are several things you may not know about the survey calculations and their effects upon patient care.

The sample size may create unacceptable margins of error - but the survey results don’t tell you that
Press Ganey has stated that a minimum of 30 survey responses is necessary to draw meaningful conclusions from the data it receives and that it will not stand behind statistical analysis when less than 30 responses are received. Despite this statement, comparative data still gets published about hospital departments and about individual physicians when less than 30 responses are received. For example, Dr. Sullivan’s hospital receives approximately 8-10 Press Ganey survey responses per month. Even with this small sample size, Dr. Sullivan’s hospital still receives monthly reports from Press Ganey analyzing the data. During one month, Dr. Sullivan’s emergency department ranked in the first percentile within Press Ganey databases. Two months later, his emergency department ranked in the 99th percentile. How did they do it? Actually, any actions their group took probably made little difference in the subsequent survey data. By the time they were able to take action, some of the data had already been collected for the subsequent month -- in which his group received accolades for their excellent satisfaction scores. Which percentiles were representative of their emergency department’s performance? Probably neither. The small sample sizes just created unreliable data upon which the conclusions were based.

The time you spend with critically ill patients may make another department’s satisfaction scores better . . . while making yours worse
Many studies have shown that the time a patient spends waiting for medical care is inversely proportional to that patient’s satisfaction with the visit. Suppose that a patient is brought by ambulance in respiratory distress. After nebulizer treatment and BiPAP fail, you have to intubate the patient. Then the patient’s blood pressure drops. You start inotropic medications, initiate antibiotics, and actively manage the ventilator settings. After an hour and a half of work, the patient is stabilized. You then spend another 30 minutes discussing the patient’s condition with family members, contacting consultants, and writing admission orders. How will the outstanding medical care that you provided affect your satisfaction scores? If anything, your satisfaction scores may drop due to all of the patients who graded you lower because they had an excessive wait while you were busy saving a life.

Patients admitted to the hospital and patients transferred to other hospitals do not receive Press Ganey emergency department satisfaction surveys. While some questions about the emergency department may be included on inpatient surveys, the answers to those questions count toward the inpatient satisfaction scores, not the emergency department satisfaction scores.

The pressures to improve emergency department satisfaction scores may create a significant dilemma with emergency department staff. An online survey of 717 respondents performed by Emergency Physician’s Monthly on its medical blog “WhiteCoat’s Call Room” showed that more than 16% of medical professionals had their employment threatened by low patient satisfaction scores. In addition, 27% of respondents stated that their income was in some way tied to satisfaction scores.

When faced with a decision between improving satisfaction scores and unemployment, a clear -- and potentially deadly -- conflict of interest occurs. Should emergency physicians and nurses provide appropriate yet time-consuming medical care to high acuity patients or should they provide a minimal amount of medical care to the sickest patients so that they can focus more attention on patients who will be completing satisfaction surveys? Sometimes, especially in single-coverage emergency departments where staffing has been cut due to budget constraints, “doing both” may not be an option.


Patient satisfaction data is not random
Did you know that Hillary Clinton won the Democratic presidential nomination in 2008? Really, she did. A random sample of voters from Pennsylvania showed that she was the clear winner. Failing to fully randomize data can adversely impact even a large survey’s conclusions to the point that those conclusions become invalid. As in the election example used above, Press Ganey’s data are not random and are not representative of an emergency department’s patient population.

We already know that Press Ganey’s satisfaction surveys exclude admitted and transferred patients, which creates a significant bias toward low acuity patients. Emergency departments with a large percentage of admits may have lower satisfaction scores solely due to the decreased survey sample pool and to the increasing wait times encountered by low acuity patients while staff is trying to stabilize higher acuity patients.

Another source of non-randomization in Press Ganey’s patient satisfaction data is that patients who leave without being seen will not receive a satisfaction survey. In addition to decreasing the randomness of the sample size, such a bias could create an incentive for staff to encourage unhappy patients in waiting rooms with non-urgent complaints to leave the hospital emergency department without treatment.

Yet another bias against random samples in Press Ganey’s patient satisfaction surveys is that by default, patients can only receive a satisfaction survey every 90 days. While the intent of this limitation is evident - to keep “frequent flyers” from skewing data - the effect is to decrease the randomness of the data ... and to further limit the data’s reliability.

Press Ganey has stated that “external validity requires that you only draw conclusions from the patient population that you are sampling.” However, the reports that Press Ganey generates draw conclusions from a sample of non-admitted patients who have not been treated in 90 days and who have actually been seen by a physician in the emergency department. Instead of limiting the conclusions to this subset of patients, Press Ganey applies its satisfaction scores to “emergency department” as a whole a group much larger and more diverse than the patient population being sampled.

The lack of randomization in Press Ganey data samples was recently highlighted during a press relase regarding emergency department wait times. Press Ganey reported that its 2009 data showed Utah emergency department patients had an average length of stay of 8 hours and 17 minutes, noting that the wait was the worst in the country and calling the wait “staggering.”

Utah ACEP then investigated the claims and discovered that Press Ganey had limited access to data  from 65% of all the emergency department visits in Utah. When Utah ACEP reviewed data on 80% of emergency department patients from 2009, it found that the average length of stay in Utah was three hours and 29 minutes – far shorter than Press Ganey’s allegations, and actually ranking Utah in the top 15 states for emergency department throughput.

“Response errors” may dramatically affect survey results
According to the book Asking Questions: The Definitive Guide to Questionnaire Design (Jossey-Bass, 2004), there are four basic factors related to response error: memory, knowledge, motivation, and communication. Each of these has a significant effect on patient satisfaction survey data.

For example, the time lag between a patient’s emergency department visit and the receipt of a survey in the mail may affect a patient’s memory of occurrences in the emergency department.

Patients who are asked to rate the medical skill and quality of physicians or nurses, who are asked to assess the skill with which phlebotomists take blood, or who judge whether medical personnel “took their problem seriously” often have little knowledge upon which to base their assertions.

Patients who are unhappy due to an excessive wait or because they did not receive requested medications may be motivated to show their unhappiness by grading all aspects of their care low, even when most aspects of the care they received were exceptional. Dr. Eric Armbrecht, a statistician and Assistant Professor for St. Louis University’s Center for Outcomes Research echoes this concern, noting that many survey respondents will simply mark the same response throughout all the answers to a survey. He stated that, in general, those who respond to surveys are either very satisfied or are very unsatisfied and want to make a point. These responses tend to cause a “bimodal distribution” with peaks at either end of the scale.
When the problem of secondary motivation and response error was discussed with Press Ganey representatives, they acknowledged that they “heard about this frequently,” but that their surveys would not allow patients with readily apparent ulterior motives (such as those patients seeking narcotics prescriptions) to be excluded from data since it could lead to “cherry picking” patients and could impact the quality of the Press Ganey database.
While these sources of error are not unique to patient satisfaction surveys, it is important to recognize the impact that they may have upon the results of patient satisfaction data.

Catering to patient satisfaction scores increases health care costs
Another question in the Emergency Physicians Monthly survey asked respondents to rate on a 1-10 scale how patient satisfaction scoring affects the amount of testing that they perform. Forty one percent of medical professionals decreased the amount of testing performed while 59% increased the amount of testing they performed due to the effect of patient satisfaction surveys. From a numerical standpoint, with “1” representing a “maximum decrease” in testing performed and “10” representing a “maximum increase” in the amount of testing performed due to effects of survey data, the change in amount of testing performed due to satisfaction data averaged a score of 6.3 – a mild increase.
The increase in testing that survey results tends to cause may also set up a conflict of interest with hospitals that strive to improve patient satisfaction data but that also stand to benefit financially from the increased testing that results from attempting to improve satisfaction scores.

The threat of low survey scores frequently results in inappropriate medical care -- and sometimes causes poor patient outcomes
In the Emergency Physician’s Monthly survey, 48% of health care providers reported altering medical treatment due to the potential for a negative report on a patient satisfaction survey, with 10% of those who altered treatment making changes were medically unnecessary 100% of the time. Examples of medically unnecessary treatment provided to improve satisfaction scores included performing unnecessary testing, prescribing medications that were not indicated, admitting patients to hospitals when they did not need hospital admission and writing work excuses that were not warranted. More importantly, 14% of survey respondents stated that they were aware of adverse patient outcomes that resulted from treatment rendered solely due to a concern with patient satisfaction surveys. These adverse outcomes included allergic reactions to unnecessary medications, resistant infections and clostridium difficile colitis from unnecessary antibiotic prescriptions, kidney damage from contrast dye, and medication overdoses.

Hospital liability could increase from the effects of patient satisfaction scores
Pressuring medical providers to improve satisfaction scores to the point that they provide medically unnecessary testing or that they admit patients to hospitals inappropriately may become a source of liability for hospitals. If adverse patient outcomes due to unnecessary medical treatment can be tied to pressures that hospitals place on the medical staff to improve patient satisfaction scores, civil liability to the hospital could result. Knowledgeable lawyers could allege that hospitals or physicians cut corners with critically ill patients in order focus attention on patients who will be receiving satisfaction surveys. In addition, as Medicare payments are scrutinized more closely, billing Medicare for treatments or hospitalizations that are provided solely from pressure to improve patient satisfaction scores will likely receive increased attention from Medicare RAC auditors. A pattern of such overutilization, if able to be substantiated, may be sufficient to warrant sanctions against a hospital. Health care providers who are able to prove how pressures to improve patient satisfaction scores unjustifiably increased costs to Medicare or Medicaid may choose to file “whistleblower” lawsuits in hopes of earning up to 30% of the recovered overpayments hospitals receive. Any perceived retaliation against providers who file these qui tam lawsuits subjects hospitals to even further liability under whistleblower statutes.

Conclusion
More than six in seven of the health care professionals responding to the Emergency Physicians Monthly survey believed that patients used the threat of negative satisfaction scores to obtain inappropriate care. While it is unlikely that 86% of patients are obtaining inappropriate medical care, the health care providers’ negative perceptions of how patients are using satisfaction surveys show the significant detriment that satisfaction surveys have had on the physician/patient relationship. Overemphasis on satisfaction data, especially when that data may be unreliable, is likely to increase the likelihood of inappropriate medical care, increase the costs of health care, demoralize health care professionals, and increase liability for hospitals in the future.
 

Comments   

# Excellent articleSetu Mazumdar, MD 2010-09-22 12:00
This is a great article by Dr. Sullivan and Dr. DeLucia. You have said what needs to be said but more importantly you've backed up your assertions with logic and rationale.

It's too bad more emergency medicine physicians are not speaking up about this issue. I and my colleagues have been personally threatened with "low patient satisfaction scores" as a means to change behavior, and I bet you many other EP's have been threatened as well.

Unfortunately part of the problem also lies with an inherent conflict of interest with some emergency department directors. Think about this: if a hospital administrator singles out a few physicians each month as having low patient satisfaction scores, what is the emergency department director (physician) to do? We know that most of these complaints are frivolous but if the emergency department director has his monthly stipend dependent upon pleasing the administration, whose side will he take even if the complaint is bogus?

Let's see: side with the physician and risk losing $10,000-$15,000 per month as a stipend or side with hospital admin.

Now I'm not suggesting that all directors do this, but there is absolutely the potential for a conflict of interest here and it needs to be discussed, disclosed, and openly explained by ER directors to the physicians who work alongside them.

The second point I'd like to make is that what's disturbing about Press Ganey is the focus on negativity towards physicians. What about all of the wonderful things we do every day, every shift (day or night), every patient regardless of ability to pay? The ultimate risk we take is seeing a patient, not getting paid, and taking on liability. Which other profession does that and does it so incredibly well?

And this gets me to the last point, which is....shame on us for letting this happen. What exactly are we afraid of? I see so many physicians whisper about this issue during a shift but no one is brave enough to say, "Enough!"

Thanks for a great article and hope this stimulates a discussion.

Setu Mazumdar, MD
President, Lotus Wealth Solutions
www.lotuswealthsolutions.com
Reply
# One more thing...Jason 2010-09-27 10:29
Even if you do not see a patient, you can still get a negative survey. For example, there was a nurse practitioner working one evening when I was. I have to cosign his charts. I NEVER saw a certain patient. The patient HATED the NP. The patient duly filled out a scathing Press-Ganey survey. It went to me (because I'm the doctor - the mid-levels at that facility don't get P-G rated). It impacted my scores. P-G said that there was nothing they could do. Get this - they said that they didn't want to skew their statistics. OK, a survey that doesn't belong to me is skewing your data, but you can't remove it because it would skew your data (which is already excessively sketchy)?

And, at my prior job, two consecutive quarters in the bottom 3 of the list got one considered for termination (although not automatically getting canned).
Reply
# Patients are NOT to be relied upon!Fuzzy_Lugnutz 2010-09-27 21:26
I am sadly a "frequent flyer" within the local ERs due to a really rotten auto-immune disease progressing & running amok & have been ill since 1997. In my time as a chronically ill pt., I have come to recognize that the public that the hospitals serve are NOT to be relied upon for truth, be it about themselves or others. I never realized that the surveys sent to my home address, with what I consider redundant & stupid questions was taken seriously by the morons running the hospitals. Unless the CEO of a hospital is a doctor, nun, or priest: they are so uninterested in both the staff & pts. & have no understanding that many people treat the ER as a personal "store" for free pregnancy tests, narcotic medications (most often not needed for chief complaint, if there really is one), a warm bed, hot meal, baby sitting of children or elderly relatives & other nonsense. While I have had some doozies when it comes to ER docs & nurses not paying attention or believing me (pre-dx) when I would give myself to their care, 99% of the time, I was taken care of & taken out of harm's way, which should be the 1 & only thing the stuffed shirts running the hospitals should concern themselves with. It's a truly sad state of affairs when the pitifully done & very leading (to draw negative commentary) questionnaires are given more power to a doctor/nurses performance review than the number of patients saved or healed. I guess I will do my best to always fill those stupid things out with nothing but wonderful comments for all involved, even when I feel slighted, just to make sure that those who have had a hand in keeping my soul connected to my body & make sure the quality of my life is good will not have to stand in a room with a bunch of stuffed shirt dildos doesn't have the power to ruin their lives based solely on the lies of the patients they'd saved but denied a full 7 course meal before surgery, or denied a 25 mg. shot of demerol "for the road." Have faith o might physicians, some of we mere mortals actually care more about you than venting on a piece of paper.
Reply
# John Smith 2010-09-30 13:24
Press Ganey runs their monthly and quarterly reports based on when surveys are received - not based on dates of visit. So unless you only look at annual reports (like CMS HCAHPS), the ups and downs seen in monthly and quarterly reports is inaccurate.
Reply
# MDPaul Delaney 2010-10-01 21:18
After reviewing our hospital-wide Press-Ganey scores, we were "coached" on how to improve them, because we had too many "very good" scores, not enough "excellent" scores.

So what the stats are really measuring is how well you and your staff persuade the patient to give a high score (or get rid of low scorers). These are compared to other were the same thing is going on.

Reminds of a recent radio program about NYC police downgrading crime reports (e.g. rape turns into trespassing) to improve their crime statistics.
Reply
# John Smith 2010-10-05 11:57
The wait time is self-reported on Press Ganey surveys, not extracted from an EMR. So these "national studies" where states are compared by ER waiting time is highly questionable.
Reply
# Jordan Barnett MD 2010-11-05 01:20
I just wrote a blog regarding surveying in the ED. I whole-heartedly agree with this article. Blindly following survey data without paying attention from whom it is from as well as the dichotomy between EM training and what is being demanded is leading us down a slippery slope!
Reply
# EmergencyDabeet 2010-12-02 10:59
“Overemphasis on satisfaction data, especially when that data may be unreliable, is likely to increase the likelihood of inappropriate medical care, increase the costs of health care, demoralize health care professionals, and increase liability for hospitals in the future."

It is sad to see that there is so much negative sentiment on relating to patients, empathy and compassion in healthcare.
Patients want what we want when our friends, family or even ourselves present to an ED. Do you wait? ---go to triage, registration, wait for a charge nurse to place you in a room than wait for a provider to come and evaluate? NO, you call ahead, reserve a room and may have to say, “be nice to her she is my Aunt”
Do you want competent, informative, and pleasant healthcare providers?

If you don’t care about patient satisfaction than you are saying that you DON’T care about:
Alignment of resources
Streamlined and proven Processes
Accountability of people.
They all go hand and hand.

If you change the way patients perceive you in the clinical area- fix your body language and your attitudes toward non emergent cases and remove the "what are you doing here, this is not an emergency" from your mind, Your patients will trust you when you tell them "you don't need a cat scan." If you practice true patient satisfaction you actually order fewer tests and it makes your job easier.


How come physicians that have poor patient perceptions tracked by a patient satisfaction survey (that care about fixing it) see scores improve?
Explain to me how 15 out of a 20 Doctors on a staff working the same shifts have great scores and the other 5 don't? Is it the data or the individuals? Of course you need the returns to make conclusions. Most clinicians will have 300 to 400 surveys yearly.
The statistical reliability of any survey could be questioned; in fact the author’s own statistical criticisms of Press Ganey could be used against the data collected for this article.

Patients’ perceptions that a provider cared about them has been studied and proven to decrease malpractice claims.

I know you’re thinking that clinical quality is all that should matter in healthcare. Patients come to an Emergency Department with an understanding that you will make the right diagnosis. You go to bed every night thinking about your clinical decisions and work very hard on your clinical acumen to be a great health care professional. Your patients expect that from you. Is it good enough to be average in healthcare?
Is this any different in other industries that pertain to life and death? When you get on an airline for your next medical conference and it is in Las Vegas, Do you expect to land in Idaho? You expect to arrive in Las Vegas, God forbid not on time.

Would that airline ever get another consumer again if it couldn't deliver that?
On that same flight, what made you satisfied?
How you were treated?
Were you treated with respect?
Were your questions answered?
Were you made comfortable?
If you answered no to these questions, would you recommend that airline to someone else?

Conclusion:
Superior patient satisfaction increases market share and profits while decreasing medical legal liability. It also makes your job easier.
Reply
# PAScott Cranick 2010-12-26 22:25
In response to the post by Dabett's post... are those airlines and their 'great' service forced to provide those services to everyone who shows up at their gate whether they can pay for it or not? I think not. They have the luxury of only having to service those who can pay for it. That's the key. All of the services that you expect to be treated with respect and pampered you pay for. ER docs don't get paid any extra to save lives with a smile, but they will get sued for no saving that life, smile or not.
Reply
# radiation therapistjohn 2011-01-07 11:09
Press Ganey is the biggest waste of time i have seen in my whole career in the health care field. Sometimes patients of mine that love me, and i get attached to them as well, turn in bad PG reports. Patients that even brought gifts to the department. Sometimes we get errors and comments that were directed to departments not even affiliated with our hospital. But press ganey says it can't be changed. Ha, can't be changed. The bottom line is- the warped thinking that started all of this in the first place: YOU CANT relate feelings and individual thought to NUMBERS. Two things that have nothing to do with each other.
Reply

Add comment

Security code
Refresh

Popular Authors

  • Greg Henry
  • Rick Bukata
  • Mark Plaster
  • Kevin Klauer
  • Jesse Pines
  • David Newman
  • Rich Levitan
  • Ghazala Sharieff
  • Nicholas Genes
  • Jeannette Wolfe
  • William Sullivan
  • Michael Silverman

Earn CME Credit