I knew a doctor who worked for a large healthcare organization in the Chicagoland area.  As part of the group practice, he was subject to evaluation, which was partly based on patient satisfaction surveys.  One of the questions patients were asked about their physicians was whether he or she saw them in a timely fashion.  These evaluation instruments were important, as they helped determine bonuses and supposedly provided feedback regarding performance.  The idea was to improve patient care.  However, this physician’s schedulers once booked him for four patients at one time.

In a similar vein, healthcare organizations are evaluated by the Joint Commission on Accreditation of Healthcare Organizations.  The JCAHO mandated some sort of patient satisfaction surveys, so a hospital that I worked for in the 1990s engaged a well known firm to perform the task.  The product was a shiny tome that drew all sorts of inferences, but the response rate was 19%.  This is laughably inadequate–to avoid skewed information, the response rate should be at least 80%, preferably 90%.  Moreover, while the hospital had a significant Hispanic population (about 25%), the surveys were in English only.

Are These Surveys Useful?

Of course no one can object to the idea of evaluating quality.  But, at least in healthcare, as shown above, they are of questionable value–and they may even be counterproductive.  For instance, a famous study found that patients for whom more tests were ordered gave higher satisfaction scores.  This fact of course incentivized physicians to order unnecessary tests.  And, crucially, the patients who got more tests had a higher mortality rate.

The evaluation instruments used today are also outdated, and are very sloppy in terms of statistical conclusions.  A physician who provides exemplary care may get an unfairly low overall score, because one patient who doesn’t like the weather gives poor ratings.  Such flawed information can have a devastating effect on physicians, and some studies on physician “burnout” implicate the satisfaction surveys as a cause of depression in doctors and even early retirement.

W. Edwards Deming is the father of the quality initiatives in industry.  Even he, however, recognized the pitfalls.  He wrote:

…the merit rating system nourishes short-term performance, annihilates long-term planning, builds fear, demolishes team work, and nourishes rivalry and politics.  It leaves people bitter, crushed, bruised, battered, desolate, despondent, dejected, feeling inferior, some even depressed, unfit for work for weeks after receipt of rating, unable to comprehend why they are inferior.  It is unfair, as it ascribes to the people in a group differences that may be caused totally by the system they work in.

It’s Everywhere

The obsession with evaluation is not confined to healthcare.  After buying toothpaste at a large chain drugstore, I find an email regarding how my experience was.  At the end of a phone call for which I was on hold for five minutes, and got entangled in an endless loop, I am asked how I enjoyed the experience (duh).  Leaving a restroom, I am presented with three buttons to rate the cleanliness of the facility.  It goes on and on.  And I am skeptical that any action is taken on these surveys.

Why?

I suspect that all of the evaluation movement is a way to bestow the illusion that organizations are really pursuing quality.  The contrary may be true–and haphazard evaluation may even be harmful.  At the root of the problem is the reliance on things that can be easily measured.  You can easily count the number of colonoscopies a physician orders, but rating empathy is another level.  Until someone figures out an effective way to objectively measure some of the most important subjective qualities of a physician, the satisfaction ratings as they are currently constituted should be trashed.

NOTE:  Do not rate this article.