[Maynard]

Don't know about the rest of you, but I'm stuck in this kind of drivel more and more at work - supposedly moving to a 'data driven' world, but in reality it is this type of skewed unrealistic binary thinking, coupled with threats that 'not making the numbers' (usually all max values) means defective performance. I keep waiting for somebody with half a brain to finally graduate from business school and wake up to this, and how it ends up defeating most of the value of the measure. I keep thinking back to a recent Lutz editorial, where he says flat out that the VW debacle was the CEO's fault, adding something like 'if you make people fear to tell you s/t you don't want to hear, they start to lie, and you don't know what is really going on'. to me, this is basic common sense, not some high level guru-speak.

[BEM-S4]

Originally Posted by Maynard

Don't know about the rest of you, but I'm stuck in this kind of drivel more and more at work - supposedly moving to a 'data driven' world, but in reality it is this type of skewed unrealistic binary thinking, coupled with threats that 'not making the numbers' (usually all max values) means defective performance. I keep waiting for somebody with half a brain to finally graduate from business school and wake up to this, and how it ends up defeating most of the value of the measure. I keep thinking back to a recent Lutz editorial, where he says flat out that the VW debacle was the CEO's fault, adding something like 'if you make people fear to tell you s/t you don't want to hear, they start to lie, and you don't know what is really going on'. to me, this is basic common sense, not some high level guru-speak.

Biggest problem with most of this type of thing is incentive structures are all screwed up and not properly aligned. Sales teams for example often push tons of legacy less profitable product because it's an easier sale, and they hit their numbers and max out their personal bonus. The BU misses out on profit from better margin next gen stuff that's a tougher sell for the sales team. That's just one example imagine a huge corporation like VW or anything else it happens a million times a day. Compounded by the fact that managers juice their numbers get promoted and leave a mess for the next guy to clean up. It's almost impossible to get perfectly right.

[Viffermike]

There isn't a method out there that doesn't have major flaws, ladies and gents. Thing is, many large, decentralized companies consider such surveys the easiest and most reliable way to measure customer satisfaction and outlet competence.

The fundamental balance is this: How does a company gather satisfaction data from hundreds of nodes (read: dealers) without making the collection so involved that nearly no consumers are willing to invest the time to fill the survey out? This is statistical survey/data analytics theory 101 kind of stuff.

Data is only as good as two major things:
1). Its volume
2). Its reflection of as large and even a cross section of the study group (customers) as possible

If you make the surveys too involved, you automatically skew the survey to the point of worthlessness because those with major complaints will be, by far, the largest portion of the subject field to fill them out. Yet if you make them too easy, you don't gather enough meaningful information to solve issues.

That's why many companies -- not just BMW -- actively promote the 'all 10s' request. It's a skew metric that basically equates to a data point that indicates satisfaction, but not complete satisfaction -- and it's a far more valuable metric than surveys that don't ask deep enough questions or surveys that have an extremely low response rate. Algorithms exist that incorporate this skew effect into data analysis not as raw itemized multipoint data, but as its own data point.

Personally ... I don't fill them out. I'm too honest and critical for my own good.

[msej449]

I worked for a European multinational and they had a very sophisticated evaluation system which I came to admire. The problem with most evaluations are that they suffer from an inevitable 'rightwards bias' over time i.e. start out at expecting normal performance around the 5-7 mark and slowly but surely the scoring moves towards 9-10. In effect, everything gets compressed to far right of the scale. There are ways to handle this, but they generally take a bit more time and effort, which most second-rate executives aren't prepared to invest.

[6734202]

I had my CA call me after I filled out a dealer survey. Evidently anything less than 10's and super excellent means the CA (or likely SA) will not receive a monetary bonus for the month. Seems ridiculous to me...I scored nothing less than 8. The CA told me he has to tell people that it's 10 or nothing so he doesn't get screwed.

[B_Rice]

Originally Posted by 6734202

I had my CA call me after I filled out a dealer survey. Evidently anything less than 10's and super excellent means the CA (or likely SA) will not receive a monetary bonus for the month. Seems ridiculous to me...I scored nothing less than 8. The CA told me he has to tell people that it's 10 or nothing so he doesn't get screwed.

In corporate's eyes, it really is 10 or nothing.

[3.0L]

Originally Posted by Viffermike

That's why many companies -- not just BMW -- actively promote the 'all 10s' request. It's a skew metric that basically equates to a data point that indicates satisfaction, but not complete satisfaction -- and it's a far more valuable metric than surveys that don't ask deep enough questions or surveys that have an extremely low response rate. Algorithms exist that incorporate this skew effect into data analysis not as raw itemized multipoint data, but as its own data point.

Personally ... I don't fill them out. I'm too honest and critical for my own good.

Originally Posted by B_Rice

In corporate's eyes, it really is 10 or nothing.

And this, folks, is precisely why I have finally decided to ignore dealer surveys.

I feel better already!

[NVHoo]

We have 2 dealerships in Vegas and I quickly learned one has terrible service. Shortly after I bought my used 128 I took the car in for several electrical issues, one being the radio stopped working. It took them over a week to fix it and every time I called I got a different story on the progress (they finally had to replace the ECU and had to get the replacement from BMW). When I went to pick it up I asked what was wrong with the radio and he said "What radio problem?". The radio item was on the second page of the original service write up and no one ever looked at the second page, they had to replace the radio, and I was driving a loaner for a second week. Needless to say they did not get high marks on my survey. The Service Manager called me about my survey and said my review cost the service advisor a big bonus and would I please revise it? His argument was that it was an honest mistake, they did their best (as bad as it was), so they should get good marks. I refused to change it and because of the Service Manager's pressure and fear of retribution form the service staff, I never returned to that dealership. I HATE the feedback system because it is not anonymous and leads to issues like I had that makes me feel very uncomfortable. So I don't fill them out anymore.

[Elvisfan]

The bottom line is that is the system. The dealer and we have to deal with it. Unless someone really pisses me off,they will get 10s. It would have to be really bad for me to do something that would effect their pay. I don't know what I would have done with the missing the radio thing.
I just filled one out for my Acura. One of the questions was about value based on price. Yeh,right. $75 for 2 air filters. No way that was getting a 10. It shouldn't even be on the form. I did N/A. I wonder if that will have an effect?