the misery of metrics and measurements destroying job satisfaction

Yesterday I chose to get wet in Ipswich town centre to demonstrate about Mr Gove.  Okay, that’s bull, but I was roadie for the day as I ran the PA 1 for the NUT strike demo about pay and conditions, and Mr Gove.

The NUT rally. can't work out why this iPod photo is such bad quality; is it me or do mobiles always take crap pictures?

The NUT rally. Can’t work out why this iPod photo is such bad quality; is it me or do mobiles always take crap pictures?

I’m not a teacher and don’t have kids so he isn’t specifically my problem. However, some of the themes sounded familiar. In particular the rise of collecting ‘data’ for performance measurement systems and the trends of micromanaging the shit out of white-collar jobs was exactly the sort of thing that pissed me off about work. I wrote about digital Taylorism in 2010, and the NUT’s Jon Parker indicates the issues that sound similar – listen to the crowd response to ‘data’ being collected pointlessly 2

Ipswich NUT John Parker on data and metrics (MP3 1min)

There’s a case to be made that The Firm was trying to squeeze their old gits out of the place, which is why they employed pointless pricks to produce software systems to piss people off. This doesn’t seem to apply to teaching, however, where it seems the working environment is such that 2 out of 5 teachers quit within the first five years, there’s presumably no imperative to thin the ranks at a time when Britain is experiencing a baby boom and somebody presumably has to teach them.

Now some of the changes to the workplace are the result of secular trends like globalisation and technology, which at least does somebody some good even if the end of the boat Western workers are in is sinking. But the stupid pursuit of pointless performance metrics making jobs a misery seems to be 100% own goal. Not only do we have to employ useless patsies  to collect the pointless performance data to piss people off, but the measurers are usually paid more than the people who do the work being measured, because of the Peter Principle.

That’s the trouble with the homogenized management theories that come out of MBAs. Theories and fads go through companies like a dose of salts. and because we have people benchmarking along the lines of bollocks like ‘business best practice’ they all follow the same bullshit until the next fad comes along that is going to be the Holy Grail and sort out the crap that the last fad made. Let’s have a sample of bullshit MBA fads from my working life.

  • Empowering employees
  • TQM (total quality management)
  • winning edge – mindset management
  • investing in people
  • managing my performance
  • shareholder value (that’s 1 year share price hiking so the CEO can Maximise his Apparent Performance by buying today at tomorrow’s cost)
  • Agile development (in a big firm?)
  • six-sigma
  • just-in-time
  • business process re-engineering
  • mission statement
  • outsourcing
  • Putting Customers First
  • core competency

All of these can work well some of the time in specific instances. None of them work when applied across the board like velveeta. One of the worst things they must teach people on MBA courses is that there is a silver bullet. You see these wet-behind-the-ears young pups promoted into a situation beyond their competence as they wax lyrical about the next best thing that’s going to transform everything and have to keep a level gaze… Because you know that it’s never different this time, and it wasn’t different the last ten times either. One size does not fit all. And these berks have insufficient experience of the real world to have had that belief in the silver bullet beaten out of them in the school of Real Life™. Reorganisations are political, they are the new Top Banana and Chief New Broom acting like a tomcat 3, spraying his mark on the organisation. They are not functional.

The teachers are just taking the same hit from performance management theory which is a Current Big Thing – tell people how shit they are doing, preferably every quarter, because you can manage expectations about pay that way. That’s obviously the way to motivate people to do better. That toe-rag Tom Peters has a hell of a lot to answer for. You get what you measure. Right. You can measure a pint of beer easily enough. How do you measure a teacher? A CEO? An income tax inspector? Ah, teachers, that’ll be exam results then? What about if they have to teach a whole bunch of stupid kids then, or the kids of parents that don’t really give a shit and probably shouldn’t have been encouraged by Tony Blair to have ‘em in the first place? Ah, let’s measure how clever they are when they enter school. Right, so how do you measure how clever they are? Is cleverness the only dimension of success – maybe a reduction in sociopathic behaviour and not kicking the shit out of the municipal bus shelter is a good outcome too? How do you measure the civic street furniture not trashed by the little tyke because he’s inspired to do something else? Measurement always has a problem with the counterfactual and the road not travelled. And so on. I’m with Lord Kelvin when it comes to measuring things that have a numeric answer that matches with the aspect of reality you’re trying to get, but when it comes to people the belief that Tom Peters prosyletised that ‘measurement works’ seems to be responsible for a lot of hurt in the workplace, and some not  particularly great outcomes. If you link people’s pay to metrics you get those metrics, but you don’t usually get great performance 4.

However, thankfully this is no longer my problem :)

Notes:

  1. Our farm isn’t on the electricity network but every so often we want to all get together and have a party so I have a music system of a couple of hundred watts  run off a 12V leisure battery. Using this saves having to wheedle a mains power feed from some local business or run a genny in a public place with all the safety issues.
  2. the dreadful distortion is because the recorder was overloaded, the Ermine delivered a better quality PA service to the crowd
  3. I have some trouble picturing Michael Gove as a tomcat, he’s a bit on a weedy side for a big old ginger tom
  4. for example, CEO pay since the 1990s, NHS waiting lists and beds, Enron, the Global Financial Crisis, the list goes on
27 Mar 2014, 12:15pm
by Marky mark


Ah, welcome to my world. I recognise many of those come-day-go-day saviours you listed. Large Government computer systems seem particular prey to the latest fad. We are about to embrace Agile (as this season’s range from ‘Emporer’s New Clothes’). Yet whenever we ask the question ‘How are you going to reverse engineer that methodology onto a legacy system made up of huge monolithic programs?’ we get told that ‘we’ll worry about that later’.

Later means, of course, ‘after we’ve all been promoted to other projects on the back of introducing this new fad and you’re left trying to make it actually work’.

27 Mar 2014, 12:28pm
by Grumpy Old Paul


Your experience with management fads of the 1980s and 1990s uncannily mirrors my own. I even coined an acronym describe the syndrome:

YAFP – yet another f*ing panacea. (Inspired by yacc)

In my last but one job, the Firm invested heavily in TQM importing wholesale Phil Crosby’s flavour and sending all of the staff on extensive training courses. The holy text was ‘Quality is Free’ IIRC. I used to threaten to put a sign on my screen reading ‘Quality is Crap’.

In my last job, at one stage, my department was committed to spending 10% of their time realising ‘The Vision’. Can you believe it!

Finally, another acronym for you, MMM (Mindless Metrics manufactured by Morons).

Mindless because they aren’t thought through and don’t relate to core functionality. No thought is given to the impact of meeting the metrics on core functionality. The most common attributes of metrics I encountered were ease of measurement and irrelevance.

Does anyone besides me think it was odd that the NHS whose core functionality is to improve health and longevity of life didn’t spot Shipman earlier through mortality statistics? No mortality metrics collected on GP practices?

However too many people within the NHS have a very weak grasp of statistics. Remember Roy Meadow – see http://www.richardwebster.net/cotdeaths.html for an interesting account.

@Marky Mark – how did you guys hold Agile at bay for so long? Agile and legacy does sound like a recipe for ‘interesting times’

@GOP that TQM fad in the 90’s was grim, eh? I added a couple more fads that you reminded me of.

> The most common attributes of metrics I encountered were ease of measurement and irrelevance.

I think you’ve put your finger on the big problem with metrics all round. An awful lot of what’s worth having in the world of work isn’t measurable as a number, though you know if you have it on not. If you get what you measure then you need to avoid measuring the irrelevant and incidental like waiting times – the job of the NHS is to fix people where possible.

28 Mar 2014, 8:17am
by Neverland


Don’t agree with you on this one

While Gove is quite uncious grown-up version of Harry Enfield’s Tory Boy, measurement and diversity in teaching (or the NHS for that matter) are good ideas

Without measurement how can you work ut how to improve things (cf. the rather worthless data driven UK olympic sports program has won a lot of medals)

Without diversity how can you work out if different models for organising schools represent a better model than the majority one we have now

Its normal to not to particularly like your job, thats why somebody pays you to come in there and do it

Getting the right measurement systems in place can work and create improvements, e.g. the improvements in the inner London school system over the last 20 years

@Neverland a good challenge, but I still disagree :) Measurement works where the desired variables are one dimensional or can be captured if multivariate. Your sport example is a classic case of something designed to be measured as one variable. That’s what sport is – time to finish the race, goals scored etc. We take great care to control the inputs too – we test the balls against specifications, level the playing fields, switch batting ends and test for drugs. It’s not surprising that measurement works to improve outcomes in sport because it’s easy to specify what success looks like and inputs have been controlled in the interests of a fair contest.

Taking your second example, it shows a common fallacy in business metrics.

> Getting the right measurement systems in place can work and create improvements, e.g. the improvements in the inner London school system over the last 20 years

Correlation is not causation. A lot of other things have changed about inner London over the last 20 years. For instance Londoners have gotten a lot richer, the improvements could equally be that parental support/involvement is greater than it used to be, since this tends to correlate with increasing wealth. London, and Inner London in particular grew richer faster than the rest of the UK. Maybe inner London local authorities had more money per child to spend on schools than other districts.

That is the trouble with a lot of metrics – you can’t roll back and test the counterfactual- what if the change you make that you believe will improve the measurement isn’t made? Other factors may be giving you the win for free, in which case you’d actually be better off not taking the course of action that you’re now convinced made the improvement.

Most of the problem with metrics is business management seems to be simplistic and short-term analysis – I’ve never done an MBA but there is a silver bullet must be part of MBA 101.

> Its normal to not to particularly like your job, thats why somebody pays you to come in there and do it

Well, yes, though why does it have to be like this? The economy is there to serve humans, it isn’t a force of nature. It’s shaped by people, and if it makes an increasing number of people miserable then there’s a case for changing how it works. It could be justified if the change improved something else for others (globalisation for instance – at a guess it is making other humans richer though First World average workers poorer, but by less than the beneficiaries) but if it’s self-seeking make-work then no.

If you’re going to measure then do it right or go with skill and experience and open debate accepting the uncertainties of the real world. Developing a feedback system that is precisely measuring the wrong variable is an excellent way to end up getting not what you want. There is no law of Nature saying what is easy to measure is desirable. Nor that what’s desirable is easy to measure.

F’rinstance incentivising CEO pay on share prices looked like a great way to go to maximise shareholder value in the 1990s. It ended up in the increasing capture of the value added by business going to the senior employees rather than the shareholders as this Forbes article postulates, along with indicating that measuring the value of a CEO is an intractable problem.

@Ermine – oh that list brings back memories! Just as I was leaving MegaCorp they were establishing an Agile practice – this in an incredibly complex multi-national who someone one quipped, “moves at glacial speed”. The smallest unit of measurement was Quarter

I also spent a lot of time there developing metrics and KPIs :)

oh and thanks for the link on your site btw

Ermine, love this one. Particularly the MBA fads list (which I shouldn’t be saying as a prof at a business school). Neo-liberalism’s obsession with measuring performance is detrimental to many occupations; mainly because focusing on performance is in the way of achievement.

I am with the teachers on this one; and with the lecturers. Measuring performance make us teach more and care about learning so much less.

29 Mar 2014, 6:36pm
by Neverland


@Ermine

You seem to ignoring the process of learning from experience when designing metrics to measure things and simply stating in a luddite fashion that it can’t be done

Didn’t someone ever teach you the benefit of experimentation to test theorem?

If we could put a man on the moon 45 years ago I’m sure a decent set of performance measurement metrics for secondary schooling is not beyond the human race with some trial and error

Ooooh, yes – I remember Six Sigma. Every shade of bonkers!

@Neverland When I look at the school system between now and when I was a child I am not so sure I see evidence

“of learning from experience when designing metrics”

and I was taught that a hypothesis needs to give consistent and repeatable results to be confirmed ;)

@Maria I like the elegance of the observation – “mainly because focusing on performance is in the way of achievement.”

30 Mar 2014, 9:34pm
by Caroline


Thanks for this Ermine. I wholeheartedly agree.
In my experience, from the university sector, focussing on measuring performance is not only in the way of achievement; it also stiffles creativity and innovation. Lecture courses now need to have “learning outcomes” stated up front, which doesn’t allow for free thought and learning (or achieving) something other than that intended.
I imagine the same applies in schools and firms.

Aside from the pointless, arbitrary nature of the metrics sapping a teachers will to teach, you create a culture where sociopaths can flourish and prosper.

You’ll recall from the last couple of investigations into healthcare fubars a culture of fear, bullying, secrecy and cronyism can be created. Surprisingly you get the same outcome in schools. Perfect.

@Jane – Ha just had a nasty flashback to a six sigma blackbelt infestation, much bollocks with such straight faces.

7 Apr 2014, 7:14pm
by Ray MacDonald


Digital Taylorism is alive and well in Canada as well.
I retired early in 2005 after a 35 year career as a food scientist in industry. For that entire time I was engaged in new product development for the Canadian market. Today the food scientists who replaced me do no-mind “deployment” of generic products developed in the Company’s central labs in the US, Netherlands or UK. It won’t be long until the lab I worked in will be closed.

8 Apr 2014, 9:54pm
by ermine


@Ray sounds like what happened to the place I worked. I was shocked to find, in a meeting with Greg Barker of the DECC that of the bunch fo local engineering firms that might be of note as incubators, My old workplace that used to be a premier research facility wasn’t mentioned at all. The caretakers amd pusillanimous MBAs that guided the decline over twenty years should be ashamed.

Hopefully somewhere in the list there are the startups what will become the leading lights…

@Caroline, Nathan – I guess this is what happens when we lose sight of values for the measurable. I think that Oscar Wilde fellow summed it up nicely with knowing the price of everything and the value of nothing!

[…] wrong with doing that to people in a professional organisation. We seem to search more and more for stupid metrics and valueless numbers rather than seeking knowledge. The world is complex, it’s messy, and […]

[…] using sophisticated micro-reward systems. This sort of thing started to really piss me off at work, stupid metrics on irrelevant areas being used to herd and control people, and it appears to be going on in the consumer space too. […]

 

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>