22 Jan 2014, 5:28pm
rant:
by

11 comments

  • January 2014
    M T W T F S S
    « Dec   Feb »
     12345
    6789101112
    13141516171819
    20212223242526
    2728293031  
  • Archives

  • empty metrics are strangling vision and leadership

    I’ve spent the last year and a quarter trying to avoid anything that smells like work, but stood in for someone who fell ill at the last minute at a conference on how to leverage the private sector and the third sector in Suffolk. There’s no money in local government, hence the absence of the public sector in the roll-call…

    Some things are clearer when you stand back and look at them from a distance. I listened, and much was about ‘collecting evidence for metrics’, and somewhere in the back of my mind a refrain was building. It reminded me of all the stuff on performance management instead of inspiration, and I realised that what has gone wrong in much of business is that process is being elevated above common-sense in an attempt to make things more transactional, so that interfaces between companies and business units can be defined with greater precision.

    Precision without accuracy means you may always miss – every time

    It is the mark of an educated man to look for precision in each class of things just so far as the nature of the subject admits

    Aristotle

    The trouble is precision and accuracy are not the same thing. It’s too easy to end up precisely getting not what you want…

    you can be precise but not accurate

    you can be precise but not accurate. In most things accuracy is what you want, rather than precision. You don’t want to be always shooting the wrong guy 😉

    Metrics emphasise precision rather than accuracy, particularly when implemented by people without a scientific background. The Guardian has a wordy version for people who prefer narrative to visuals, but basically in the battle between precision and accuracy precision is easier to assess but accuracy is what you want. You’re better off generally shooting the right guy even if you take out the wrong-un every so often rather than always shooting the fella to the right of the bad guy.

    The tragedy is that it’s really, really, easy to tell if you are precise – you always get the same answer, or at least the spread is small. Whereas it’s a bugger to know if you are accurate, particularly if you are trying to find out something unknown. If your spread is wide you may well be centred on the right target but your opposition will holler the place down yelling out at your inconsistency. So in practice we focus on measuring things that tend to be consistent, even if the answer is wrong because we had to ignore too many pesky variables that throw the accuracy off.

    Process makes us focus on metrics, and thus we easily end up with a culture with

    The endless search for evidence nobody needs, to tick boxes that shouldn’t be there, to impress people who don’t really care.

    There’s an answer to this. We used to know what it was. And we desperately need it now. It’s called leadership.

    We humans live in an uncertain world, and yet we still have to take decisions. We will get things wrong sometimes. Some things are amenable to analysis and science, and these should be assessed that way. Engineering projects often have a lot of this – nowadays we can determine if the bridge will stay up given the likely loading, unlike the Victorians, who often had to over-engineer some of their great works after they discovered the hard way than erring to over-engineering was preferable to under-engineering.

    1311_Tay_bridge_down

    Not everything to do with science is amenable to exact analysis, particularly when it comes to the unknown unknowns, which is part of science’s core business. Sometimes the balance of probabilities is the best you can do.

    James Delingpole is absolutely right. It’s perfectly possible that global warming doesn’t exist and/or if it does then it’s not anthropogenic. It’s just not incredibly likely given the data that has been collected.

    Many things in public policy are a balance of opposing forces. There’s never enough money, never enough time, there’s always more than can be done, and sometime alleviating one thing can lead to unintended consequences. There is no right answer, and cod metrics and getting your tickboxes in a row won’t help. All they will do is deflect the blame away from you and into the ‘Lessons will be learned’ catch-all refuge of last resort.

    Nowhere was this clearer than in the health and environment session. Britain seems to suffer from an appalling level of mental health problems, and one of the proposals was to get children to experience the natural environment before they were 12 – apparently the period before 12 is when a lot of attitudes and predilections are formed 1. There is significant anecdotal evidence that exposure to the natural environment and exercise are favourable to good mental health.

    Now I had the assumption that this is why every child has parents, in the case of a species such as humans that has a long period of dependency. However, it doesn’t seem to happen often enough in today’s Britain, and so the job falls to the schools in loco parentis

    After several proposals on how to measure the success of the result if something were done along these lines, I couldn’t hack it anymore, and told the assembled folk how this had happened for me. I grew up in New Cross, in London, in this area, and the primary school was also there.


    View Larger Map

    The teachers did not rely  on metrics. They relied on something older, something that has served humanity for thousands of years, mostly for good though sometimes for ill. It’s called

    Leadership

    The teachers knew they were teaching a bunch of city kids. So one spring, when it was sunny, they took us to Telegraph Hill, I believe, which is to the southwest of this map. I still remember lying on the grass and looking at the deep yellow of buttercups as they explained flowers, and how been pollinated them, and the cycle of nature 2. There were no metrics, no customer satisfaction forms to fill in, no key stage whatever garbage. The school headmaster knew his school, its intake, and led from the front. Resulting in a memory that stayed, even after four decades have rolled by.

    Curiously enough the Ermine seemed to carry people with the idea. Leadership did seem to matter, and indeed several other delegates recalled their schools doing this without funding, using local resources. Despite that the challenge was countered with another issue  as people said that schools have no funding to go into the natural world.

    Now if my inner city primary school could find green space within school crocodile walking distance then I have difficulty in believing that this is insurmountable, but I let this failure of leadership be – I was primarily as an observer, this isn’t my problem. I suspect the perfect is being the enemy of the good here. That’s again where leadership trumps box-ticking. Sometimes good enough is good enough, and it’ll do – you do the best you can with what you have to hand and then move on 😉 It would be nice to have the money to do it at Minsmere, but if the local rec or the town park is the the best that can be offered then it’ll do. I can show you sparrows, blue tits, great tits, robins, blackbirds, starlings, black-headed gulls in the rec near my home. a couple of hundred yards away in the town cemetery there are jays, crows, squirrels, green woodpeckers, chaffinches, greenfinches, great spotted woodpeckers, song thrushes and sometimes fieldfares. It really isn’t hard.

    Leadership is what we seem to be sorely lacking now, in many areas of life both corporate and political – the courage to have convictions. Its handmaiden, initiative, seems to be a little bit in short supply too, run out of town by the processes and procedures that seem to gum up the works. Where decisions can’t be avoided, we now often try and hide behind precision without accuracy, meaningless metrics collected without awareness of sample bias.

    We establish processes and systems so that faceless and distant bureaucracies can micromanage the operation from afar, maximising their metrics and results, run by vapid caretakers in systems designed by committee. When something goes titsup in such a system, it is never anybody’s fault, and inquires are held and ‘lessons will be learned’. Colour me unreconstructed, but sometimes a good honest apology along he line of ‘we really screwed up there, sorry to the people affected and I will do my damnedest to avoid this happening to us again’ would be a good deal more convincing than the mealy mouthed ‘lessons will be learned’ bullshit. Who will learn the lessons, and if they were hired to do the top job, then why the bloody hell are we paying them all this money to duck and dive their responsibility for this SNAFU rather than taking it on the chin?

    Every organisation should have the equivalent of this

    We need more of these signs

    We need more of these signs

    “If we’re going to be damned, let’s be damned for what we really are.”

    Picard, Encounter at Farpoint

    At work these spurious metrics seem to be what Farhad Manjoo calls the ‘Gamification’ of the Office (hat tip to Monevator for this one). I saw that first hand at The Firm and it did for me – it robbed working for a living of everything worthwhile. The Firm contracted a company called Successfactors to implement a software system where quarterly you’d have to gather evidence on what you were doing, how you were doing, get SMART 3 objectives.

    And with this endless measurement and metrication, the fire began to flame out. I found it alienating and hateful. For over twenty years I hadn’t had any general problem with the annual appraisement process  – I was assessed by people who knew what I was doing and its setting. I didn’t always agree with everything but  the overall process gave a fair balance between short term delivery and long-term improvement of one’s craft. Above all I could respect the people, their competence, skill and judgement.

    Successfactors and the American HR consultants turned that on its head. I am introverted, and to some extent a loner, I believe in achieving results through understanding and learning. I do my best work on  my own or with one or two others. I’m perfectly happy to present things once I have enough results to present, but what I don’t respond well to is endless interrupts. I could live with that, but the stupid performance management stuff, quarterly, with metrics of people giving 5 minute seminars and how often they used Instant Messaging that counted for 25% one one’s score drove me nuts. People ended up taking up massive amounts of time on all hand events giving 5 minute seminars on stuff that could be better learned from Google.

    There’s no point me detailing the particular problems of implementation. Farhad Manjoo did that far better than I can – I recognise every one of the ills he cites – The Firm was there ahead of time.

    Too much energy, resources and time are being spent on the worship of false metrics, in an attempt to avoid the difficult business of leadership – the art of taking decisions in the face of uncertainty, and knowing that sometimes you’ll be wrong. It’s often better to be wrong 20% of the time than to be right 20% of the time and paralysed by analysis 80% of the time. Micromanagement does that to people, and the worship of false numbers leads to micromanagement, and so the cycle turns and starts anew.

    Numbers have their place – it is knowable whether the bridge will stay up. It is knowable whether cheap Poundland batteries are actually cheap or they just look cheap. There are places where you have to run with the balance of probabilities. there are other places where it’s good to ‘fess up that you know that you don’t know – the classic Rumsfeldian treatise on epistemology and the known unknowns and unknown unknowns. A lot of economics seems to fall in the latter category.

    This is because systems with feedback mechanisms in them are the worst of all – they are often susceptible to chaotic behaviour that cannot be known other than in terms of probabilities, and then only if you can characterise and model them properly. Social sciences have a lot of this, where individual humans are adaptive creatures all trying to maximize their experience, with different time lags in the information flows. It’s why the stock market has such a shocking volatility, as all of this is worked out between independent actors, it is the analytical principle that underpins the efficient market hypothesis behind passive investment. The fundamental presumption behind passive investment is the myth of continual growth – it’s worked so far and has given a reasonable long-term trend. You are buying a slice of that and going along for the ride, with diversification averaging out company and sector volatility and decades long accumulation of holdings integrating the temporal volatility against the desperately low 4-5% or less real over inflation trend. 4

    However, systems involving humans aren’t perfect – it leads to old hands acknowledging that the market is not efficent, albeit not inefficient enough to get ahead using that awareness. Or perhaps yes, but the opportunity is temporal – we humans are prone to collective mania and the madness of crowds where everyone decides the world is ending 😉 This human capriciousness applies across the economy and across the social sciences.

    Something else I heard at the conference, for instance, was that there really do seem to be an awful lot of young people who have been deprived of, what in my day would have been called competent parenting, which is why it falls to the schools and/or society in general to try and compensate for the lack. It amazed me that everything seemed to be about identifying the problems without asking the fundamental question how the bloody hell did we get here?

    In my inner-city London primary school which had a single class of 32 of each age group, there was only one pupil that left unable to read and write. In relatively affluent Suffolk, forty years later, a fifth seem to struggle 5 Something has changed, and not for the better. One common theory is that the standard of teaching has fallen.

    However, an alternative could be that the economic cost of having children is not as devastating as it used to be, partly due to Gordon Brown’s misguided war on child poverty – where poverty was determined in relative terms. You aren’t normally allowed to even think that because it’s everybody’s right to have children, innit? Well, I don’t know, not really, not when you bail in everybody else to help clean up the mess you make and screw up the poor bastards that you invoke into the world by not giving a shit. I’d lob having kids on the responsibility side of the balance as opposed to the rights side, but obligations are unfashionable it seems.

    Or it could be something else completely different. Whatever the cause, Britain doesn’t seem to be a happy place to be young for a lot of  people, which seems rough for what is one of the ten biggest economies in the world. You’d have thought somebody would be on the case of trying to unearth the cause of this problem and nutting it at source, so that people don’t end up wringing their hands in similar conferences in 20 years time about kids that don’t seem to be getting adequate parenting… Once again, however, it takes leadership to dare stick one’s head up and ask these awkward questions about how we’ve ended up down this hole with a shovel in our hands.

    We humans aren’t logical brains on a stick and yet we have to deal with an uncertain world somehow. We need hope, grit and determination to prevail in the face of uncertainty, and what we don’t need are false numbers precisely measuring the wrong things, because measuring the right things is either impossible or it’s too hard. Better to know we are whistling in the dark; knowing that you don’t know is better than believing that you do, when you don’t know the right thing. If we applied the resources we applied to getting false metrics to actually trying things to solve the problem we’d get there a lot faster, and the third sector in particular, though well-meaning, seems terrified of being seen to have screwed up.

    Screwing up is part of life. There are no guarantees. Let’s do some real investing in people and empower them to acknowledge that known unknowns exist along with their demon spawn the unknown unknowns. To pinch another line from the fictional Jean-Luc Picard

    Sometimes, you can make no mistakes, do everything right, and still lose.

    Picard

    We’re richer than we ever were. We want for a lot less than we used to. But we’re not putting this to good use, because we seem to be building a system that existentially pisses a lot of people off. It dangles futile distractions in front of their eyes, while making essential things dearer and non-essential things cheaper. We measure the stuff that doesn’t matter because we can, and give up trying to qualify things that do matter because it’s all just too hard. That’s no way to make a better world.

    Notes:

    1. there was some supporting evidence asserted, I have to take that on trust
    2. my parents had already shared some of these essentials with me, to give them their due 🙂
    3. Specific, Measurable, Attainable, Realistic, Time-bound
    4. In a bad year you can eat a 50% fall year on year from temporal volatility. That’s ten years worth of the roughly 5% real terms assumed up trend, worst case, which leads to the old rule of thumb that you shouldn’t commit to the market resources that you will need to call on within the next 5 years. Alternatively, if you are in the market but will need to raise a lump sum, eg on retirement and buying an annuity, you should pull yourself out of the market gradually over the 5 years preceding the lump-sum payment
    5. I’m not exactly comparing like with like, there were no key stages and all that malarkey. It is possible that the bar for being considered able to read and write is set higher now than it was then

    “The endless search for evidence nobody needs, to tick boxes that shouldn’t be there, to impress people who don’t really care.”

    Yes, yes, yes – I wish I could copy this post to every single manager and professional in the NHS.
    Feedback forms, stats to the nth degree, targets, balanced scorecard – this is all it is and it actually doesn’t IMPROVE anything. It’s just “make work”.
    But if you ask the question “Why?” or “What for?” at a meeting, you become regarded as a troublemaker. And after a while you become resigned to the “groupthink” and stop saying anything as you seem to be the only one asking those questions.

    I am a professional – let me do those things that make me a professional – not produce endless stats or provide “evidence” for quality systems. It can all be manipulated anyway. You can have a quality “system” but it will still produce crap at the end.

    And I hate the cliche “lessons will be learned” – I want to throw things at the TV when people say that – what it actually means is, “nothing will change until the next time we get caught out”, when we will trot out the same phrase again.

    @Sara crikey, it’s kinda scary that this rot seems to be far worse in two key area for the well-being of people in Britain. The NHS as you say, and from this existential howl it appears that the problem exists in our State schools too.

    The Firm isn’t getting any better with that either from reports from the front-line, but at least if the vile performance management seizes the system up nobody dies or has their life-chances shafted…

    That was a depressing but familiar read.
    Trouble is everyone is so scared of losing what little funding they get, they can’t / won’t point out what a lot of old cobblers it is (even if they haven’t bought into it).
    Does a bunch of tick boxes mean that the elderly patient isn’t starved and dehydrated and dirty? – No, it doesn’t, because care is a complex, messy business for complex, messy people and done by complex, messy people (a lot like education). It’s not making widgets.

    27 Jan 2014, 2:32pm
    by Matt Smith

    reply

    I know it’s not for everyone, but have you watched Adam Curtis’ “The Trap”? It covers similar ground to your piece, although I think your emphasis on how paralysing the metrics are is an important development of this.

    @Sara- With Ermine’s permission, you could maybe forward the post to Roy Lilley @ NHSmanagers.net??
    @Ermine – another very enjoyable read but maybe copped out a bit when it got to people’s responsibility when having children:-)

    @Sara that’s the problem – everybody being scared. You need to become of independent means to be able to afford to critique the claptrap, and then you may as well just walk away… You’re welcome to forward to that Roy geezer though 😉

    I demanded (and received an answer) for the distribution of marks into the 5 buckets of performance management. Tragically I’d need 100 results to make this statistically meaningful – that’s 25 years of quarterly results, so they were safe in the knowledge that they weren’t going to be caught out forcing the distribution of results by statistical analysis. Which I should have thought of before I picked that fight with the head of division.

    @Matt that looks like a great programme – it’s to be had at archive.org. I enjoyed Century of The Self.

    @Romany ah but the signal was picked up, was it not? And I’m getting a little bit more mellow being retired 😉

    Too reasonable by half ermine, metrics are the devils own beachhead for the institutionalisation of psychopathic behaviour.

    I dodged the bullet for a couple of years with the “waiting for emergent company strategy to align my own objectives and KPI’s with” line, which worked well, until it didn’t 🙂

    Watching the public sector getting mashed up with this stupidity is all the more painful because you can’t just walk away from those jobs.

    However the whole premise is irrational and unstable, it all falls apart in time.

    […] already had the rant about how The Man is gamifying the office and turning it all into a Kafka-esque numbers game. But you can eventually buy your way out of The […]

    […] perfectly capable of engineering, still. It was the stupid gamification of the workplace and nutty management that I tired of and made me want out. Yes, maybe I was more mentally unstable […]

    […] in the workplace that turned what had been an interesting career for the most part into a gamified paint-by-numbers miasma of mockery, metrics and mendacious quarterly ‘evidence’ for the performance management system is also providing new routes to market and ways of […]

    […] to try and work out what you are going to start to all do, it just grows, along with the empty metrics and targets collated because it can be done 5. ERE again identified the problem – the workplace is […]

     

    Leave a Reply

     
  • Recent Posts

  • Subscribe to Simple Living In Somerset via Email

    Enter your email address to subscribe to this blog and receive notifications of new posts by email.