empty metrics are strangling vision and leadership

I’ve spent the last year and a quarter trying to avoid anything that smells like work, but stood in for someone who fell ill at the last minute at a conference on how to leverage the private sector and the third sector in Suffolk. There’s no money in local government, hence the absence of the public sector in the roll-call…

Some things are clearer when you stand back and look at them from a distance. I listened, and much was about ‘collecting evidence for metrics’, and somewhere in the back of my mind a refrain was building. It reminded me of all the stuff on performance management instead of inspiration, and I realised that what has gone wrong in much of business is that process is being elevated above common-sense in an attempt to make things more transactional, so that interfaces between companies and business units can be defined with greater precision.

Precision without accuracy means you may always miss – every time

It is the mark of an educated man to look for precision in each class of things just so far as the nature of the subject admits

Aristotle

The trouble is precision and accuracy are not the same thing. It’s too easy to end up precisely getting not what you want…

Metrics emphasise precision rather than accuracy, particularly when implemented by people without a scientific background. The Guardian has a wordy version for people who prefer narrative to visuals, but basically in the battle between precision and accuracy precision is easier to assess but accuracy is what you want. You’re better off generally shooting the right guy even if you take out the wrong-un every so often rather than always shooting the fella to the right of the bad guy.

The tragedy is that it’s really, really, easy to tell if you are precise – you always get the same answer, or at least the spread is small. Whereas it’s a bugger to know if you are accurate, particularly if you are trying to find out something unknown. If your spread is wide you may well be centred on the right target but your opposition will holler the place down yelling out at your inconsistency. So in practice we focus on measuring things that tend to be consistent, even if the answer is wrong because we had to ignore too many pesky variables that throw the accuracy off.

Process makes us focus on metrics, and thus we easily end up with a culture with

The endless search for evidence nobody needs, to tick boxes that shouldn’t be there, to impress people who don’t really care.

There’s an answer to this. We used to know what it was. And we desperately need it now. It’s called leadership.

We humans live in an uncertain world, and yet we still have to take decisions. We will get things wrong sometimes. Some things are amenable to analysis and science, and these should be assessed that way. Engineering projects often have a lot of this – nowadays we can determine if the bridge will stay up given the likely loading, unlike the Victorians, who often had to over-engineer some of their great works after they discovered the hard way than erring to over-engineering was preferable to under-engineering.

Not everything to do with science is amenable to exact analysis, particularly when it comes to the unknown unknowns, which is part of science’s core business. Sometimes the balance of probabilities is the best you can do.

James Delingpole is absolutely right. It’s perfectly possible that global warming doesn’t exist and/or if it does then it’s not anthropogenic. It’s just not incredibly likely given the data that has been collected.

Many things in public policy are a balance of opposing forces. There’s never enough money, never enough time, there’s always more than can be done, and sometime alleviating one thing can lead to unintended consequences. There is no right answer, and cod metrics and getting your tickboxes in a row won’t help. All they will do is deflect the blame away from you and into the ‘Lessons will be learned’ catch-all refuge of last resort.

Nowhere was this clearer than in the health and environment session. Britain seems to suffer from an appalling level of mental health problems, and one of the proposals was to get children to experience the natural environment before they were 12 – apparently the period before 12 is when a lot of attitudes and predilections are formed[ref]there was some supporting evidence asserted, I have to take that on trust[/ref]. There is significant anecdotal evidence that exposure to the natural environment and exercise are favourable to good mental health.

Now I had the assumption that this is why every child has parents, in the case of a species such as humans that has a long period of dependency. However, it doesn’t seem to happen often enough in today’s Britain, and so the job falls to the schools in loco parentis

After several proposals on how to measure the success of the result if something were done along these lines, I couldn’t hack it anymore, and told the assembled folk how this had happened for me. I grew up in New Cross, in London, in this area, and the primary school was also there.

The teachers did not rely  on metrics. They relied on something older, something that has served humanity for thousands of years, mostly for good though sometimes for ill. It’s called

Leadership

The teachers knew they were teaching a bunch of city kids. So one spring, when it was sunny, they took us to Telegraph Hill, I believe, which is to the southwest of this map. I still remember lying on the grass and looking at the deep yellow of buttercups as they explained flowers, and how been pollinated them, and the cycle of nature[ref]my parents had already shared some of these essentials with me, to give them their due :)[/ref]. There were no metrics, no customer satisfaction forms to fill in, no key stage whatever garbage. The school headmaster knew his school, its intake, and led from the front. Resulting in a memory that stayed, even after four decades have rolled by.

Curiously enough the Ermine seemed to carry people with the idea. Leadership did seem to matter, and indeed several other delegates recalled their schools doing this without funding, using local resources. Despite that the challenge was countered with another issue  as people said that schools have no funding to go into the natural world.

Now if my inner city primary school could find green space within school crocodile walking distance then I have difficulty in believing that this is insurmountable, but I let this failure of leadership be – I was primarily as an observer, this isn’t my problem. I suspect the perfect is being the enemy of the good here. That’s again where leadership trumps box-ticking. Sometimes good enough is good enough, and it’ll do – you do the best you can with what you have to hand and then move on 😉 It would be nice to have the money to do it at Minsmere, but if the local rec or the town park is the the best that can be offered then it’ll do. I can show you sparrows, blue tits, great tits, robins, blackbirds, starlings, black-headed gulls in the rec near my home. a couple of hundred yards away in the town cemetery there are jays, crows, squirrels, green woodpeckers, chaffinches, greenfinches, great spotted woodpeckers, song thrushes and sometimes fieldfares. It really isn’t hard.

Leadership is what we seem to be sorely lacking now, in many areas of life both corporate and political – the courage to have convictions. Its handmaiden, initiative, seems to be a little bit in short supply too, run out of town by the processes and procedures that seem to gum up the works. Where decisions can’t be avoided, we now often try and hide behind precision without accuracy, meaningless metrics collected without awareness of sample bias.

We establish processes and systems so that faceless and distant bureaucracies can micromanage the operation from afar, maximising their metrics and results, run by vapid caretakers in systems designed by committee. When something goes titsup in such a system, it is never anybody’s fault, and inquires are held and ‘lessons will be learned’. Colour me unreconstructed, but sometimes a good honest apology along he line of ‘we really screwed up there, sorry to the people affected and I will do my damnedest to avoid this happening to us again’ would be a good deal more convincing than the mealy mouthed ‘lessons will be learned’ bullshit. Who will learn the lessons, and if they were hired to do the top job, then why the bloody hell are we paying them all this money to duck and dive their responsibility for this SNAFU rather than taking it on the chin?

Every organisation should have the equivalent of this

“If we’re going to be damned, let’s be damned for what we really are.”

Picard, Encounter at Farpoint

At work these spurious metrics seem to be what Farhad Manjoo calls the ‘Gamification’ of the Office (hat tip to Monevator for this one). I saw that first hand at The Firm and it did for me – it robbed working for a living of everything worthwhile. The Firm contracted a company called Successfactors to implement a software system where quarterly you’d have to gather evidence on what you were doing, how you were doing, get SMART[ref]Specific, Measurable, Attainable, Realistic, Time-bound [/ref] objectives.

And with this endless measurement and metrication, the fire began to flame out. I found it alienating and hateful. For over twenty years I hadn’t had any general problem with the annual appraisement process  – I was assessed by people who knew what I was doing and its setting. I didn’t always agree with everything but  the overall process gave a fair balance between short term delivery and long-term improvement of one’s craft. Above all I could respect the people, their competence, skill and judgement.

Successfactors and the American HR consultants turned that on its head. I am introverted, and to some extent a loner, I believe in achieving results through understanding and learning. I do my best work on  my own or with one or two others. I’m perfectly happy to present things once I have enough results to present, but what I don’t respond well to is endless interrupts. I could live with that, but the stupid performance management stuff, quarterly, with metrics of people giving 5 minute seminars and how often they used Instant Messaging that counted for 25% one one’s score drove me nuts. People ended up taking up massive amounts of time on all hand events giving 5 minute seminars on stuff that could be better learned from Google.

There’s no point me detailing the particular problems of implementation. Farhad Manjoo did that far better than I can – I recognise every one of the ills he cites – The Firm was there ahead of time.

Too much energy, resources and time are being spent on the worship of false metrics, in an attempt to avoid the difficult business of leadership – the art of taking decisions in the face of uncertainty, and knowing that sometimes you’ll be wrong. It’s often better to be wrong 20% of the time than to be right 20% of the time and paralysed by analysis 80% of the time. Micromanagement does that to people, and the worship of false numbers leads to micromanagement, and so the cycle turns and starts anew.

Numbers have their place – it is knowable whether the bridge will stay up. It is knowable whether cheap Poundland batteries are actually cheap or they just look cheap. There are places where you have to run with the balance of probabilities. there are other places where it’s good to ‘fess up that you know that you don’t know – the classic Rumsfeldian treatise on epistemology and the known unknowns and unknown unknowns. A lot of economics seems to fall in the latter category.

This is because systems with feedback mechanisms in them are the worst of all – they are often susceptible to chaotic behaviour that cannot be known other than in terms of probabilities, and then only if you can characterise and model them properly. Social sciences have a lot of this, where individual humans are adaptive creatures all trying to maximize their experience, with different time lags in the information flows. It’s why the stock market has such a shocking volatility, as all of this is worked out between independent actors, it is the analytical principle that underpins the efficient market hypothesis behind passive investment. The fundamental presumption behind passive investment is the myth of continual growth – it’s worked so far and has given a reasonable long-term trend. You are buying a slice of that and going along for the ride, with diversification averaging out company and sector volatility and decades long accumulation of holdings integrating the temporal volatility against the desperately low 4-5% or less real over inflation trend.[ref]In a bad year you can eat a 50% fall year on year from temporal volatility. That’s ten years worth of the roughly 5% real terms assumed up trend, worst case, which leads to the old rule of thumb that you shouldn’t commit to the market resources that you will need to call on within the next 5 years. Alternatively, if you are in the market but will need to raise a lump sum, eg on retirement and buying an annuity, you should pull yourself out of the market gradually over the 5 years preceding the lump-sum payment[/ref]

However, systems involving humans aren’t perfect – it leads to old hands acknowledging that the market is not efficent, albeit not inefficient enough to get ahead using that awareness. Or perhaps yes, but the opportunity is temporal – we humans are prone to collective mania and the madness of crowds where everyone decides the world is ending 😉 This human capriciousness applies across the economy and across the social sciences.

Something else I heard at the conference, for instance, was that there really do seem to be an awful lot of young people who have been deprived of, what in my day would have been called competent parenting, which is why it falls to the schools and/or society in general to try and compensate for the lack. It amazed me that everything seemed to be about identifying the problems without asking the fundamental question how the bloody hell did we get here?

In my inner-city London primary school which had a single class of 32 of each age group, there was only one pupil that left unable to read and write. In relatively affluent Suffolk, forty years later, a fifth seem to struggle[ref]I’m not exactly comparing like with like, there were no key stages and all that malarkey. It is possible that the bar for being considered able to read and write is set higher now than it was then[/ref] Something has changed, and not for the better. One common theory is that the standard of teaching has fallen.

However, an alternative could be that the economic cost of having children is not as devastating as it used to be, partly due to Gordon Brown’s misguided war on child poverty – where poverty was determined in relative terms. You aren’t normally allowed to even think that because it’s everybody’s right to have children, innit? Well, I don’t know, not really, not when you bail in everybody else to help clean up the mess you make and screw up the poor bastards that you invoke into the world by not giving a shit. I’d lob having kids on the responsibility side of the balance as opposed to the rights side, but obligations are unfashionable it seems.

Or it could be something else completely different. Whatever the cause, Britain doesn’t seem to be a happy place to be young for a lot of  people, which seems rough for what is one of the ten biggest economies in the world. You’d have thought somebody would be on the case of trying to unearth the cause of this problem and nutting it at source, so that people don’t end up wringing their hands in similar conferences in 20 years time about kids that don’t seem to be getting adequate parenting… Once again, however, it takes leadership to dare stick one’s head up and ask these awkward questions about how we’ve ended up down this hole with a shovel in our hands.

We humans aren’t logical brains on a stick and yet we have to deal with an uncertain world somehow. We need hope, grit and determination to prevail in the face of uncertainty, and what we don’t need are false numbers precisely measuring the wrong things, because measuring the right things is either impossible or it’s too hard. Better to know we are whistling in the dark; knowing that you don’t know is better than believing that you do, when you don’t know the right thing. If we applied the resources we applied to getting false metrics to actually trying things to solve the problem we’d get there a lot faster, and the third sector in particular, though well-meaning, seems terrified of being seen to have screwed up.

Screwing up is part of life. There are no guarantees. Let’s do some real investing in people and empower them to acknowledge that known unknowns exist along with their demon spawn the unknown unknowns. To pinch another line from the fictional Jean-Luc Picard

Sometimes, you can make no mistakes, do everything right, and still lose.

Picard

We’re richer than we ever were. We want for a lot less than we used to. But we’re not putting this to good use, because we seem to be building a system that existentially pisses a lot of people off. It dangles futile distractions in front of their eyes, while making essential things dearer and non-essential things cheaper. We measure the stuff that doesn’t matter because we can, and give up trying to qualify things that do matter because it’s all just too hard. That’s no way to make a better world.