Build or burn?

KPIs are broken

KPIs and their cousin the balanced scorecard are great in theory but not in practice.  In each occasion I’ve prepared and reported KPIs and balanced scorecards, I never escaped the feeling that the whole enterprise was a gigantic waste of time.  There’s something about KPIs that is broken.

The theory behind KPIs is quite strong.  ‘If you can’t measure it, you can’t manage it’.  In many circumstances, I’m sure this is true.  But I’m not so sure it holds at the top table.  At senior levels I’m tempted to counter with a different maxim: ‘If it can be measured, it’s not strategically important’.

Perhaps the problem lies in my execution.  Here has been my experience:

1)  Moving the goal posts

No sooner do you set a target before people conspire to fiddle the reporting to make it look like it’s being achieved.  I once worked in an exporting business.  Exporting is a time-consuming process.  A ship is not a very fast mode of transport.  Moving containers onto lorries and into warehouses is also quite slow, especially if there are complicated customs paperwork to process.  In this particular business, our suppliers never put us at the front of the queue because our volumes were quite small compared to their usual orders.  In short, our customer service was a big problem and order fulfilment times were a key indicator of the service our customers were experiencing.  Over the years, our shipping department had fine-tuned the way of measuring when customer orders had been received and dispatched to the point where it was an accurate measurement of how well the shipping department performed but, given everything else in the supply chain, bore little resemblance to when the customer experienced.  They fiddled the figures to reflect what they could control.  Strategically, that business had a structural problem with its suppliers but we spent all our time arguing with ourselves and our customers over the definition and measurement of KPIs.  This illustrates what always happens if you select a KPI as a tool of strategic analysis but then start using it as an operational tool.  The goalposts get moved.

2)  Too much information

The NHS is an enthusiastic reporter of KPIs.  I recall leafing through a board pack for an NHS hospital which contained page after page of KPIs, graphs and measurements of all kinds.  They had a whole department of people called something like ‘medical statistics’.  This group churned out data for internal and government consumption.  I can tell you no one would have time to read it all.  There were over 30 pages of reports prepared on a monthly basis.

3)  What does it all mean?

Every balanced scorecard I’ve ever produced or been asked to produce has always looked, how can I put this, a bit odd.  In an attempt to get a lot of different type of information on a single page, it has typically been split into boxes, often with bold use of colour, sometimes with graphs, sometimes with tables, with a little bit of commentary beside.  To avoid the font becoming too small, I’ve seen it developed for printing on an A3 page format.  The overall result has always been a mess.  Faced with this kaleidoscope of a report, most board members have found it unintelligible and turned to the document’s owner and asked for a verbal report of what it all means.

4)  A stick to beat someone with

I was once responsible for an FM contract in which we broke the service we wanted down into something resembling a job description and called everything that we wanted done a KPI.  Given the tortuous arguments we had had earlier about what was key and what wasn’t, I was content that we had at least gone down a comprehensive path and could clearly state what service level was required.  However, we then went on to develop a concept of ‘service failure points’ to try to measure what level of service failure was acceptable and what the consequences of that failure would be.  The system we developed, advised by an FM consultancy, was ridiculously complex and unfathomable on even the second and third reading.  It was designed to form the basis of making deductions to the FM contract to hurt the FM provider for underperformance.  At the end of the tender process I sat down with the successful supplier and suggested we completely disregard the whole service failure point system and instead agree an alternative between themselves and our FM manager.

5)  Canary in the mine

The best KPIs are one’s that give you advance warning of trouble ahead.  Any KPI that informs you about the state of your forward order book is going to be useful.  This is particularly relevant for organisations that work on long-term (eg over 1 year) contracts.  Metrics that tell you about your customer service are helpful too.

Lessons?

If these examples have a common theme it is that you should be cautious before turning KPIs and balanced scorecards into the sort of regular monthly reporting of, for example, financial and management accounts.  Look for different governance opportunities (eg the strategic review) to communicate KPIs in addition to the regular monthly board meeting.  And generally, keep the reporting of internal metrics focused on junior and middle management levels, leaving only the externally-focused KPIs for senior management.  As old-fashioned as it seems, sometimes a table of a numbers can be easier to absorb than all the flashing lights of a balanced scorecard.

Quantitative analysis can only take you so far when it comes to strategic decision making.  Beyond that point, the ability to make reliable measurements is limited because of uncertainty.

View all blog articles