Why bother benchmarking?
Published in ComputerWorld UK
Context and the battle for continuous improvement
IT benchmarking can seem esoteric, dull or even nerdy – until you realise that it is can be a critical catalyst for business transformation. How so? Because as the saying goes: ‘You cannot aim to go somewhere unless you know where you are starting from.’ And benchmarking tells you where you are – not where you would like to be.
What is it?
To begin with, what actually is benchmarking? Essentially it is the comparison of an organisation’s practices and performance against A) its peers and B) industry best practice standards. But good benchmarking doesn’t just look at IT in isolation. It also seeks to identify those innovations, processes and practices that can optimise the business by, say, improving customer services, streamlining processing cycles and spending smart.
Some see benchmarking as a tedious back office task that must be done periodically to reassure management that the IT department is not underperforming their peers and/or their own previous baseline KPIs. Sometimes it is undertaken with reluctance and other times a study is undertaken when there is a service delivery problem that can’t be resolved any other way.
Some CIOs have used benchmark consultancies in the past and found the results wanting: often because the report provides general data but offers no useful guidelines as to how to use the information to implement change or improvements. Others think benchmarking is something they would like to do at some point but have more pressing demands on their ever-shrinking budget.
A business investment
To the enlightened benchmarking is a critical business tool. This view is based on the understanding that technology underpins nearly every enterprise function and the better it works the healthier the company. The 2010 Global Bench¬marking Report surveyed 450 organisations in 44 countries and found that 68% of them used benchmarking and of those that were not, 60% planned to with the next 3 years. The main benefits cited were A) improving performance and processes B) understanding how other organisations operate and C) addressing major business strategy issues.
Strategy for continuous improvement
For organisations like this, benchmarking isn’t a rear-guard action for troubleshooting – it is a proactive measure with a three-fold aim: To establish a framework for continuous improvement; to maintain or achieve best-in-class status; and to identify better value-for-money. This last is not simply about cutting costs, but rather striking the optimum balance between cost and service quality.
All this assumes, of course, that the benchmark study is done by an independent (unbiased) consultancy that not only has access to accurate, up-to-date, extensive and meaningful peer and market data, but also knows how to translate this information into recommendations for real and achievable improvements. This holds true whether the study is done for private/public sector organisations or for a service provider, because at the end of the day they both face the same issues: the need to provide the best goods and services at a competitive price to win market share, improve margins and remain competitive.
How does it work?
The first step in benchmarking is to establish a baseline by looking at every aspect of the IT operation from staffing, infrastructure components and workflows to the performance quality and cost of the services provided (eg. the data centre, desktop support, etc.). This involves a structured approach to data collection starting with surveys and stakeholder interviews. But a survey is just the start. To be of any use, the data gathered from these surveys – which depend on the biases and accuracy of participating individuals – then needs to be validated within context. What do we mean by this? Well, as an example, a survey may reveal that inhouse clients feel they are paying too much for their IT department’s services. But feedback means little unless looked at in the context of other influencing factors.
The context of cost
Some studies focus only on cost- which is not surprising given the tradition of benchmarking for the sole purpose of finding savings. But this misses the bigger point. Taken out of context the issue of cost is misleading because it is rarely about ‘cost’ alone – rather it is about ‘value’, a quality that can only be measured in relation to other influences. In IT benchmarking, there are a range of parameters that should be taken into account – such as volume (of calls to the help desk, number of incidents resolved, etc.), staffing levels, the scope of a service and the quality of its delivery, the complexity of an operation (for example, how many legacy systems must be maintained), and the maturity of an organisation’s process methodology. For example, if a company has built in ISO standards they are likely to be more efficient and cost-effective than one less structured.
Getting the big picture
Industry sector is also contextual. There is sometimes a tendency to seek peer comparison within one’s own market exclusively. Yet while being at the top of the ‘league’ may be good for morale it isn’t as valuable as knowing where one stacks up in the ‘premiership’. A cross-industry comparison with organisations of a similar size, IT complexity, service profile etc., from different industries such as government, healthcare, manufacturing, fast moving consumer goods and financial services offers much more insight. By looking at how others in different industries do things, the additional scope can offer valuable new insights into new methodologies and best practices.
Having obtained all this enriched data, it can then be harnessed for a wide range of business analyses and projections. A fast, accurate and costs-effective way to do this is by using virtual modelling which applies ‘what if’ scenarios to measure various outcomes. For example you might ask:
‘If we were to cut X staff, what would be the maximum service quality we could expect and what would be the corresponding savings?
‘If we reduce our outsourcer SLAs to Y what staff would we then need to take up the slack and what, if any, savings would we achieve?’
‘If we were to merge our IT operations with another organisation or region, what would be the estimated savings?’
Having looked at the KPIs, the next step is to compare provisioning models. These might include in-house, outsourced or a hybrid of both; traditional vs cloud-based providers; new procurement models like SaaS and IaaS; shared (rationalised) services and others. A good consultancy can help to identify the cost and efficiency levels of in-house vs. outsource provisioning. And if they have up-to-date supplier price catalogues they can also compare the most attractive price/quality options for specific services on an apples-to-apples basis.
Of course, no organisation wants to be found wanting, and so may avoid the very idea of benchmarking. But this is a short-term view. In our experience there are essentially three ways to think about improvement: The first is that you must be in shortfall if discovered to be less than perfect. The second is ‘Why fix what ain’t broke?’ The third – and the one that gets the best results – is realising that everything everywhere should be in a continual state of improvement and the only way to keep growing is to keep changing.