Metrics-Driven Analysis, Modelling and Benchmarking of IT

  • NHS IT budgets are under continuing pressure
  • Where to cut back while minimising business impact?
  • What services can be delivered with reduced budgets?
  • At what service levels?
  • How can metrics analysis and virtual modelling help?

The “Scalpel” Approach to IT Optimisation and Transformation

A modelling Methodology Rooted in Facts to Support Decision Making

NHS IT is experiencing a revival of sorts, with increasing recognition of its strategic importance to patient outcomes and with targeted funding for deployment of clinical solutions, however, pressures on IT budgets are never far away. In this paper we discuss a metrics based approach to assessing and modelling the underpinning IT infrastructure and its management, suggesting a measured approach to informing decisions with direct impact on staffing, knowledge retention, processes and quality of business support.

Background

Pursuit of Efficiency in Changing Times

While an ageing population may be a key factor driving an increased demand for NHS services, rarely a week passes without the NHS funding gap making the news. With the prolonged budget freezes of recent years, “non-essential” budgets such as IT are often squeezed. Given the on-going debate on the size of the NHS funding gap, the sustainability of the service, and related issues, the challenges ahead for NHS IT remain critical.

While directing funds towards clinical solutions will help, such significant transformation programs are creating additional IT pressures and workloads.

In the coming year it is expected that focus will increase towards the provision of a more integrated health and social care service, which implies better integration of systems between the NHS and local councils.

Finding big savings, in the backdrop of the presently pressured economics of IT raises serious concerns. It brings into focus the potential avenues available, if any, to find these savings in an already pressured and transforming IT environment.

The Integrated Digital Care Fund

NHS England is focusing its IT funding towards projects that capture and link clinical and care information. As with the first tranche of funding, it is also promoting further investment in specific clinical solutions.  Trusts which have obtained funding in the first tranche will be assessed on a “clinical digital maturity index”. Those seeking to apply for funding from the second tranche will need to present a suitable business case, evidencing a strong ROI or “value for money”, in line with the guidelines and funding needs to be matched 1:1 by the Trusts. However, infrastructure investments and proposals that do not demonstrate a clear contribution to improving care pathways and transforming clinical/professional workflows are not eligible.

Trusts seeking funding need to demonstrate a clear understanding of the need for clinical and professional leadership from the boardroom to the frontline, workflow and service redesign capability, robust project management, a credible sourcing and supplier management approach, effective governance and access to relevant technology expertise.

This translates into developing a detailed IT strategy which has the commitment of the board governing the Trust. This then needs to be backed up with a clear transformation plan to deliver the strategy.

ImprovIT has worked with several NHS Trusts to help in these areas.

The Clinical Digital Maturity Index

To drive meaningful adoption of the funded clinical systems a maturity index has been developed by EHI Intelligence in partnership with NHS England. The model is focused on the take-up of solutions and variances, aimed at driving better patient outcomes through use of IT solutions.

Whilst the journey to “digital maturity” is focused on adoption of clinical solutions, it does increase the dependency on the people, knowledge, skills and processes required to run an optimized and efficient IT service.

Modelling  

Scalpel Approach to Efficient Transformation

The following is inspired by true facts and can relate to any NHS Trust.

Faced with a savings challenge, an NHS Trust is to identify and implement large cuts (of the order of 7% per annum) over the next three years. This situation is compounded by several services being protected; thus the cuts have to be largely found from expense budgets of support services, such as IT.

In recent years IT has already made considerable cost reductions. This has been achieved through careful management of resources, re-negotiating of existing contracts, changing of suppliers, leaving vacant posts unfilled and also examining sharing options.

Regardless of the loss of resources the IT department has managed to maintain IT services at an acceptable level.  However, it is left with little flexibility for implementing new projects. There is a plethora of projects to deliver to enable the Trust to meet its objectives. Whilst other departments may also need to transform, IT is seen as crucial in helping realize outcomes. This will require new, and in most cases additional IT services, along with appropriate and suitably skilled staff and knowledge.

Undoubtedly all these factors combine to put a high workload on the IT staff and could jeopardize day-to-day service.

While there may be potential for merging or sharing selected IT services, these remain largely undefined in the context of on-going deployment and transformation programs. In the interim therefore maintaining key IT staffing becomes imperative.

Options

In order to meet the challenge of the budget cuts a few potential approaches are possible. These may include staff cuts, reduction in services and handing over responsibility of IT to an outsource provider. Simple as they appear prima-facie, these approaches carry risks if undertaken without careful and measured consideration (a scalpel and not a hammer). There are issues to consider, such as the right balance between service requirements, service levels, reliability, repeatability, skills retention and its longer term impact.

Staff cuts directly increase the risk in the medium and longer term to the delivery of the service needs and service levels. The option of reducing available service scope and levels is unlikely to be acceptable in the climate of more demands on technology; and outsourcing for cost savings alone is never an advisable approach.

An alternative, more considered approach, is hence most advisable.

Approach

This involves undertaking a detailed metrics analysis of the present state of IT, and the use of modelling to identify the right balance between staffing levels (thus cost) and the quality of IT services that could be delivered (the best practices advisable and the optimal costs).

This approach delivers the benefit of analysing the several inter-related factors which drive IT cost, complexity and service. Using the correlation of these composite facts, the real options available to the Trust can be identified; alongside their impact on services experienced by users and budgets.

Unfortunately high level data obtained from surveys does not suffice to achieve a modelling capability or to identify the impact of alternative delivery options and alternative service levels. To support this approach it is advisable to engage an independent and specialist expert, particularly one with a deep understanding of NHS Trust IT operations and pressures. The insights on present day costs, changing cost drivers, best practices within and outside the NHS world are necessary to undertake such modelling.

ImprovIT have a strong and recent track record of helping NHS Trusts through several expert assignments to support management. These have included IT metrics analysis, modelling and benchmarking studies, development of IT strategies and transformation plans and IT services sharing feasibility studies. These reviews have supported Trusts seeking foundation status, planning for funding applications and supporting optimization challenges.

Real World, Live Data 

With new investments into clinical solutions, the IT infrastructure and the management structures i.e. the people and processes in place to manage IT, become fundamental. The analogy of automobiles (or clinical solutions) needing a well-managed and maintained motorway (IT infrastructure) comes to mind.

An independent assessment and real world comparison of all the IT service towers becomes highly valuable when new solutions are to be deployed. Such assessments are fairly detailed and use data spanning several inter-dependent variables driving the service costs and performance. For instance, these variables may include the volumes, the staffing, productivity, quality and the scope of the services. Also reviewed are complexity, management processes and the speed of response.  Along with other factors, this data provides an understanding of scale and complexity of an IT service being delivered. This then allows the selection of a suitable peer group to compare against for the service being analysed.

For each services tower studied a detailed report is produced, focusing on the Key Performance Indicators (KPIs), identifying the gaps and therefore the potential areas for improvement and savings. This also presents a highly a relevant dashboard for executives to manage.

Metrics Modelling

Below is a list of IT services towers that can be studied for benchmarking and modelling. Also listed are a few examples from the several metrics considered for each IT service tower.  There are several more metrics, KPIs, indexes and ratios which are often required to help answer specific questions and “what if” scenarios. These obviously depend on the needs and scope of the study and the key management questions being addressed.

  1. Service Desk/Help Desk

Example Metrics –

  • Cost per Contact
  • Cost per Incident
  • Incidents per FTE
  • Call abandoned rate
  • 1st Time Fix rate %
  1. Servers – Hosting & Management; Cloud provisioned services

Example Metrics –

  • Cost per logical server
  • Ratio logical to physical servers
  1. Data Centre & Storage

Example Metrics –

  • Cost per installed GB
  • Cost per Sq/M
  1. Desktop/Workplace
  2. Networks – LAN & WAN
  3. Applications Development & Support
  4. Infrastructure Projects
  5. Outsourced: SaaS or Managed Services
  6. Email Services
  7. Printing Services
  8. Telephony

For each service tower data is gathered and analysed around 7 key attributes that govern the cost and performance of the IT service tower in question:

  • Costs (or prices, if buying in a service).
  • Volumes of all aspects of the IT service.
  • IT Staffing (expertise, capability, qualifications & knowledge).
  • Quality (Service targets / defined and achieved).
    • Service scope (what is included or excluded)
    • Complexity (the business aims of the organization, legislation, regulation, equipment mix, sites supported, etc.)
    • Processes (what is in place/ how well used that helps the IT department function optimally).

    Each of the above attributes is inter-linked and a change to one impacts the others.  Therefore it is possible to undertake accurate modelling from a standpoint of any of the attributes.

For example:

Staff Cuts – what services levels are possible with the remaining staff? (How should they be best organized, with what skills, experience etc. and what are the cost saving possible?)

Service Level Cuts – what staff will be needed and what cost savings will be made?

Simplify a complex situation – what is the impact on quality (and usually customer satisfaction) staffing and costs?

What is the impact of alternative delivery methods on costs, staffing, services (in-source, outsource, sharing services or merging IT departments?)

What benefit could there be by committing to Process Improvement? – improved service quality and user satisfaction at lower costs.

Further  cost savings – what staffing, service quality, services, etc., are possible (optimum) for a revised cost and provide the IT services in the same manner as now (i.e. what can be achieved for 75% of the budget for an in-house IT service provision?)

Typical Modelling Outcomes

Depending on the scope of the metrics analysis undertaken, modelling can seek out answers and options to inform decision-making on broad to precise questions, such as:

What are the strengths and weaknesses of the Trust’s IT?

What is the optimum service delivery and staffing that can be achieved for a defined budget?

What opportunities for cost saving and efficiency improvements exist, when compared against appropriate peers (from all markets)?

What are the “best practice” and “best costs” recommendations when comparing against suitable peers?

What costs and service levels are possible by implementing alternative delivery models?

What IT services are best suited for sharing or merging (this does require similar analysis of the potential sharer.)

Study Scope

To get best results from a commissioned study, it is ideal to identify and agree with the specialist consultancy engaged the most appropriate scope and solution. Below are the starter options tailored by ImprovIT, based upon extensive work done with NHS Trusts.

Base Modelling – This analysis assumes no change to staffing levels or service delivery models i.e. an “as- is” model. It seeks to assess the str

engths and weaknesses of the present state of IT, using various metrics and measurements, to help find answers to questions such as:

  • What scope and service levels can be delivered with reduced IT budgets?
  • Which critical areas need strengthening?
  • Which services can be delivered at lower service levels with minimal impact?

The study includes guidance on the key metrics to track and manage to ensure the correct transition to a future state. The scope of study is often preferred by public sector organizations challenged by budget constraints. It is highly beneficial in establishing the baseline of the current operating model. It also lays down the foundation for further examination (as described in options 2 and 3) and informs some imperatives for IT strategy and transformation planning.

IT Services Benchmark – This analysis goes deeper and includes Base Modelling. The metrics and measures for IT infrastructure and applications portfolio (selected sample) are analysed in detail. This is benchmarked against suitably selected peers for each service being delivered. Peers are selected from various sectors, based on a similar scale and complexity of the services being delivered. The outcome of such analysis helps seek out further optimization opportunities, brings our best practices, targets for key metrics and inputs to the transformation planning ahead. This provides the best value in terms of time-line and cost to achieve maximum insight and guidance. This option is often the preferred route.

Advanced Modelling – On the basis of the above options, further questions requiring answers can emerge – such as – alternative service delivery structures and approaches; the potential trade-offs and risks; possibility of sharing services with another entity (requiring similar analysis of the other entity in question), and so on. Advanced Modelling can vary significantly as questions to be addressed can be unique and context sensitive.

There are some future options to help guide the Trust on tracking on-going progress, to assess that the change programs and transformations implemented are on track with targeted improvement outcomes and cost savings. For Trusts undertaking IT Benchmarking the following further options are also available:

Provision of updated benchmark peer data (usually 12 monthly). This guides on how the market has moved, and allows comparison with the present current state of key measurements put in place by the Trust.

An “after-state” benchmarking study within 12-18 months, to inform the Trust of how well their transformation programs have delivered and how their current IT efficiency compares with the changing world.

A study to analyse and advise on the potential of Sharing of IT services is a different proposition and needs addressing in a collaborative manner with all potential entities concerned.

Conclusion

NHS IT has never been and will never be static. With new investments the pressures do not ease, but actually intensify – adding new management challenges, while seeking out additional savings while keeping IT services maintained at levels the business requires.  The “hammer” approach to cost cutting is fraught with risks short and long term. In this climate, the more sophisticated “scalpel” approach is recommended – one which is measured, based on facts and real world comparisons can infuse fresh thinking and methods. In this approach the help and support of an unbiased independent expert consultancy can be invaluable. There is need to create and study “virtual models” of a Trust’s IT operations, basing it on the correlation between key variables of service cost and performance. This is the most efficient and effective method finding answers to the tough questions, of cutting costs without bleeding the service.

IMPROVIT

Headquartered in the UK, ImprovIT is an international business technology consultancy that specializes in the use of IT benchmarking disciplines and analysis of business technology to deliver metrics driven advice to executives enabling improved value decisions.

ImprovIT’s Virtual Modelling methodology, allows simulations of inter-related parameters, such as cost, complexity, quality, and so on; guiding recommendation and decisions on independent and fact-based analysis, rather than opinion. Underpinning the rigorous methodology is a regularly enriched Data Repository of granular IT metrics, refreshed through their regular engagements with enterprises and with services providers.