Benchmarking- When is a First a First?

Comparison of the performance of subsidiaries in conglomerates, different UK NHS Trusts and the many English universities tends to involve benchmarking.  This entails measuring the performance of an organisation's products, services, processes, or outcomes against those of other organisations.  It often involves comparison of certain parameters, known as key performance indicators (KPIs).  These may be profit margins, return on capital employed, client retention rate and customer satisfaction in commercial businesses.  They could include average waiting times, complication rates and mortality values across hospital trusts.  KPIs often comprise of metrics such as continuation rate, progression rate and degree classification levels in UK universities. 

However, are KPIs and metrics a reasonable method to compare and benchmark the performance of different organisations?

Let us take as an example, identical triplets who attended three different universities and studied three very different courses, one in health, one in social sciences, and one in computer sciences.  Remarkably, these triplets received the same marks, some extremely high and some quite low, in their respective modules in the second and third year.  Prospective employers would expect them all to receive the same degree classification, but some universities use different processes to benchmark and classify degrees. 

Triplet A attended University A to study Course A which classifies degrees based on the average of the best eight module results from the twelve modules completed in years 2 and 3.  Those students who receive an average of 67.5% or higher from their best eight module results receive a first class degree.  Triplet A was awarded a first class degree, with an average of 74.5%. 

Triplet B attended University B to study Course B which classifies degree based on all the results in years 2 and 3, calculating the average for each year, and then weighting 1:2 (so year 3 has a higher weighting).  Those who receive an overall weighted average of 68.5% or higher are awarded a first class degree.  Triplet B was awarded a second class degree, with an average of 67.8% (which would have been awarded a first class degree from University A). 

Triplet C attended University C to study Course C which classifies degree based on all the results in years 2 and 3, calculating the average for the two years (with no weighting).  Only those who score an average of 70% receive a first class degree.  Triplet C was awarded a second class degree, with an average of 64.1%, despite receiving the same module marks as Triplet A and Triplet B in the second and third years. 

Prospective employers would rank the triplets, A, B then C based solely on degree classification (as the KPI). 

Comparison of the triplets’ degree classification (and ranking) becomes much harder after reviewing the courses.  Course A included studying an average of 4 topics per module which were assessed by essays and group work and practical tests; Course B included an average of 6 topics per module which were assessed by essays and some open book examinations; Course C included an average of 8 topics per module which were assessed by close book examinations aligned to professional external examinations. 

Simple benchmarking and over reliance on KPIs can be a risk: it tends to ignore the qualitative issues and the environment circumstances.  It should be a starting point not an end point.  No business can be judged on numbers alone.  And people are much more than just numbers and grades.

The Management Accounting Team

Finance, Accounting and Economics Department