Lies, damned lies, and graduate statistics
Posted by
At T&P we are strong supporters of providing advice based on hard data – see our research into high potentials, our career management advice based on over 1600 interview of people with successful careers, and our hard figures presented for awards. However sometimes statistics reported in the press are not at all helpful, and can misinform key decisions.
For example, recently the Guardian published an article with a headline blaring “Oxford and Cambridge outperformed on employability”.
This article contained a medley of statistics, which basically reported that 6 months after graduating, 8% of Oxford students and 5.1% of Cambridge students were not in jobs or further employment. And some FE institutions had lower percentages than this. The conclusion drawn was that Oxbridge candidates were less employable than the other universities’ graduates. This shows the capacity of even the quality press to create stories out of undigested facts.
Talking to a university contact about employability statistics, she was quick to point out that when the ‘employed’ figure is not broken down into graduate level/non-graduate level jobs, it’s very misleading. Her own university had an almost perfect record – 99% in employment or FE, 6 months on. “It’s because our university attracts a lot of people from poor backgrounds. They can’t afford to sit at home and wait for the right job. They will take any job immediately after graduation, in order to earn their keep.”
Is it any surprise that there are more Oxbridge students sitting in leafy gardens, making targeted applications, 6 months on?
This is why I like the quote “All measurement corrupts”…
The moment you measure something, the people with a vested interest will start to find creative and often counter-productive ways of ‘winning the game’. Always worth noting, any time you decide to measure anything.
Another example follows, with perhaps a daring suggestion on how to reduce the 86 graduate applications for AGR member jobs that force most graduate employers to wade through endless applications. The high number of applications forces employers to use shortlisting mechanisms. Unfortunately, the great majority of shortlisting mechanism have a serious negative impact on social mobility. For example, there is a proven negative impact of using a 2i cut-off, using UCAS points, using life experience measures, or limiting your search to a restricted group of universities. Each of these will reduce the chance of appointing graduates from non-professional backgrounds.
One slight improvement is that this year, more papers are quoting “86 applications for every graduate job” rather than “86 applicants for every graduate job”. However, this still creates a doom laden view of graduates having a tiny chance of getting a graduate job. I have not to date seen ANY newspaper report that explains that graduates apply for more than one job. Since from our questioning of recent graduates, they applied for an average of 20 jobs, this suggests that the 86 applications is in reality 4 applicants per vacancy, pounding on a lot of doors.
Having listened to the AGR survey figures being reported for many years, it was interesting to note that for 3 years the AGR summer survey launch commented on a rise in the number of applications. This led to press headlines trumpeting 46 jobs per graduate vacancy, 67 jobs per vacancy etc.. Guess what – applications leapt up by about 20 the following year, as frightened graduates redoubled their efforts. This happened for 3 years in a row. Then there was a year when the number of applications figure was not commented on, and didn’t appear in most press reports. The next year, applications dropped by 20. The next year the number of applications was mentioned again; the year following, applications leapt up again.
Final year students read articles on graduate employment. The scarier the figures, the more applications they will send.
If you measure and report a high number of applications per vacancy, the consequence is that next year there will be even more applications to plough through.
It’s obviously a key priority to help graduate employers to shortlist high quantities of applications without introducing social mobility bias. Is it possible that the best thing the AGR can do to reduce the burden of shortlisting, is to stop quoting publicly how many applications there are per job? It would make it harder for the press to misreport the figures as the headline to scare stories – which in turn seems to create a massive and unnecessary shortlisting headache for graduate employers!
Shouldn’t we do everything we can to go back to the days when final year students put in a few high quality applications, for the jobs they really want?
Take Away
As a serious point, we are interested in having a brainstorm on how shortlisting can be done without negatively impacting social mobility. Anyone who has any useful ideas, or would like to be involved in this discussion – do please contact us.