511: Letter of Recommendation

Have you ever written a letter of recommendation? Do you think they are helpful?

Full episode script

In February of 2014, the blog NeoAcademic wrote:

It is certainly intuitive that someone writing about someone else whom they know well should be able to provide an honest and objective assessment of that person’s capabilities.  But despite their ubiquity, little research is available on the actual validity of recommendation letters in predicting academic and job performance. They look like they predict performance; but do they really?

There’s some very strong data — both quantitative and qualitative – on both sides of this question.

Aimee Batemen wrote on LinkedIn, quote:

Back when I was a recruitment consultant, if I had a candidate’s profile, I’d skip straight past the experience and go straight to the recommendations first. The competition for jobs was so competitive, I needed to look for that ‘edge’. The same rings true today.

There’s a lot of data that, from a search side and purchasing side at least, what other people have to say has a big impact on things. As far back as 2013, 88% of people who responded to an online survey read online reviews to help make their decision about a local business. 39% did so on a regular basis.

That would indicate that yes, reviews and recommendations do have a big impact. Another study that looked at the impact of language in reviews academics write for one another — and found that, in general, women tended to be described with language that was not as strong or complimentary as men, even when the letters themselves were written by women. The authors of that 2016 study argued that this may have some correlation to the fact that women tend to face more difficulty attaining higher academic positions.

That would be another notch in the idea that these letters may have a big impact. Yet returning to the NeoAcadmic post — quote —

In a recent issue of the International Journal of Selection and Assessment, Kuncel, Kochevar and Ones examine the predictive value of recommendation letters for college and graduate school admissions, both in terms of raw relationships with various outcomes of interest and incrementally beyond standardized test scores and GPA. The short answer: letters do weakly predict outcomes, but generally don’t add much beyond test scores and GPA.

This script may vary from the actual episode transcript.