Why Net Promoter Score failed its first test in NHS

January 2, 2016 Rick Harris

Research commissioned by the Government's Care Quality Commission found the use of NPS style questions (Net Promoter Score) as an indicator of patient experience to be confusing. Indeed the report goes as far as to say in no uncertain terms: "We do not recommend the Net Promoter Score for NHS use. Interviewees reacted badly to the concept of ‘recommendation’, particularly in the mental health setting, and a number of interviewees misunderstood what the question was asking."

The research was carried out by the respected Picker Institute Europe and was seeking to identify the best single ‘overarching’ question to ask patients and service users about their experiences of healthcare. But despite revisions and re-testing, the NPS-style question of recommendation fell down again and again - specifically by asking patients whether they would be happy for their friends and family to be treated by the service they received.

Instead, what did test well was asking people to score their overall experience of care from 0 (I had a very poor experience) to 10 (I had a very good experience).

Despite all this, The NHS implemented the Friends & Family Test anyway, and it's likely here to stay.

Our View

We've been using 0-10 scales as part of our Lifelines customer experience research for many years, both in healthcare and other industries, and have found them to be consistently effective. But we'd also recommend the NHS asked patients for their expectation of delivery too. This would help identify any pre-existing concerns about different types of treatment, perhaps from previous experience, prior to the treatment currently being reviewed. But such additional insight would need qualitative understanding. For example, a patient concerned about hygiene standards in a hospital prior to having surgery might cause a low expectation score. By understanding the reason for the scoring, a significant improvement could be made to e.g. patient presentation, by reassuring them of cleanliness, and perhaps even publishing quality standards to reinforce this.

It comes down to quality researchers asking the right questions, framed in a methodology that enables patient needs and experience to be accurately interpreted. We agree with Dr Penny Woods, Chief Executive of the Picker Institute, who responded to the research:

“This work underlines the importance of using robust, tested questions for patient surveys. Asking the right questions can give us invaluable insight into people’s experiences of healthcare, but asking the wrong questions risks collecting data that is misleading or invalid.

As for the future use of NPS in healthcare, we remain skeptical. Customer Faithful uses NPS widely in its work outside of healthcare, so we are broadly advocates of the technique. But NPS is built to work for openly competitive industries where the 'customer' has a genuine choice, such as telecoms - if they don't like one provider, they can go to another. In public healthcare, the concept of choice is limited. Patients may, in theory, choose a certain service provider (GP surgery, even clinician) but in practice, they have too little information and geographical access to make a informed decision. In addition, most patients' own experience is of NOT choosing between NHS providers, and therefore the mindset is not one of competition. Without a comparison set, recommending one provider over another becomes difficult, even abstract.

Set in this context then, it's hard to see how NPS is likely to be a very useful measure, at least not without many other supporting metrics.