How should you act on comments in employee surveys?
Situation
The Internet has made it easier and quicker than ever before to collect and participate in user feedback.
Set in the context of employee engagement, this has created a large increase in the number of employee surveys being sent out, not just by large corporates that had traditionally run annual questionnaires already, but across smaller businesses too. These are often in the form of ‘pulse’ surveys, which are shorter in length but more frequently received, usually addressing a particular ‘hot topic’.
Alongside collecting scoring questions on a 0-10 scale, most survey software now enables open-ended ‘comments’ boxes to be optionally added, giving the participant an opportunity to add context and specific examples to evidence the reason for the score they gave.
Complication
Whilst this is a great learning opportunity for businesses to learn from employees, it does bring up two particular problems for HR and other organisational departments seeking to use and respond to the insights:
Volume of data– analysing comments is slower and far less easy to process than numerical answers that can be graphed and compared in real time. An individual survey of a few hundred people can produce thousands of comments. Failing to analyse the comments can make the organisation appear disinterested, whilst even a slow response can be perceived as showing lack of senior leader commitment.
Negative feedback is usually more demanding of action– the sentiment in comments can be positive and negative (sometimes both in the same comment, which makes analysis harder!) But our experience shows that negative comments tend to involve employees pointing out problems or negative change with a hope and/or expectation of it being fixed.
Recommendations
We have developed a number of different processes for handling and responding to comments in employee surveys, which vary according to the size and nature of each organisation.
Nevertheless, there are some overarching principles and techniques which we believe work for all:
Get your comments organised and cleaned properly first – there are two key steps to this, which will save you a lot of heartache later:
a. Use employee engagement software to capture comments– there are now at least 20 software vendors offering good-quality, well-designed survey tools, with pricing models that make the software accessible to all sizes of business, even down to micro-businesses. Having such software underpinning your surveys enables you to capture and categorise comments very quickly, ensuring that any manual analysis time you spend is focused on where tech is least able to help. That said, now read the next point!
b. Manually check the accuracy of automated sentiment analysis – whilst most survey software offers some form of sentiment categorising (positive, negative, neutral), some are a lot more accurate than others! Always take a sample (we suggest 10 per cent) of comments that have been sentiment-labelled by machine and sense check it yourself. Look for things like sarcasm and irony, as well as how well the software copes with nested comments (a second comment bundled into the first one). As a rule of thumb, trial the software first, and if you find the accuracy is below 70%, it is not worth using at all – go trial another vendor (we can help with recommendations here).
2. Sort comments into themes by frequency and urgency– if an issue is coming up a lot, that’s clearly worth addressing. If it’s a frequently raised positive theme, it can be used to celebrate success, and to scale up and across the organisation. However, if it’s negative, it needs to be context-assessed and then addressed. Context addressing can include things like “Is this company-wide, or only in certain departments/offices/job roles?” We usually advise conducting listening and engagement activities such as focus groups to explore deeper into the issue. Other less commonly raised issues might still need urgent and priority attention. Examples might include accounts of bullying in the workplace, or policy breaches such as data protection or safety matters. We typically work with clients to develop a set of hot-spot keywords to monitor, both in advance of the first survey, and then added to during later ones.
3. Match your survey frequency to the speed of your response to issues – there are few things we have come across in employee engagement surveys that are worse than participants feeling they have to raise the same issue time and again. For example, if you ask a question about flexible hours, and get lots of comments about it, make sure you have communicated to your employees what you have heard, and what you’re doing about it before you poll them on the topic again. This does not mean you have to solve every issue raised before running another survey, but we highly recommend demonstrating that you have recognised the issue, and are taking steps or developing a plan to explore it further and address it as appropriate.
4. Look for ways to link and/or measure comments to metrics – for an organisation to take employee engagement comments seriously, it needs to be able to track not just what is raised but the success of any actions or initiatives taken to respond to them. For example, if 'learning & development' was an issue that employees felt was holding back their career, and the business responded by investing in skills programmes, it would be important to be able to assess the impact. In such an example, frequency and sentiment of 'learning & development' comments may be used to gauge success, as well as harder metrics such as retention rates of those completing training vs. those who have not. Such tracking can be used not only to test the success of initiatives for scaling up/down, but also for communication too, both internal and external. Comments can be very insightful not just for understanding what matters to employees, but how they feel about such issues too. By utilising some of the latest technology in emotions text analytics, organisations can identify and achieve tangible improvements in employee well-being, including factors contributing to mental health, happiness and motivation.
In summary, we expect employee engagement to continue to grow as a key factor both in how people choose where to work, and how organisations are able to maintain a happy and productive workforce.
Engagement survey comments are no longer the ‘ugly duckling’ of survey data – seen as hard to interpret and difficult to apply. Instead, the development of technology is turning employee comments insight into a ‘beautiful swan’ - even to the point of becoming a source of competitive advantage.
About Customer Faithful
Customer Faithful is a research-led consultancy, specialising in the patient experience, customer experience and employee engagement. Our mission is to help organisations to put what their customers and employees value most at the heart of the business.
Rick Harris - Founder of Customer Faithful - will be speaking on this topic at the Hauser Forum in Cambridge in January 2019. For more information click here.