When we design a customer experience for our clients, we typically create 3 core deliverables, which form a top-down hierarchy of detail:

1. Experience Pillars – these high-level principles underpin what you want the customer takeaway to be. They define how you do what you do
2. Journey Steps – these describe how customers define and feel about the process or ‘journey’ of finding, choosing, buying, using, living with your brand
3. Touchpoints – these detail the specific places and issues that really matter to customers – you need to excel at these points.

These deliverables form the guidelines for both designing new experiences and improving existing ones. Ultimately, though, it comes down to implementing activities and initiatives, which actually make the experience happen on the ground.

In their eagerness to implement, many companies rush out ‘quick wins’ – things that seem no-brainers, benefiting both the customer and organisation, fast to roll-out and cheap to do. Sometimes firms have a formal scorecard for deciding which initiatives to tackle, adding some ‘business case rigour’ to the customer experience.

Yet, amazingly, we see hardly any firms use one of the simplest and most effective techniques for improving a proposition – it’s been around for decades in the direct mail industry, and it’s called AB split testing.

A typical direct mail example is to try to increase response rates to a catalogue. In order to appeal to the widest audience, two different catalogue covers might be created, with everything else the same. Version A is the traditional style of execution, and Version B tests a new style. The two styles are mailed to a randomly split list of customers, and the response rates compared for success. They key point here is that the cover test involves changing just one variable.

Now let’s imagine applying that same simple technique to customer experience within other industries:

Travel

The Issue - there’s heaps of research showing that business and leisure passengers want wifi internet access en route, to stay in touch and reduce their ‘downtime’.

The AB Split Test - Airport and train operators could offer internet hotspots, split-testing:

a) unlimited free access at one terminal and b) free 15 minute-only access at another

If the limited service is promoted as enabling the airport to keep the service free for as many passengers as possible, it may be that some passengers will willingly pay a premium for longer access.

Banks

The Issue – our research shows that many customers would like to be environmentally friendly by opting out of paper for electronic bills, but are wary of not having an ‘official’ copy in case of a query. They also don’t like being charged for paper bills either. Meanwhile, banks want to save money on paper, and demonstrate green credentials for their brand. Banks could split-test

a) an offer to new customers of on-line billing where any request for a paper version is available free on demand

b) offering an annual £5 account credit or charity donation if customers choose on-line billing

It may be that both test alternatives are viable for the bank, but the AB split test can provide valuable insight into whether customers prefer a tangible reward incentive, or simply the reassurance of access to a paper bill if needed.

Next Steps

The key steps to using this classic direct mail technique for customer experience improvement are:

1. Be clear about what the customer wants and doesn’t want – what’s the customer goal here?
2. Develop two potentially viable options for trial – don’t bias towards one or the other, make it a fair fight
3. Consider both the customer and business benefits of the initiative – if it’s to be successful, it may need to be rolled out at scale
4. Ensure that whatever initiatives are trialed, none compromise the company brand values, and so send mixed messages. In a world of instant social media, split tests may not stay secret for long, so be prepared to honestly and openly explain the customer benefit you’re trying to achieve by running the test
5. Even simple AB split tests can turn up some surprising results. If in doubt, run qualitative follow up research with the customers involved, to understand why they responded the way they did. There is no shame in learning the full picture, and it’s a lot wiser than bluffing later in the boardroom!

Want to get started? Contact rick@customerfaithful.com