Win/Loss Analysis is for winners

You have probably heard of Win/Loss Analysis, but what is it, and why should you be doing it? The world’s top companies know that obsessing over the customer’s needs leads to long-term success, and Win/Loss Analysis is one technique for gaining powerful customer insights.

People working at product companies have an emotional stake in sales success (hopefully). This usually causes various cognitive biases which affect their ability to understand sales performance objectively. Win/Loss Analysis provides a dispassionate and consistent approach to capturing feedback from market interactions. It’s a structured method of gaining feedback from decision-makers, usually in the form of a short telephone interview. At the end of the day, what’s important is what the customer thinks about your product, not what you think!

Who should carry out the Win/Loss Analysis interview?

Client interviews must be conducted by someone who has not been directly connected with the sales engagement. People who have been directly involved will be disappointed, perhaps even angry about the result, or elated, and even arrogant; this will usually cause them to go into the interview with preconceptions which affect their receptiveness. They will suffer from confirmation bias, which skews one’s interpretation of the evidence to back up what one already believed.

Furthermore, the sales person will have built up a relationship with the client. This will affect the client’s ability to give frank feedback. The client may even wish to comment adversely on the sales person’s performance!

Product Managers can often be the go-to person for running the interview and analysing the results. Better still, get someone with no direct stake in the success of the product sale to do it. This can be somebody from a customer services department, or an outsourced research agency.

When should the Win/Loss Analysis interview be done?

It’s important to conduct the interview very soon after the decision has been made, for several reasons:

• The period immediately after the client has made a decision is a crucial opportunity to obtain their frankly expressed views. Their reasons for making their decision will be front of mind.
• If you leave it more than a week or so before conducting the interview, you may lose this ‘freshness’.
• The client may also feel the need to justify the decision, but this will fade quickly.
• When someone has just decided not to buy your product, they often feel a moral obligation to provide constructive feedback. However, they will soon be heavily involved in implementing the successful bidder’s product; the relationship with you will quickly become unnecessary to them.

What’s the format?

The Win/Loss Analysis interview is normally a telephone call lasting between 15 and 30 minutes. It’s best to get the client to agree to do the call before the buying decision has been made. In the event of a loss, you can sometimes find the lines of communication going dead very quickly.

During the call, stay friendly and polite at all times, and certainly don’t argue with the client! Remember, this isn’t a debate, it’s a survey, and you want their honest views. When you’re finished, thank them for their time and tell them what you’re going to do with the information. It never hurts to leave a good impression, and you never know how your behaviour may affect future buying decisions.

Typically the questions you’ll ask will be around the following topics:

• What led to your considering our product in the first place? How did you find us? Was it a recommendation?
• What does decision-making in your organisation look like?
• How did you find the sales process? (This can lead to some frank feedback about sales people as well as process).
• What did you like about the product when compared to the alternative(s)?
• What weaknesses did the product have compared to the alternative(s)?
• Would you consider choosing us in the future?
• Would you recommend us to others (i.e. the Net Promoter Score question)?
• Any other comments?

Some of these questions can be scored (quantitative) rather than purely narrative (qualitative). If you do this, take care to offer a scoring scale that doesn’t have a ‘neutral’ option. For example, offer four options (very negative, slightly negative, slightly positive, very positive). This forces the respondent to offer an opinion rather than ducking the question. You can include questions pursuing specific lines of inquiry or to test hypotheses; take care not to ask leading questions which could cause a biased response.

Don’t expect instant insights!

As with any data analysis, the broader your sample, the more robust the insights that can be gleaned. Your first Win/Loss Analysis interview may yield surprising responses, or confirm what everyone already thought. Either way, you should avoid jumping to any conclusions. Wait until you’ve seen similar responses from a number of clients before attributing any significance.

Finally, it’s tempting to alter the questionnaire in response to what you learn from early responses. However, you should avoid continually tweaking the questions; you will need to gather a reasonable body of responses which can be compared in order to discover meaningful trends. You must also try to ensure that interviews are conducted consistently and in as scientific a manner as possible. By persevering with a strategically-driven approach, you will come to see that gaining direct customer feedback in this way pays back many times over.