Category: customer interviews

User Feedback vs. Behavior

When product managers interview customers to better understand their unmet needs, you will hear them explain their problems. If you ask them how they currently use your software, they will also explain in vivid detail how they use it. Unfortunately, often there is a distinct difference between what humans say and humans do. Let me illustrate with a simple example. If you ask users if they like to scroll to find information on a web page, they will say No. They would tell you that they would like all of the information above the foldline (think about it as the top section of your web page that can be completely viewed without scrolling). However, in numerous usability tests I have done, it has been consistently observed that the first thing users do when they land on a web page is to start scrolling.  So make sure that you mix in observed user behavior with what users tell you to truly understand their real problems. This is what the field of ethnography is all about.

Thoughts? Please share your perspective via comments.

If you enjoyed this post, please
1. Consider leaving a comment or subscribing to the feed to receive future articles delivered to your feed reader or by subscribing by email via the text box in the column on the right.
2. Spread the word via your twitter or Facebook accounts.

Five guidelines to prioritize feature requests

As a product manager, you are very likely to have more feature requests than what you can put out in a given release. In my case, a good product manager’s job at release planning is figuring out what to eliminate from consideration – you have to make hard decisions – that is what you are paid to do.

Please note that my background is working on products that have mass market appeal and did not have to deal with things like customer funded development – where a customer throws money at you to get a feature that only they will use.

Here are the five guidelines I have used effectively ….

1) How MANY customers will ever USE it?

We held a strong line on this – if the feature was meant just for the selected few (no matter how much money these customers had), we said No to the customer. You may say – no way, it will not work in my case. My reply – have you tried – Have you told your customer No and told him why? – if not try it.

I will give you an example – we had a very well known consumer company in Japan as a customer – they had bought a few licenses but had the promise of buying a whole lot more licenses – they had a lot of money. We visited them in Japan, they visited us here, met with our executives etc. – they tried to push their agenda hard on us and they had all sort of ideas as to how the software needs to work. We questioned them hard to get to the bottom of their underlying problems – why, why, why do you need to do something. They had some great ideas and some that applied only to them – we readily agreed to do the former and rejected the latter with solid business reasons why we won’t do it – not enough customers have the need.

They respected us at the end – their feedback was that we were the first vendor to have grilled them on their requests and were bold enough to turn them down, as opposed to other vendors who were ready to do whatever they wanted. They realized that we were a company that knew what we were doing and our willingness to grill them, told them that we wanted to make sure we were solving their problems the right way.

2) How OFTEN will customers USE it?

Is this something a customer will use once a year, once a month or something they would use everyday. Why does this matter? Remember the phrase “death by thousand cuts” – if something that needs to be used everyday is not supported or is very inefficient to do, it kills user productivity. This may not be something that will make a press release when you launch a new release or in your product demo, but this is something that will get you a standing ovation from your existing customers. Believe me, I have experienced this multiple times where the smallest change scored the highest in customer satisfaction. Stuff such as choosing the right default values, remembering the last used options fall into this category.

Taking how MANY customers would use it and multiplying it by how OFTEN they will use it, you get what I define as Velocity of a new feature.

VELOCITY = How MANY customers will use X How OFTEN they will use

Features with highest velocity should be serious contenders to make into your new release.

3) Will this open up new markets?

It could be something that could be asked by your smallest customer – but it could be this brilliant idea that could change the rules of the game and open up a completely new market for you. You as a product manager need to make this call.

I have been asked in the past if we implemented feature requests only from large customers with lots of licenses – my response always has been – size does not matter, quality of the idea does. This is what product managers get paid to do – take such ideas and make a business out of it.

4) Is this considered table stakes by the market?

There could be features you would need to compete in the market – if you don’t have these basic things, you are not even considered a player. These ones should be obvious if you know your market. For example, pivot tables or charting are considered table stakes for a serious spreadsheet application.

But tread this one with caution – this is where sales can take you for a ride. They will say “we cannot compete because we don’t have this X, Y and Z” and the list could be endless. For those being raised by sales, ask them to put you in touch with customers who would not buy without this feature – then from the customers get to the real problem that is solved by the feature – for all you know, you may have an innovative way to do it, or can come up with one that might change the rules of the game.

5) Is the feature a building block to the real thing?

This could be something determined by R&D (architectural changes) or some other user facing feature that is needed to support the final feature.

If you consistently use these guidelines, you end up making a business case why you want to turn down a feature request – this even works with executives because it is now based on facts and not opinions. After all this, if some higher up overrides your decision, it is not in your hands, but you know you did your analysis.

You want to talk to customers – ask me, I was a customer once …

Have you heard this one before – I have – internal pundits claiming they know what the market wants because at one point in time (read “eons” ago, before your market segment even existed), they used to be in the customer’s shoes.

“Hey, I used to do product design”

or

“I used to be a salesman”

or

“When I used to run YYYY department”

they say ….

I have found a quick counterattack for this –

“Great, when did you do that?”

“In 1997, I used to work at XYZ Co.”

“Hummm, 11 years ago – so you mean to tell me that the world has remained stagnant and nothing has changed in those 11 years – did I hear that right? (OK, not exactly in the words above, but you get the point).

That usually stops this “we have answers here in the building” or “the way I did is how the world works”.

Yes, there are certain product decisions that you have to make drawing on your past experience, but saying that we know what to build because I was a customer once, is nothing but a recipe for failure – especially in the high tech arena, where the way you did it last month is probably not valid anymore.

It amazes me how many companies claim to be customer driven, but then they limit their product managers from traveling because the travel budget is tight, but on the other side the product development budget is a big leaky bucket funding products no one will ever want to buy.

Where do your customers get information?

When interviewing customers to determine their needs, take the time to also ask them where they get information that keep them up-to-date in their profession.

1) Are there organizations that they regard in high regard that a recommendation from such organizations is considered valued?
2) What magazines do they read frequently?
3) Do they have a blog?
4) Do they read blogs? Which ones?
5) Do they read or contribute online discussion forums?
6) Are there places which your customer absolutely disrespects or would not want to have any association with?
7) Do they read analyst reports – like Gartner, Yankee Group, IDC etc.

    You want to find out where they hang out professionally online and offline. This is because if and when you build a product/service that solves their needs, you want to make sure that the new product/service is promoted well where your potential customers hang out. So, there is more to learn in customer interviews than just the unmet needs – take it as an opportunity to profile your customer very well.

    Passion vs. decibels – How to manage vociferous customers

    In my product management career, I have always had some users who were more vociferous than most of the others. Their decibel level when they asked for new enhancements or yelled at you for the bugs in the software was orders of magnitude higher than majority of the users. The vast majority of these users were loud because they cared, they were passionate about your product. Hence, don’t discount these users – listen to them, allow them to vent and take action to fix their problems if appropriate.

    However, you as a product manager need to do a fine balancing act you – some of the feature requests from these vociferous users could be for functionality that applies just to them, because of what they use your product for or the unique way in which they do a common task. You have to listen to them, but also be willing to say No with a justifiable reason. All of us ask for more, but not everyone expects to get everything they ask for (if you don’t ask, you don’t get – so there is not much downside to asking). As long as you tell them why you would not do something, they will understand – getting back to them tells them you care.

    I remember an instance where I was reading through the enhancement requests we had received when I came across a tirade from a user. It was an email bomb filled with expletives how our product sucked and how he thought we were all a bunch of idiots. I picked up the phone and called the user. I introduced myself and told him that I was calling him about the enhancement request he had send in. First thing he said was a sincere apology for having writing such a tirade (I don’t know if he thought I was going to sue him) and said he was having a bad day and the software was not working the way he wanted to get his job done. I asked him not to be apologetic and asked him to help me understand the deficiency in the software. We spend a good 20 minutes on the phone understanding the issue.

    Guess what likely happened after the call – I created a passionate user for our product because I took time to read what he had written (however unpleasant),  call him and discuss the issue to see how we could improve our product so that he can be successful with what he does. Do you think he would have shared this experience with his colleagues or other users in his community? I absolutely think he would have. What happens if he encounters someone else who has a similar gripe – he is going to tell them we care and to talk to us. I created another big proponent who is going to spread the word about our product and about us as a company.

    Creating passionate users and listening to them is one of the most important parts of your job as a product manager. After all, the answer is not in the building – it is very important for all of us to get out of our busy schedule of internal meetings and take the time to talk to real users.  After all, they are the ones who buy our product.

    Avoid excuses for not conducting customer visits

    Customer visits have always been one of my pet subjects because the only way I have learned to deliver good products is by getting out of the building and talking to real people who buy or will buy my products. At SolidWorks, customer visits was ingrained into our working culture and all of us had an MBO of conducting six customer visits every quarter.

    Jeff Lash has written a very good article on his blog titled Avoid excuses for not conducting customer visits. It is definitely worth reading.

    Pain points vs. Requirements

    I was involved in a conversation last week where we were trying to unearth customer pain points. One of the software product managers explained that main pain points were integration and scalability. This is what set me thinking about the difference between pain points and requirements. The  two are not interchangeable – in fact there is a parent child relationship here which if you ignore, you could end up building the wrong or a less preferred product.

    Pain points are what customers are going through because they do not have a product that solves their problem. For example, before the advent of wireless devices, a customer pain point could have been that he cannot make a phone call while he was in the car – notice that nowhere in this pain point is wireless mentioned. Putting wireless into this sentence would be jumping to a solution as to how to solve the pain point. It is extremely important that product managers understand the real pain points. Here are some other examples – all cooked up for illustration.

    1) I am in sales and I get a lot of customer queries via email. I need to respond to these quickly to close the deals. This is especially true towards the end of the quarter. But given that I am travelling and in meetings at customer sites, I am unable to do this because I cannot get to my email unless I am in the office and this costs me time and money. For example, in one instance last quarter I missed closing a deal worth $1M by one day. (I guess RIM probably heard this a whole lot and came up with the Blackberry – but do you see anything about a Blackberry or a mobile device in the customer’s description of his/her pain point? – does it say anything related to integration or scalability or ease of deployment in the description?)

    2) I usually get out of the office after the banks have closed. Hence I am unable to get to the bank before they close to withdraw money from my account. Hence I have to remember to withdraw money on Saturday mornings, enough to last me one week. I feel uncomfortable carrying all that money around where ever I go.  BTW, this is still true in a lots of places in India (what do you think could solve this? I can think of two – ATMs, credit cards)

    3) I end up paying a lot of late fees because I forget to mail my payments on time or because the postal service takes more time to delivery my check to the credit card company. For example, last month I ended up paying $58 in late fees. It is hard to predict when exactly to mail the payment. I cannot afford to send the money way ahead of time because I have to wait for paychecks to get deposited (Could very well have been one of the pain points that created online banking)

    You get the idea. It is very easy to fall into the trap of starting to describe product requirements and not pain points. It is not easy to do tjos because the creative part of our brains loves to start thinking about how to solve the problem. But take control – make sure that you understand the pain points very clearly before you start listing requirements or evaluating product ideas.

    Voice of the customer – Tip#7 – Tackling the language barrier

    Language BarrierUnless your product is used only in the US, you as a product manager should make sure that you are listening to the “global” voice of the customer. Customers in other countries typically have vastly different needs than customers in the US. Localization of your product is something you should account for in the very first release if you know that your product will be sold in non-English speaking countries. Even if your first release product launch plan is limited to the US market, you should make sure that R&D accounts for localization such that when you do decide to sell in the non- English speaking world, you will not have to rearchitect your product. Localization is not cheap, nor is it easy to implement as an afterthought.

    You as a product manager should also plan on visiting these customers to understand their requirements before you build the product. Even users in the English speaking world have different needs – assuming that customers in the UK have the same needs as those in the US would be making an assumption to your own detriment. One way to figure out who or how often to visit non-US customers is to look at the percentage of established customer base (or potential market for new products) in the major geographic market segments. Thus, if 45% of your revenue comes from the North American market, 30% from Europe, 10% from South America, 10% from Japan then you should plan on visiting customers (prospects) in these countries using the same ratio. You should then use market segmentation based on verticals to figure out which specific customers to visit.

    In my experience, on-site customer visits in countries outside of US/Canada are best arranged through your local rep. This will help you overcome the language barrier and also driving in these countries. This will also help you establish a better rapport and expectations with the aid of the local rep who is more in tune with the local culture.

    In certain countries like Japan, it is difficult (if not impossible) to have high level discussions on the customer’s goals and tasks. Often, you meet with folks in the trenches who use your product everyday as opposed to management folks and hence your discussions usually revolve around a laundry list of specific enhancement requests. You should try nevertheless to set expectations and send them a discussion guide long before the meeting so that they know how you would like to structure the meeting. Here is where the local reps can really help overcome the language barrier. After all, whatever you want to send to the customer will need to be translated before it can be send.

    Listening to the “global” voice of the customer is not easy and not cheap. But it is something you cannot afford to neglect if your product is sold internationally. With some careful planning, it can be done. If you are going to do this for the first time, make sure you budget enough time, money and resources for couple of “learning trips” before you can fine tune to get the most out of these international visits.

    Voice of the customer Tip #6 – Don’t listen to the same voice

    You have decided to get out of your office and embark on the journey of discovering unmet needs of customers by talking to customers and listening to them express their unmet needs. One of the pitfalls to avoid is talking to the same customers – customers that you know very well, those that that love your product – and then claiming that you have gone through the motions of talking to customers. If you do this, you will end up making a product that meets the needs of the few.

    You have to keep rotating the customers you talk to. You cannot be talking to the same customers over and over again. Find customers you have never visited, new customers who have just bought your product, or customers who have been using your product for a long time. You should be prepared to listen to customer who may not have all good things to say about your product. In fact bad news is the best good news you could get from a customer visit, because they are actionable pieces of information you could take back and get fixed. But, bear in mind that just because one or two customers told you that they liked something or absolutely hate something in your product, it does not mean that the whole world shares that opinion.

    What has worked me in the past is making customer visits something I do all the time throughout the year. I try to be out of the building one day a month visiting two customers during that day. Make it part of your working culture. If you are going somewhere attending a conference or on a pre-sales call or to visit some particular customer, try to find other customers in the area that you have never visited and extend your stay by a day and visit them. This lets you justify the travel expense you would have incurred anyways.

    Good sources for names of customers you could possibly visit include:

    1. Your tech support
    2. Enhancements database (if you have one)
    3. Your sales organization
    4. Existing customers (they always know someone else that you could visit in their area)
    5. Prospects database

    Figure out your objectives of doing the customer visits (what are you trying to accomplish) and then try to figure out who you need to talk to. Then make sure more than 50% of those you would talk to would be first time customers who you never talked and listened to.

    How true is this in your company?

    I have found this picture to be very hilarious and after having talked to different people working in different companies, I am led to believe this is very true in a lot of companies. (I give the credit to the original creator of this picture whose name is unknown to me)

    image0011.jpg

    To avoid the above situation, in my opinion a product manager has to do three fundamental things:

    1) Thoroughly understand the customer problem, rather than taking what the customer tells you at face value. Need to do a deep dive with the customer using the concept of Five why’s. While doing this, you need to make sure you need to involve your engineers/qa etc. or take the time to educate them about the customer problem that needs to be solved.

    2) Engage the customer throughout the product development process to ensure that you are building what he is really looking for. My mantra is that it is never too early to show anything to your customers. Sign NDA’s if you have to, but engage them early. Get them to review specs, let them play with early code. The whole idea is to know if you are building the right thing, that you are rowing the boat in the right direction. The boat could be leaking water at this time because it is not finished, but you want to make sure that you are building the right boat and rowing it in the right direction.

    3) Educate other departments in your company about the customer problem and why you are creating this product/service so that they can align their tasks with what you are trying to do.

    Voice of the Customer Tip #4 – Practice active listening

    Customers are wishing someone would listen to their needs and concerns, as opposed to talking to them. You sales people, your marketing people, your competitors and everyone else is talking to them about the products they should buy. But companies that are successful listen and observe their customers. So before you call or visit the next customer, commit to listening. I sometimes write “Don’t talk” as a reminder on my note pad so that I constantly remind myself to shut up and listen. After all, we have two ears and one tongue, but unfortunately the latter gets used a whole lot.

    There are couple of active listening techniques I use all the time:

    1) Paraphrasing – After the customer has told you something, repeat back to the customer what he just told you in your own words. This serves two purposes – tells the customer you were listening and also makes sure that you did get the essence of what the customer told you

    2) Followup questions – Ask follow up questions based on what the customer told you. This again reinforces to the customer that you indeed were listening and interested in knowing more about what he just said.

    Remember that if you indeed listen to the customer, you will also be building trust and rapport with the customer. Not many people do this and hence you will be the one the customer remembers. After all, it is all about people relationships and not about products. Happy listening !!

    Voice of the Customer – Tip #3 – Five Why’s

    It is very typical while talking to customers that they ask you for a very specific enhancement. As a product manager, you should make sure you do not fall into this trap of accepting that the solution proposed by the customer is what they want. Customers are good at what they do, but cannot be considered as the best people to design your product. As a product manager, it is your job to do a deep dive and find the real pain point for which the customer has proposed the said solution.

    This is where the concept of Five why’s come in. I believe this concept has its roots in the six sigma philosophy (though I am not sure about this). Nevertheless, the trick is to ask the customer why he is proposing a solution. Keep asking why’s until you really arrive at the root of the problem. It is not necessary that you may have to ask five why’s or that you have to stop at five, but the idea is to make sure that you keep probing until you fully understand the customer’s real problem that needs to be solved. Some simple examples on 5 why’s can be found on the isixsigma site.

    Before you do this, I have found that it is important that you let the customer at the start of your interview that you would be asking some basic questions (or the why’s) during the interview. Let them know that this is only to make sure that you fully understand their problem. The last thing you want the customer to think is that you are questioning his/her judgement or that you getting defensive based on their input. So I always make it a point to tell this to the customer right at the outset (lay the ground rules first to make sure everyone is on the same page).

    Once you truly understand the problem, you can engage the smart people in your development team to come up with the best possible solution to solve the problem. Be ready to be surprised that in some cases, the solution proposed by the customer has nothing to do with the real problem. In fact, in some cases it may even make the customer’s problem worse. You are the expert in your product and it is up to you to figure out how to solve the customer’s problems in the best way.

    Voice of the Customer Tip#2 – Role of Explorer

    As a product manager, you could be asked to visit customers to help close a sales deal or to trouble shoot a customer problem along with a technical support person. None of these visits can be considered as part of your effort to listen to the voice of the customer. This is because in either of these cases, you role is to overcome the customer’s objections to close the deal or to find workarounds to the problem faced by the customer. In both of these cases, you are not listening to the unmet need of the customer, you are trying to sell or get your product to work. You are the one who is doing the talking.

    When you visit customers to listen to their voice, you should be listening and not talking. Human beings cannot do both of this at the same time (sales people may think otherwise !). When you are visiting customers to listen to them, your role is that of an explorer. You have to keep your eyes and ears open. Customers do not always tell you the whole story, not because they are withholding information, but in many cases sometimes they themselves do not recognize the pain points they have. It is up to you as an explorer, to ask the right questions and to get the customer to tell you their real problems (not solutions) that if solved would create a product differentiation for your product.

    Voice of the Customer Tip #1 – Start with softball questions

    Whenever you start talking to users whether it is face-to-face or over the phone, first of all make them feel at ease. Users tend to be a) skeptical whether vendors trying to sell them something int the guise of a conversation and b) fearful of exposing their ignorance of the product you are talking to them about (you hear them say”may be I am doing something wrong” or “I am sure I am doing something it was not designed to do” and so on).

    To make users feel at ease, do the following things at the start of the conversation

    1) Tell them that you are not a sales person (ie. if you are not) and that you are not trying to sell them anything. You are trying to get their honest feedback about your products so that you can make it work better for them.

    2) Tell them that you want to hear both the good and the bad news about the product. Hence, tell the customer not to sugar coat anything. You are not here to defend anything about your product but to make sure that you get honest feedback from them about your product. However, make sure that if the customer blames your company for some issue that is not under your control, do not ratify (you are hearing only one side of the issue), but acknowledge that you have noted down the issue and you will make sure it is brought to the attention of the right people in your company who can resolve the issue to the customer’s satisfaction.

    3) Once the stage is set, start with some softball questions. Not everyone likes to talk especially about your product. So ask them to talk about what they know best – their business and their products. This prevents them from starting the discussion with a laundry list of enhancements, but at a much higher level. After all they are ONLY using their product to get better at their business. Hence this helps you to understand their business processes, how your product fits into their processes and then allows you to slowly move the conversation towards pain points, unmet needs and then gradually bringing the focus to your products.

    I have used this technique over the last several years and I have found that it works very well to build the rapport with the customer.

    %d bloggers like this: