User Feedback vs. Behavior

When product managers interview customers to better understand their unmet needs, you will hear them explain their problems. If you ask them how they currently use your software, they will also explain in vivid detail how they use it. Unfortunately, often there is a distinct difference between what humans say and humans do. Let me illustrate with a simple example. If you ask users if they like to scroll to find information on a web page, they will say No. They would tell you that they would like all of the information above the foldline (think about it as the top section of your web page that can be completely viewed without scrolling). However, in numerous usability tests I have done, it has been consistently observed that the first thing users do when they land on a web page is to start scrolling.  So make sure that you mix in observed user behavior with what users tell you to truly understand their real problems. This is what the field of ethnography is all about.

Thoughts? Please share your perspective via comments.

If you enjoyed this post, please
1. Consider leaving a comment or subscribing to the feed to receive future articles delivered to your feed reader or by subscribing by email via the text box in the column on the right.
2. Spread the word via your twitter or Facebook accounts.

Five guidelines to prioritize feature requests

As a product manager, you are very likely to have more feature requests than what you can put out in a given release. In my case, a good product manager’s job at release planning is figuring out what to eliminate from consideration – you have to make hard decisions – that is what you are paid to do.

Please note that my background is working on products that have mass market appeal and did not have to deal with things like customer funded development – where a customer throws money at you to get a feature that only they will use.

Here are the five guidelines I have used effectively ….

1) How MANY customers will ever USE it?

We held a strong line on this – if the feature was meant just for the selected few (no matter how much money these customers had), we said No to the customer. You may say – no way, it will not work in my case. My reply – have you tried – Have you told your customer No and told him why? – if not try it.

I will give you an example – we had a very well known consumer company in Japan as a customer – they had bought a few licenses but had the promise of buying a whole lot more licenses – they had a lot of money. We visited them in Japan, they visited us here, met with our executives etc. – they tried to push their agenda hard on us and they had all sort of ideas as to how the software needs to work. We questioned them hard to get to the bottom of their underlying problems – why, why, why do you need to do something. They had some great ideas and some that applied only to them – we readily agreed to do the former and rejected the latter with solid business reasons why we won’t do it – not enough customers have the need.

They respected us at the end – their feedback was that we were the first vendor to have grilled them on their requests and were bold enough to turn them down, as opposed to other vendors who were ready to do whatever they wanted. They realized that we were a company that knew what we were doing and our willingness to grill them, told them that we wanted to make sure we were solving their problems the right way.

2) How OFTEN will customers USE it?

Is this something a customer will use once a year, once a month or something they would use everyday. Why does this matter? Remember the phrase “death by thousand cuts” – if something that needs to be used everyday is not supported or is very inefficient to do, it kills user productivity. This may not be something that will make a press release when you launch a new release or in your product demo, but this is something that will get you a standing ovation from your existing customers. Believe me, I have experienced this multiple times where the smallest change scored the highest in customer satisfaction. Stuff such as choosing the right default values, remembering the last used options fall into this category.

Taking how MANY customers would use it and multiplying it by how OFTEN they will use it, you get what I define as Velocity of a new feature.

VELOCITY = How MANY customers will use X How OFTEN they will use

Features with highest velocity should be serious contenders to make into your new release.

3) Will this open up new markets?

It could be something that could be asked by your smallest customer – but it could be this brilliant idea that could change the rules of the game and open up a completely new market for you. You as a product manager need to make this call.

I have been asked in the past if we implemented feature requests only from large customers with lots of licenses – my response always has been – size does not matter, quality of the idea does. This is what product managers get paid to do – take such ideas and make a business out of it.

4) Is this considered table stakes by the market?

There could be features you would need to compete in the market – if you don’t have these basic things, you are not even considered a player. These ones should be obvious if you know your market. For example, pivot tables or charting are considered table stakes for a serious spreadsheet application.

But tread this one with caution – this is where sales can take you for a ride. They will say “we cannot compete because we don’t have this X, Y and Z” and the list could be endless. For those being raised by sales, ask them to put you in touch with customers who would not buy without this feature – then from the customers get to the real problem that is solved by the feature – for all you know, you may have an innovative way to do it, or can come up with one that might change the rules of the game.

5) Is the feature a building block to the real thing?

This could be something determined by R&D (architectural changes) or some other user facing feature that is needed to support the final feature.

If you consistently use these guidelines, you end up making a business case why you want to turn down a feature request – this even works with executives because it is now based on facts and not opinions. After all this, if some higher up overrides your decision, it is not in your hands, but you know you did your analysis.

You want to talk to customers – ask me, I was a customer once …

Have you heard this one before – I have – internal pundits claiming they know what the market wants because at one point in time (read “eons” ago, before your market segment even existed), they used to be in the customer’s shoes.

“Hey, I used to do product design”

or

“I used to be a salesman”

or

“When I used to run YYYY department”

they say ….

I have found a quick counterattack for this –

“Great, when did you do that?”

“In 1997, I used to work at XYZ Co.”

“Hummm, 11 years ago – so you mean to tell me that the world has remained stagnant and nothing has changed in those 11 years – did I hear that right? (OK, not exactly in the words above, but you get the point).

That usually stops this “we have answers here in the building” or “the way I did is how the world works”.

Yes, there are certain product decisions that you have to make drawing on your past experience, but saying that we know what to build because I was a customer once, is nothing but a recipe for failure – especially in the high tech arena, where the way you did it last month is probably not valid anymore.

It amazes me how many companies claim to be customer driven, but then they limit their product managers from traveling because the travel budget is tight, but on the other side the product development budget is a big leaky bucket funding products no one will ever want to buy.

Where do your customers get information?

When interviewing customers to determine their needs, take the time to also ask them where they get information that keep them up-to-date in their profession.

1) Are there organizations that they regard in high regard that a recommendation from such organizations is considered valued?
2) What magazines do they read frequently?
3) Do they have a blog?
4) Do they read blogs? Which ones?
5) Do they read or contribute online discussion forums?
6) Are there places which your customer absolutely disrespects or would not want to have any association with?
7) Do they read analyst reports – like Gartner, Yankee Group, IDC etc.

    You want to find out where they hang out professionally online and offline. This is because if and when you build a product/service that solves their needs, you want to make sure that the new product/service is promoted well where your potential customers hang out. So, there is more to learn in customer interviews than just the unmet needs – take it as an opportunity to profile your customer very well.

    Passion vs. decibels – How to manage vociferous customers

    In my product management career, I have always had some users who were more vociferous than most of the others. Their decibel level when they asked for new enhancements or yelled at you for the bugs in the software was orders of magnitude higher than majority of the users. The vast majority of these users were loud because they cared, they were passionate about your product. Hence, don’t discount these users – listen to them, allow them to vent and take action to fix their problems if appropriate.

    However, you as a product manager need to do a fine balancing act you – some of the feature requests from these vociferous users could be for functionality that applies just to them, because of what they use your product for or the unique way in which they do a common task. You have to listen to them, but also be willing to say No with a justifiable reason. All of us ask for more, but not everyone expects to get everything they ask for (if you don’t ask, you don’t get – so there is not much downside to asking). As long as you tell them why you would not do something, they will understand – getting back to them tells them you care.

    I remember an instance where I was reading through the enhancement requests we had received when I came across a tirade from a user. It was an email bomb filled with expletives how our product sucked and how he thought we were all a bunch of idiots. I picked up the phone and called the user. I introduced myself and told him that I was calling him about the enhancement request he had send in. First thing he said was a sincere apology for having writing such a tirade (I don’t know if he thought I was going to sue him) and said he was having a bad day and the software was not working the way he wanted to get his job done. I asked him not to be apologetic and asked him to help me understand the deficiency in the software. We spend a good 20 minutes on the phone understanding the issue.

    Guess what likely happened after the call – I created a passionate user for our product because I took time to read what he had written (however unpleasant),  call him and discuss the issue to see how we could improve our product so that he can be successful with what he does. Do you think he would have shared this experience with his colleagues or other users in his community? I absolutely think he would have. What happens if he encounters someone else who has a similar gripe – he is going to tell them we care and to talk to us. I created another big proponent who is going to spread the word about our product and about us as a company.

    Creating passionate users and listening to them is one of the most important parts of your job as a product manager. After all, the answer is not in the building – it is very important for all of us to get out of our busy schedule of internal meetings and take the time to talk to real users.  After all, they are the ones who buy our product.

    Avoid excuses for not conducting customer visits

    Customer visits have always been one of my pet subjects because the only way I have learned to deliver good products is by getting out of the building and talking to real people who buy or will buy my products. At SolidWorks, customer visits was ingrained into our working culture and all of us had an MBO of conducting six customer visits every quarter.

    Jeff Lash has written a very good article on his blog titled Avoid excuses for not conducting customer visits. It is definitely worth reading.

    Pain points vs. Requirements

    I was involved in a conversation last week where we were trying to unearth customer pain points. One of the software product managers explained that main pain points were integration and scalability. This is what set me thinking about the difference between pain points and requirements. The  two are not interchangeable – in fact there is a parent child relationship here which if you ignore, you could end up building the wrong or a less preferred product.

    Pain points are what customers are going through because they do not have a product that solves their problem. For example, before the advent of wireless devices, a customer pain point could have been that he cannot make a phone call while he was in the car – notice that nowhere in this pain point is wireless mentioned. Putting wireless into this sentence would be jumping to a solution as to how to solve the pain point. It is extremely important that product managers understand the real pain points. Here are some other examples – all cooked up for illustration.

    1) I am in sales and I get a lot of customer queries via email. I need to respond to these quickly to close the deals. This is especially true towards the end of the quarter. But given that I am travelling and in meetings at customer sites, I am unable to do this because I cannot get to my email unless I am in the office and this costs me time and money. For example, in one instance last quarter I missed closing a deal worth $1M by one day. (I guess RIM probably heard this a whole lot and came up with the Blackberry – but do you see anything about a Blackberry or a mobile device in the customer’s description of his/her pain point? – does it say anything related to integration or scalability or ease of deployment in the description?)

    2) I usually get out of the office after the banks have closed. Hence I am unable to get to the bank before they close to withdraw money from my account. Hence I have to remember to withdraw money on Saturday mornings, enough to last me one week. I feel uncomfortable carrying all that money around where ever I go.  BTW, this is still true in a lots of places in India (what do you think could solve this? I can think of two – ATMs, credit cards)

    3) I end up paying a lot of late fees because I forget to mail my payments on time or because the postal service takes more time to delivery my check to the credit card company. For example, last month I ended up paying $58 in late fees. It is hard to predict when exactly to mail the payment. I cannot afford to send the money way ahead of time because I have to wait for paychecks to get deposited (Could very well have been one of the pain points that created online banking)

    You get the idea. It is very easy to fall into the trap of starting to describe product requirements and not pain points. It is not easy to do tjos because the creative part of our brains loves to start thinking about how to solve the problem. But take control – make sure that you understand the pain points very clearly before you start listing requirements or evaluating product ideas.

    %d bloggers like this: