Remember the old adage – You cannot manage what you cannot measure. As product managers, we have to measure revenues, number of licenses, performance of our products and a slew of other things.
It is not that we don’t measure, but unfortunately, we measure using the wrong metric. Let us spec
ifically look at two examples of how using percentages can be wrongly used.
1) Product performance: When new product releases come out, many companies tout improved performance and say “this new release improved performance by 50%”. This does not mean anything to the customer. Does a particular task now take 5 sec as opposed to 10 sec or is it now 50 sec instead of taking 100 sec? None of this matters if the customer expectation is to do the task in 3 sec or less. From the customer’s perspective, your product’s performance still stinks.
Instead measure your product performance against customer expectations and then find out how far you are from the expectations. If the customer expectation is measured in an absolute number – finish this task in “X” units of time then the performance has to measured in the same units. Only if the customer expectation is to reduce scrap or product deficiencies by 50%, then does % matter.
2) Resumes: I come across so many resumes where candidates claim their achievements using percentages – increased revenues by 100% or increased number of customers by 50%. Again, it does not convey much. Did the revenues go from $1 to $2 or did it go from $20 to $40M or from $200M to $400M? What is the hiring manager’s point of reference – he/she is likely looking at an absolute scale and not a relative scale. And if you are going to make a claim of 100% and the absolute increase is from 1 to 2, then you may want to leave it out unless say it was a multi-million dollar deal. Otherwise you will make a claim of 100% increase only to be exposed during the interview.
Sounds very simple right, but then you probably have seen as many instances of the above as I have.
Image: Courtesy of dvice.com
Software Product Manager by Gopal Shenoy 
Personally I’m not a big fan of the adage “you cannot manage what you cannot measure” as it encourages people to measure outputs, instead of outcomes. Instead, I prefer the Einstein-attribute quote: “not everything that counts can be counted and not everything that can be counted, counts”. Focus on impact, not activities.
Great point Subrata.
Calling a Beta program a success because you found the most number of bugs during the program does not mean anything. I had an executive who used to tout this. Customers think – why were there so many bugs in the first place, why did you not find them if there were so many of them? The only right metric here is – thank you for helping us find these bugs and thanks to you we fixed all of them.
Also pertinent to remember while setting up a metric is what I call the First Law of (Irrational) Measurement – “Beware of what you measure, for you’ll get just that”. Measure QA efficacy with “number of bugs” and watch your bugs in-box flood up with numerous bugs but not the single root-cause!