BLOG POST ·
5
min read ·
18.12.2019

Measurement mindset in product management - identifying the right KPIs

Lead Business Analyst

Vijay Dhatrak

A high-performing IT professional with over 8.5 years of cross domain exposure into Business Analysis, Product and Project Management. Vijay is a passionate practitioner who views agility as a mindset, not a destination.

Most product teams understand the benefits KPIs produce, but identifying the right ones is a challenge that most struggle with. This article reflects upon key guidelines and tips helpful to create acceptable KPIs and a success metric dashboard.

A product manager’s role can essentially be summarised as following:

  • Respecting customers and their voices - by knowing who the customers are and what problems do they have
  • Helping customers being awesome  - by giving them a product that meaningfully solves their problems
  • Figuring out what game a company is playing and how to always be ahead - the Strategy, Competition & Commercial viability.

Managing a product is not an elementary task though. It spans over numerous sub-processes, challenges, and dynamic forces, that involve thorough customer research, iterative product development, implementing a scalable architecture, in a constantly changing market with shifting customer expectations, varied stakeholder opinions and interests all within a significant time horizon.

Altogether these factors have tremendous potential to divert the product from its strategic long term goals or make it fairly difficult to stay on the right path.

In this context, KPIs or success metrics are handy tools for product teams. They act as a guiding light to help them stay on the right track in regards to their vision and strategy.

Key Considerations for identifying KPIs

As a rule of thumb, there are two considerations for  identifying KPIs:

  1. Using a holistic view
  2. Keeping it lean

1. Having a holistic view

This fundamentally means measuring KPIs at all levels and in all areas.

Measure at all levels

As much as your product strategy directs the KPIs of your product, releases, and richness of features, your product will also be impacted by sprint goals, epic success metrics, and stories feedback. A constructive measurement framework is fundamental to provide direction at all levels. 

Strong customer feedback on the release of an epic may result in significant changes in your roadmap features which in turn might require tweaks in future releases and the overall strategy. Simultaneously, your strategy directs which features go on your roadmap and the richness of features on your roadmap guide your backlog items. Hence, why it is crucial to measure KPIs at all levels from strategy to epics

They are all cyclic in nature and demand adjustments at all levels using the data collected at any point in the process


Measure all areas

Measuring every aspect of your product development process is essential. Measurement insufficiency in any area has a cascading impact on the success of the product. 

For example, your previous release might have increased the number of active users of your product and supported your revenue targets. But limitations on a healthy release pipeline, an ineffective regression suite, a lack of management sponsorship or weak team motivation will make it impossible to keep the momentum.

Some KPI examples of a digital product covering all areas


2. Keeping it lean by avoiding vanity metrics

A measurement framework can be costly. The more complex it is,  the more time and energy your team will spend discussing and making decisions on it. Hence why it is absolutely mandatory to keep it lean, specific and avoiding vanity metrics.

I was once working on a retail product for a company that was originally selling its goods on major e-commerce platforms (e.g. Amazon), which meant that a substantial fee was paid to such platforms on every item sold. Although it was enticing to measure customer acquisition on the new retail product we were building, this KPI alone wasn’t a credible measure of how we were performing according to our strategy. Instead, measuring the conversion of customers from other e-commerce platforms to the new retail product was of higher value. It accurately translated the reduction in fees paid to other e-commerce platforms and reduced the customer acquisition costs burden.


Other guidelines for maintaining the right metrics 

In addition to the key considerations, some additional guidelines that are helpful for choosing and maintaining the right success metrics:

  • Business goals direct success metrics
    Use business goals to choose the most authentic KPIs, as those business goals provide the definition of success
  • Tie metrics to PLC (product life cycle)
    The product lifecycle is an excellent tool for managing strategies of a product in each phase. It is staggeringly beneficial to use KPIs for managing decisions and objectives in various phases of the PLC
  • Know your audience
    It is extremely important to be aware of your audience and the appropriate manner of sourcing feedback. Especially, customer and stakeholder feedback
  • Right measurement frequency
    It isn't fruitful to measure all KPIs at the same frequency. While commercial KPIs indicate trends over medium to long term horizon, user activity KPIs should have a much lower frequency. Choose the right frequency to capture and discuss your KPIs based on your product maturity and the metric itself
  • Utilize all avenues
    Especially for qualitative KPIs. Customer service, Marketing, and Sales teams are incomparable channels to hear the customer’s voice
  • Use qualitative and quantitative KPIs
    Qualitative data through surveys or feedback captured via integrated tools like Optimizely speak volumes about user delights and frustrations. Quantitative data helps with efficient decision making. Balance them off appropriately at regular intervals.

Benefits of a measurement mindset

The benefits of a measurement mindset are manifold. They not only guide you through the product path but also help you constantly adapt the path as conditions change so that you are always heading north.

  1. Avoid cognitive bias
    It is easy to take pride in your work and let emotions steer your actions. Sometimes common trends in the market which might not necessarily be a fit for your product form a bias within you. KPIs help remove that cognitive bias.
  2. Ignore hunches and feelings
    You will find several people in your project who will make decisions on 'feelings', 'gut' or 'beliefs'. Use data to lead them on the right path.
  3. Suppress 'Hippo's'
    Using data points is a guaranteed strategy to base your rationale on and suppress the Highest Paid Person's Opinion
  4. Turn failure into opportunity
    Using data provides you with an opportunity to fail early and learn early
  5. Avoid getting lost into technicalities
    Supporting your discussions with data points helps you bring back the focus on strategy.

At Fabric we believe that using data, information and empirical evidence is essential to drive our decisions and approach. KPIs and a robust measurement framework that are unique to its product’s needs are part of our core product management and development practices. 

Related CONTENT

Are you tracking the right KPIs?

We'd love to discuss your measurement strategy, fill out this form and we will be in touch.

Thanks for your message,
we will be in touch soon!
Oops! Something went wrong. Please try again later.