Sachin Rekhi

Video: Developing a Continuous Feedback Loop
Slides: Developing a Continuous Feedback Loop

While every product team I've worked with leverages customer feedback to inform product decisions in some way, most fall short of designing their customer feedback loop to maximize the benefits to the product team of gathering, recording, and synthesizing feedback. They also often treat customer feedback as a point-in-time activity as opposed to a far more helpful continuous process. I wanted to share some of the best practices and techniques I've used for developing a product's continuous feedback loop, designed specifically to maximize the benefit of the customer feedback that your organization is already hearing.

Catalog Your Sources of Feedback
The first step is cataloging the various sources of customer feedback. Relevant customer feedback can come from a gamut of sources and it's important to think about each source, the relative strengths and weaknesses of feedback from that source, and how it fits into your overall feedback system.

Let's walk through ten of the most common sources of feedback:

1. Customer Interviews - product team's typically conduct interviews with either existing or prospective customers to learn more about their needs and experiences with their product. These could be customer validation interviews, persona development interviews, usability studies, and so much more. Customer interviews provide one of the best sources of feedback due to the depth of learning you can get from getting into the minds of users. However, they are often one of the most expensive sources of feedback from the time & effort it takes to gather it. While teams often do these at the beginning of a project, it's important to invest in them in future iterations of the product as well.

2. Customer Surveys - While customer interviews allow you to get depth of understanding of your customers, customer surveys allow you to scale your feedback process to larger audiences and to achieve statistical significance in your findings. These surveys can take multiple forms as well, whether it's market research on potential new product areas, or gathering feedback from your existing customers on their experience. For example, one of my favorite surveys are periodic Net Promoter Score surveys. They serve as a strong compliment to individual customer interviews by moving from anecdote to quantitative findings.

3. Customer Support - Customer support is talking to your customers every single day, providing a wealth of knowledge on pain points, bugs, feature suggestions, and more. It's important to create a forum by which this feedback gets regularly bubbled up to the product team for review and action.

4. Customer Metrics - Your customer metric dashboards and funnel analyses provide a view into what's resonating with customers and what's not. Knowing what parts of your product are regularly used vs not helps you quickly determine areas to double-down on or areas that need work. While they can't tell you how to fix the problem, they do become an important source of where to conduct further research.

5. Feedback Forums - Many companies create feedback forums for their users to self-help, discuss feedback ideas, report bugs, and more. If you have one, it's important to ensure these are not purely community-led and their is regular involvement from the customer support / marketing / product team that is not only participating in the discussions, but summarizing the feedback for use by the product team.

6. Sales - If your product is sold through a sales team, the sales team is an incredible source of knowledge as they are talking to potential decision-makers of your product every day. It's important to find ways to bubble up what they are hearing on a regular basis to the product team. You need to be careful to realize that their feedback will be coming from decision makers and not end-users and you'll need to analyze it appropriately.

7. Employee Feedback - Most product team's try to dog-food their product when applicable within their own company and employees using the product often have great suggestions on additions. You need to be careful here though as their feedback will often reflect power user feedback as opposed to those experiencing the product for the first time with far less motivation.

8. Social Media - Depending on your product's audience, you'll inevitably find feedback on your product across social media, whether it's Facebook, Twitter, LinkedIn; or blogs and reviews published across the web; or reviews on the App Store or Google Play store. It's important to not only use these sources as a feedback mechanism and a pulse on the sentiment of your product and brand, but also as a channel to engage those customers when appropriate. The bias with these sources are that feedback is most often polarizing - it's either coming from your strongest advocates or your most dissatisfied users.

9. Customer Success - B2B SaaS companies have started developing large customer success teams responsible for driving the adoption, engagement, and continued growth in the usage of their products within existing customers. These folks are also the best source of adoption blockers from your largest customers and should have plenty to say on how to drive quicker and more wide-spread adoption of your product with customers.

10. Churn Surveys - It's becoming common for companies to send feedback surveys to customers who have canceled or stopped using their product for some time. It becomes a great source of feedback on exactly how you lost your customer. The response rates on these surveys are typically low given the lack of motivation of churned users so you'll have to wait awhile for appropriate signal. And you'll need to think through this feedback carefully, because you may ultimately decide that some of these churned customers are not your ideal customer as opposed to assuming you need to action their feedback into the roadmap.

Develop a Feedback River
One of the first things I always do when joining a product team is to setup a feedback river - an open channel for anyone who is interested to get direct access to primary feedback on the product from across various channels. This has typically taken the form of an internal company mailing list in Gmail or Outlook, but I've also seen it as a feedback channel in Slack or HipChat. I typically require all product managers, designers, and engineering leads to be on the list, but encourage anyone across R&D, marketing, sales, customer service, and more that's interested to subscribe as well. The list usually has open write access as well for any internal stakeholder to contribute.

All feedback that is gathered across the various feedback sources is then encouraged to be shared in a reasonable aggregate form on this channel. For example, let's say the product team conducted a set of customer interviews. They are encouraged to provide both links to interview recordings as well as summarized feedback on the channel. As another example, the customer support team usually has a designated person who sends a weekly customer feedback report on the channel with details of top issues that customers have been facing as well as links to reports for further details.

This setup ends up having several benefits. First, it provides product managers, designers, and engineers the ability to regularly hear feedback directly from it's source as frequently as they would like regardless of the source of origination. As an organization gets bigger, I always find it take far more effort to get out of the building and actually talk to your customers and the feedback river makes that as bone-dead simple as possible. This also helps to ascertain patterns across the various feedback channels. Second, it serves as an internal alias that anyone in the company can use to share feedback and ideas on the product. It's usually the responsibility of a designated set of folks on the product team to respond depending on whether it falls into their product area. And third, it's a great way for new team members to quickly and authentically hear the voice of the customer and understand what excites them most about the product and their pain points.

While it ends up being a high-volume channel that can get noisy at times, I find the benefits of the ease of sending feedback to the list as well as the broad transparency end up outweighing any disadvantages. And folks just filter it into a folder and decide the cadence that makes sense for them to scan it (typically daily or weekly for product team members).

Develop a Feedback System of Record
In addition to developing a high-volume feedback river, I also always setup a feedback system of record. This becomes the source of truth for consolidated and aggregated feedback across various feedback sources designed specifically for facilitating roadmap planning.

The goal here is to create a highly systematized process such that as new feedback comes in across the various input sources, it is quickly and efficiently processed into the system of record. For each unique piece of feedback received, it should have a short description, one or more feature or product categories it falls under, and names or counts of the requestors. You should then be able to easily sort this by category and frequency.

I've typically seen this captured in one of three ways:

1. Spreadsheet - Google Sheets, Microsoft Excel - The simplest approach is to create a shared Google spreadsheet with columns for the description, category, and requestors. The benefits of this approach is that it's quick & cheap to add items and most people are familiar with it. Though the drawbacks are that it creates yet another tool the team has to use in addition to their project management tool.

2. Project management tool - JIRA, Asana, Trello - Alternatively I see folks create a ticket for each unique feedback item and then add the additional details as fields or in the ticket description. The advantages here are that your team is already familiar with the tool, it avoids double bookkeeping across multiple tools, and often provides robust reporting capabilities. The drawbacks are the friction in capturing new feedback items as well as the lack of accessibility of the tool with folks outside of the R&D team.

3. Product roadmap tool - Aha.io, Wizeline - There is a fairly new category of product roadmap and management tools that specifically enable you to capture this data with more advanced functionality for analyzing such roadmap requests. The advantages are these tools are specifically designed for this task, though they introduce a new tool you need to pay for and educate people on.

The trick to making this process work is making it as light-weight as possible to get all the data from the various feedback sources into it. If it's burdensome to do so and it's not kept up-to-date, it ends up being a waste of effort. I find that having a designated person entering data for each source of feedback enables them to quickly accomplish the task given their regular familiarity with it, but also distribute the load across various stakeholders responsible for each source. While I personally prefer the spreadsheet approach, I've leveraged the others as well because it's important to adopt whatever process you can get your whole team behind.

Incorporate Feedback Into Your Product Roadmap Process
It's critical to ensure you incorporate your feedback process directly into your product roadmap process to make the effort you put into cataloging feedback worthwhile. While this sounds simple, I've seen countless teams that have roadmap planning meetings without much of any discussion about the feedback they've recently heard from customers and whether that validates or pivots their current roadmap plans.

When having discussions about what features to prioritize, you can now have a data-driven discussion with the team based on actual data inputs from your feedback system of record of who is requesting a feature, whether you're hearing it from multiple channels, and how frequently. This moves away from individual anecdotes overwhelming the dialog or HiPPO decision-making to one based on data and truth-seeking.

The key aspect to remember when incorporating feedback into your product roadmap process is that the way to go about doing this is never simply taking the most frequently requested features and putting them at the top of your roadmap. Customer feedback is one input amongst many into your roadmap and needs to be equally considered alongside strategic product priorities driven by your team's objectives.

Similarly the value of the feedback is not simply the features customers have requested, but the synthesis of the learnings from that feedback. For example, you might notice that a subset of customers are asking for very specific power-user functionality and you might ultimately decide that rather than addressing the feedback, the user segment is no longer a target customer. Or you might hear a lot of feature suggestions / confusion / issues with a specific feature and instead of simply addressing the specific feedback, you realize you need to re-think the entire experience from scratch to avoid the current user sentiment. Or you might decide the lack of feedback and usage of a feature doesn't mean that people are enjoying it, but that it's getting limited usage and it may be time to kill it. This synthesis of learnings from feedback is absolutely critical. When getting started, I encourage teams to get together on a regular basis to have these types of discussions on how best to act on the feedback you're hearing. Over time as you get accustomed to leveraging your feedback river and system of record, you'll find this becomes a fairly natural part of your product roadmap discussions.

I hope this provides a detailed look at the continuous feedback loop I've leveraged in developing products and gives you food for thought on how to incorporate into your own team's process for getting the maximum value out of the critical feedback your team is hearing from your customers.