During the last couple of weeks, I gave my talk on “The Journey to Product-Market Fit” three times. It was in three different accelerators and training programs, and the audience was entrepreneurs and experienced product leaders.
The feedback on all three talks was great. I don’t always get quantitative feedback, but for one of these talks I had it and it was 9.55 out of 10. I was proud and happy.
Why am I telling you this?
Because I have been giving the exact same talk in a different training program over the last few months, and the feedback was very different. My scores were constantly between 7 and 8, and one session even scored as low as 4.8. Ouch.
They were actually telling me that my product (my talk) is not good.
But guess what? The talk I delivered was exactly the same in all cases (putting aside factors like the energy-level on a specific day etc.).
So if the product wasn’t the problem, what was the problem? Why did I score extremely high in the last 3 sessions, and extremely low on the previous ones?
Because the audience was different.
Unlike the last 3 sessions, the audience in the other training program mostly consisted of freelancers and people who aspire to become product managers. All smart and motivated people, but still a very different audience.
And unlike the last 3 sessions, the audience in the other training program didn’t really need the product I had to offer. They needed something else.
One of the reasons they didn’t need my product (my talk) is that their problems are very different from the ones experienced by entrepreneurs and seasoned product leaders. Their goals and motivations are different, and their challenges reside elsewhere.
The pain I was addressing with my talk, wasn’t a real pain for them, and as a result, they felt it was good for general knowledge, but not much more. A nice-to-have product at best, but not something they really need.
Their qualitative feedback also expressed a similar sense: they said it wasn’t practical enough for them and they didn’t learn anything they can use in their day job.
Given this explicit feedback, the immediate conclusion is that it’s a bad talk. That’s what I felt too — who wouldn’t? They said it very clearly.
A/B Testing My Product
Luckily, I knew exactly which audience I created the talk for, and knew that this audience wasn’t it. I decided to keep the talk as is, deliver it to the right audience and see if it works better. I still wasn’t sure it was a good enough product but decided to run this A/B test before changing anything. Note that it wasn’t a classic A/B test most of us are used to running since the variant here wasn’t in the product — it was in the audience itself.
That’s another reason I was so happy with the quantitative feedback I got last week — it validated for me that not only that the talk itself is good; it also showed me (again) that entrepreneurs and seasoned product leaders often need help in the area of product-market fit, which is super important to validate since it is the primary service I offer in my business.
When your product gets bad feedback, isn’t getting the traction you expected or generally doesn’t deliver the results you were hoping for, it is very easy to conclude that the product isn’t good enough.
Bad Product Feedback is Not Always a Product Issue
When your product gets bad feedback and isn’t getting the traction you expected or generally doesn’t deliver the results you were hoping for, it is very easy to conclude that the product isn’t good enough.
But sometimes, as in my case here, it wasn’t the product’s fault. The problem lies elsewhere.
Speaking of problems, products should be created in order to solve a specific problem, for a specific audience.
The problem and the audience always come together — a problem doesn’t exist in a void, and a problem that is relevant for “everyone” is never specific enough.
Side note and warning for B2B product leaders: if you think this is not relevant for you, you are wrong. The problem you are solving is always a problem of specific people with specific characteristics within the organization. You are not off the hook. If at all, your job is more complicated since the people within the organization who experience the problem are not always the ones who can pay for it, so you need to solve multiple problems for multiple stakeholders to be able to succeed.
So What Can You Learn From Product Feedback?
Always have a deep understanding of the problem you are solving and for whom.
The problem and the characteristics of the audience should be defined at a much more explicit level than what feels intuitively right. You can find more information and tips on this topic in my executive’s guide to strategic product management — Speed-Up the Journey to Product-Market Fit.
Make sure it is a real problem, painful and important enough for them. It’s always better to do this before you build the product, but even if your company didn’t do it up until now and you already have a product — better late than never. You don’t need to be worried, many companies evolve this way — they start with a generic need and only after they already have a few customers — and something isn’t working very well — they get into defining their product strategy more explicitly. They are much more knowledgeable to do it right at this stage — after their product rubber hit the market road — than on paper when they started.
Do whatever you can to reach the actual audience you intended for. It’s not always easy, and in some cases, you will find that other audiences are showing up instead. If that’s the case, ask yourself why this is happening: it could be because of marketing to the wrong audience, but could also be because the original audience and problem you thought of are not there. Or, they might be there, but the other audience you see has a much bigger need that your product can fulfill — you might have an opportunity there.
When things are not working well, understand the root cause before changing anything. This is generally a good practice in life, but let’s stick with product feedback :-).
When your product isn’t performing well, always get back to the problem you are solving and for whom.
Sometimes, it’s the wrong audience. In others, the problem is not painful enough. Changing the product blindly in these cases won’t help you. You should change your product strategy first (or create one if you didn’t have it up until now).
Sometimes, the problem and audience are just right, but the product doesn’t actually solve it (usually because it does not consider everything the audience needs to solve this problem, and either only takes them part of the way towards a real solution or conflicts with other constraints and needs they have). These are the cases where changing the product without changing the problem and the audience can actually make a difference.
We live in an agile world, so listening to feedback and changing accordingly should be part of our DNA. But changing without understanding what needs to change and why can do more harm than good. Always keep an eye on the bigger picture.