As a Growth PM, experimentation and learning are the two most important tools at your disposal. Growth is about figuring out which metrics you are trying to move, how solving customer problems will improve those metrics, and which levers you should focus on.
The faster you can go from identifying problems and creating hypotheses to testing and learning from the results, the more growth you can drive. Keep reading for four tips to make the most of experimentation and learning as a Growth PM.
1. Mindset shift
As a Growth Product Manager, you will be running multiple experiments, and many experiments will not give you the results you are looking for. This requires a mindset shift. A failed experiment is not necessarily a bad outcome if you learned something that helps you make a better decision or design a better experiment next time.
Core PMs also experiment, but the nature of the work can be different. A Core PM may spend weeks building and launching a new feature or experience, while Growth PMs often work through many smaller experiments and iterations to optimize an existing product and capture more of its potential.
Of course, this only works when the company culture supports experimentation and values the ability to fail, learn, and iterate.
2. Focus on the right experiments
Experiments take time and effort, so you need to be thoughtful about what you test.
Before running an experiment, ask whether the decision is actually worth experimenting on. If the cost of testing is higher than the value of getting a better answer, it might be better to make the decision and move on. For example, a small bug fix might not be worth running an experiment on.
You also may not always have enough data to run a meaningful experiment. This can happen with enterprise software products with a smaller number of users or customers. In these cases, you can rely on customer research, your GTM teams, talking to your most vocal customers and watching how they use the product, the data you do have, and your own judgment. DIY user research tools like Lyssna have also been helpful for me.
For experiments that are worth testing, use prioritization techniques like ICE + Time (Impact, Confidence, Effort/Cost, and Time to get results). Rate each dimension as Small (S), Medium (M), or Large (L) and rank the solutions. ICE isn’t perfect, but it gives you a good direction on what to work on.
(I) Impact: How much do you expect the solution to move the outcome metric?
(C) Confidence: How confident are you that the solution will actually work and have the expected impact?
(E) Effort/Cost: How much engineering and other cross-functional effort will be required to build and test the solution?
Time to get test results: How long will the experiment need to run after launch before you have enough data to make a decision? This is largely determined by the required sample size and the amount of eligible traffic.
Also think about sequencing. The highest-impact idea may not always be the first thing you work on. Sometimes a smaller initiative needs to happen first because it unlocks a larger opportunity later.
The goal isn’t to run more experiments. It’s to spend your experimentation resources on the questions that matter most.
3. Learn from every experiment
Before starting an experiment, be clear about your hypothesis, what you expect to happen, and which metrics you are trying to move. This gives you something to compare against when you look at the results.
Learning is key for a Growth PM. You need to learn from both wins and failed experiments. As I mentioned earlier, most experiments will fail. When an experiment doesn’t work, don’t immediately move on to the next idea. It’s very important to understand the “WHY” behind the results.
As a Growth PM, I find myself obsessively asking “WHY, WHY, WHY” until I get to the bottom of what is really happening. It can be mentally taxing, but it has helped me uncover insights that I might have missed by only looking at the surface.
Here are a few questions that will help you get to the bottom of the results.
What did you expect to happen?
What actually happened?
How did user behavior change?
Why did it happen?
Was the metric wrong? Was the customer problem wrong? Was the solution wrong? Or did the experiment itself not test the idea properly?
Is this for all segments or any particular one?
The same applies when an experiment works. A winning experiment doesn’t mean the solution is fully optimized. There may still be questions about how it should work, which users it works best for, or how much further you can take it.
A win gives you something to build on. A loss gives you something to learn from. Both should change what you do next.
4. Move fast
Moving fast is another key characteristic of a Growth Product Manager. You will never have perfect data, perfect research, or perfect insights. You have to make a smart guess, move forward, and learn along the way. I outlined in my previous blog post how I approach making decisions, and you can use that framework to help you move forward.
Over time, these small improvements start to add up. That’s why I think growth is a game of increments. The advantage doesn’t come from getting every experiment right. It comes from learning quickly, making better decisions, and continuously improving the product.