Marianne Stjernvall explains the evolution of CRO and the importance of centralizing your CRO Program to create a data-driven organization
Before becoming a leading specialist in CRO and A/B testing, Marianne Stjernvall was studying computer and systems science when a company reached out to her on LinkedIn for a position as a CRO specialist, which for her turned out to be the perfect mix of logic programming data and business and people.
Since then she founded the Queen of CRO where Marianne acts as an independent CRO consultant helping many organizations with experimentation, CRO, personalization and creating a data-driven culture for growth.
Previously, Marianne worked for companies such as iProspect, TUI and Coop Sverige where she spearheaded their CRO roadmap and developed a culture of experimentation. Additionally, she was awarded CRO Practitioner of the Year in 2020.
Wingify’s VP Marketing Marylin Montoya spoke with Marianne on the importance of contextualizing A/B test data to make better-informed decisions. Marianne also shared her own take on the much debated build vs buy topic and some wise advice from her years of experience with CRO and experimentation.
Here are some key takeaways from their conversation.
The importance of contextualizing data
For Marianne, CRO is becoming a big part of product development and delivery. She highlights the importance of this methodology when it comes to collecting data and acting on it in order to drive decisions.
Marianne stresses the importance of putting data into context and deriving insights from that data. This means companies need to be able to answer why they’re collecting certain information and what they plan to do with that information or data.
CRO is the key to unlocking many of those insights from the vast amount of data organizations have at hand and to pinpoint exactly what they need to optimize.
“What are you going to do with that information? You need context to provide insights and that, I think, is what CRO actually is about,” Marianne says.
This is what makes CRO so powerful as it enables organizations to take more valuable actions based on the insights derived from data.
When done right, testing within the spectrum of CRO can help move organizations into a completely different path that they were on before onto a more innovative and transformative journey.
Centralize and standard your experimentation processes first
When companies are just starting to create their experimentation or CRO program, Marianne recommends having parts of it centralized and to run tests within a framework or process to avoid teams running their own tests and executing these tests all over each other.
Otherwise, you could have different teams, such as marketing, product development and CRO teams, executing tests with no set process in place which could potentially lead to chaos.
“You will be taking decisions on A/B tests on basically three different data sets because you will be checking different kinds of data. So having an ownership of that to produce this framework and process, this is how the organization should work with these kinds of tests,” says Marianne.
With established frameworks and processes in place, organizations can set rules on how to carry out tests to get better value out of them and create ownership for the entire organization. The trick is to start small with one team and build in these processes over time onto the next team and so on.
This is especially important as Marianne argues that many organizations cannot increase their test velocity because they don’t have set processes to act on the data they get from their A/B tests. This includes how they’re calculating the tests, how they’re determining the winning or losing variation and what kind of goals or KPIs they’ve set up.
In other words, experimentation needs to be democratized as a starting point to allow an organization to naturally evolve around CRO.
Putting people at the center of your CRO program
When it comes to the build vs buy debate, Marianne argues that an A/B testing tool will not automatically solve everything.
“A great A/B testing tool can make you comfortable in that we have all the grounds covered with that. Now we can actually execute on this, but the rest is people and the organization. That’s the big work.”
In fact, companies tend to blame the tech side of things when their A/B testing is not going as planned. For Marianne, that has nothing to do with the tool; the issue primarily lies with people and processes.
As far as the build vs buy debate, before deciding to build a tool in-house, companies should first ask themselves why they want to build their own tool beyond the fact it’s more cost-efficient. This is because these tools need time to get set up and running. It may not be so cost-effective as many tend to think when choosing to build their own tool.
Marianne believes that companies should focus their energy and time on building processes and educating teams on these processes instead. In other words, it’s about people first and foremost; that’s where the real investment lies.
Nevertheless, before starting the journey of building their own tool, companies should evaluate themselves internally to understand how teams are utilizing and incorporating data obtained from tests into their feature releases.
If you’re just starting on your CRO journey, it’s largely about organizing your teams and involving them in these processes you’re building. The idea is to build engagement across all teams so that this journey happens in the organization as a whole. (An opinion that was shared by 1,000 Experiments Club podcast guest Ben Labay).
What else can you learn from our conversation with Marianne Stjernvall?
- What to consider when choosing the right A/B testing tool
- Her own learnings from experiments she’s run
- How to get HIPPOs more involved during A/B testing
- How “failed” tests and experiments can be a learning experience
