Building a successful experimentation program is a marathon, not a sprint.

I know what you’re thinking—the “marathon” analogy is worn out. 

However, after listening to a recent panel of experimentation experts at CDP Week, it’s hard to find a more accurate analogy.

The panel featured Laura Brodie, Director of Customer Lifecycle & Growth at Ritual, Mike Doll, Head of Data & Analytics at Udacity, and Josh Wexler, Chief Product Officer at Chopra Global. These experts shared their experiences developing experimentation and personalization strategies at fast-growing companies. Despite their different roles, each expert’s advice shared a common theme: You can’t build a sophisticated experimentation program overnight; it requires the right people, process, and data. It’s a marathon, not a sprint. 

Step 1- Make a Plan

When you decide to run your first marathon, the first step is often to create the right training plan for your personal fitness level. Someone who regularly runs 5k and 10k races is going to have a different training plan than someone who hasn’t run since high school gym class. 

Similarly, building an experimentation program starts with creating the right plan based on where your business is today and where you want to go. You need realistic milestones and timelines. Laura Brodie, Director of Customer Lifecycle & Growth, described Ritual’s approach:  

“We had to take [experimentation] in a step by step approach...The first is just using the data in a real life scenario and making sure it’s QA’ed properly, that everything is working as expected, and that we’re sending the data to all downstream destinations, and that it’s working...Then, after building the foundation, and having an understanding of how the data is flowing, then you can start experimentation, starting with simple tests that may not drive huge lift but exercise the muscles...Then, once we had the foundation in place, we were able to dive into a lot more experimentation.” - Laura Brodie, Ritual

Step 2 - Invest in the right tools

Once you have your marathon training plan, you need to invest in the right equipment to start training. You might need to invest in a new pair of long distance running shoes or warmer athletic gear if you’re going to train during the winter. 

The same goes for experimentation. You need to invest in the right tools to help you execute your plan, especially the right data infrastructure to manage your data. All three speakers use Segment as their Customer Data Platform (CDP) to collect, standardize, unify, clean, and activate their customer data. Segment has helped them understand the complete customer journey, and enabled their companies to invest in a data first mentality. Mike Doll, Head of Data & Analytics at Udacity, shares the importance of putting the right tools in place to manage your data:

“Invest in really clean, flexible, simple data. Keep it accessible, keep it consistent, and well governed so that data has trust.”. -Mike Doll, Udacity

Once the speakers implemented Segment, they focused on educating their teams around what data is available and what types of tests and experiences they were able to create. 

Step 3 - Start executing your plan

Once you have your shoes and your training plan, it’s time to actually put the sneakers on and start running.  But you have to start with small, manageable running goals because you might injure yourself if you try to do too much too soon.

Once data and customer teams have the tools in place and experimentation frameworks to follow, they should work in partnership to create hypotheses and an experimentation roadmap. At this stage, it’s about following your plans to slowly build up your stamina, and being patient while experimentation becomes more natural. Just like training for a marathon, running long distances eventually becomes more natural and more satisfying as you start to achieve your goals. 

Josh Wexler, CPO at Chopra Global, shared a simple framework for how he thinks about data and what to test:

“What questions do we want to know? How can we measure and find out the answers to those questions? What are we going to do if we get the answers to that question? How useful is it to understand the answer to that question and take action on it?” -Josh Wexler, Chopra Global

Step 4 - As you reach milestones, set harder goals 

Once 10 miles starts to feel manageable, you push for 15 miles. Then 20. Then the full 26.2.  But it’s going to take patience and determination to get there.  And like many addicted runners, crossing your first finish line will likely be just the beginning of a long running journey. 

Once you experience the endorphins from achieving your KPIs through experimentation, you will likely want to continue to improve your experimentation program, dive into more sophisticated personalization, and perfect the customer journey.

All three speakers reiterated that long term experimentation programs, just like long distance running, take discipline. They require taking the time to set up the right types of tests for clean results, they require working together with data experts, and they require the right data. With the right customer data infrastructure, the right data, and the right frameworks and strategy in place, you can begin to focus on the optimal customer journeys. 

"At Ritual, we use our data to create deep segments and bring our customers down a really personalized journey that sticks with them for the full tenure they are subscribed to Ritual…The journey we send someone down who has just started a prenatal vitamin...is very different from someone who has just purchased our men’s 50+ vitamin...” -Laura Brodie, Ritual

To improve your own data strategy, take our customer data maturity evaluation and receive actionable recommendations on taking the next step. 

If you want to learn more about how Segment can help you accelerate your experimentation and personalization strategies, reach out to request a demo today.