If there's one thing we've learned over the last half decade, it's that companies obsess over picking the right analytics and growth tools. Should we use Amplitude or Mixpanel? Should we use Redshift or BigQuery? All tool categories will have hundreds of players whose feature sets will allow you to accomplish 95% of your use cases with ease. We've now seen 15,000+ stack evolutions over 6+ years and we've learned one thing — It's not about picking the perfect tool. It's ALL about being in a position to quickly adapt your stack to changing business needs and the ever changing Martech ecosystem.
In this article, we'll look at real data of how stacks have evolved over long periods of time, and we'll highlight common patterns we've seen over and over again.
The new product evolution
New products are brought to market in a few distinct business stages. First a Minimum Viable Product (MVP) is built; then the MVP is given to a small group of customers for testing during the private beta. Early on the MVP will have bugs and often lack critical features required for product market fit — but the private beta period gives the product team ample time to iterate on the product until it's stable and valuable to testers. During this phase, the team will start measuring product market fit by collecting engagement event data and using retention cohorts in an analytics tool like Amplitude or Mode Analytics.
Once the new private beta cohorts come in week after week and get value, it's time to launch. The launch is all about making a splash and attracting the initial cohort of the market. This will be when acquisition goals come into play in addition to core product market fit. You'll want a web analytics tool like Google Analytics on the page as well as product engagement events in an event analytics tool like Amplitude.
As the the number of retained customers grows, the product will start turning into a business and operational teams will need to be built to support and sell the product. This is when new leads fill flow into a CRM, like Salesforce, and you'll start doubling down on paid acquisition channel using Facebook & Google Ads.
Now that we've seen how new categories are added to a product, it's time to look at real data of how customer stacks have evolved.
The bake-off
A first obvious trend is that users tend to 'bake-off' similar tools. They may turn on two at once, see how they work, and then turn off whichever one doesn't meet their business needs.
This is one giant bake-off of many tools. This Segment customer tried 7 tools at once, and decided that Mixpanel was the only tool they need, and continued to use Mixpanel for 1+ years into the present!
This bake-off shows how a company uses over 14 tools in their stack over a period of 3+ years. You can see the company evaluates email marketing tools Customer.io and Intercom over a period of 3 months, eventually choosing Intercom. This is followed by a more recent analytics parallel test of Mixpanel and Keen.io. As your company matures, different business users will want to introduce new categories or try new tools within a category. It's important to be able to cheaply do these tests.
The refresh
Of note, people seem to adopt tools in twos, threes, fours, and fives. This is usually associated with a new hire trying out a new stack during a refresh — usually a new team lead of a growth team, marketing team, or an analytics team. You should expect to hire data & growth experts that will want to bring their own stack of tools.
In this graph, we see two distinct stack refreshes following the initial setup. The first setup indicates a new product — a starter analytics tool known as Gauges and the error monitor Sentry. As the product matures, a refresh shows the addition of analytics tools (Mixpanel & Heap) and a retargeting platform (Perfect Audience). The second refresh adds another layer of paid acquisition pixels (Facebook, Adwords, Quora).
This one we dubbed the "we hired someone" graph. You can see a pattern of 3 early stage tools (CrazyEgg, Customer.io, SendWithUs) being supplemented with 14 new tools. This is associated with a new hire leveraging the Martech ecosystem to implement email & paid ads & analytics use cases.
The "throw out the old mess"
There are a number of places where it's easy to see a stack / tracking plan refresh. In these cases, there's a pretty clear set of integrations which were being used, and then were promptly disabled.
This Segment customer decided to disable 9 integrations, almost their entire stack, and then enable 13 new integrations — a complete refresh.
The GDPR refresh
As new privacy regulations come into effect, companies may choose to delete data from certain vendors in their stack.
This company decided to disable their paid acquisition stack in advance of the GDPR deadline. This included 6 ad networks like Facebook, Twitter, Adwords, and Bing Ads. The ability to respond quickly to regulation and privacy environment changes is one of the benefits of having an adaptable stack.
The hyperspeed
And if you want to approach hyperspeed you can check out Segment using Segment. 50+ integrations in two years (We don't recommend this).
Category adoption
In addition to a company's specific stack changes, we can study aggregate data to see when new product teams add tool categories. The following graph shows what percent of a specific integration was added as the first integration, the second integration and so on …
For the first tools, people approach us predominantly with an "analytics use case". Over 60% of the 'first tool' installs are made up of either Mixpanel or Google Analytics. Moreover these tools have some sort of real-time interface or data QA check.
For the 3-9 tools, we start seeing a significant uptick in advertising: Facebook Pixel and Adwords. By the fourth integration that users enable, Facebook Pixel & Google Adwords have the top spots.
Lessons learned
Stack changes are correlated with new hires and changing business goals. New products generally launch with a light stack of a few analytics and email marketing tools. Growth consultants and new hires are brought in during company scaling phases and try new growth strategies like A/B testing (Optimizely), retargeting (Adroll), or just new data to create audiences on (Clearbit). New product launches often cause entropy on the stack (such as new in-app messaging campaigns on Intercom). As a marketing organization becomes more mature, growth consultants will be brought in to manage acquisition, performance marketing, and any referral programs.
Scale and specialization allows more advanced tools to be introduced. When a company is small, with limited resources, they'll typically purchase more user-friendly, approachable tools that the product team can get into and understand easily (Intercom, Mailchimp, Amplitude, etc). As the company scales and gets more sophisticated, more complex and powerful tools are introduced (Salesforce Marketing Cloud, AutoPilot, Looker).
New analytics & growth tools enable serious strategy changes for businesses. Analytics tools lead to insights that shape new pricing & packaging models. They also lead to a better understanding of which marketing channels work best for a company, and where to double down marketing spend. Analytics tools help a company understand which content works best for their customers. In-app messaging drive better engagement and retention within the product, increasing growth.
Prepare for change. The average Segment customer collects data from 7 sources types (web, mobile, crm, email, ..) and integrates each source with an average of 8 destination categories (analytics, warehouses, email marketing tools, re-marketing and paid acquisition, ..). That means the average customer is utilizing 56 point to point integrations. As we learned above, new business strategies, new hires, and new regulations cause high entropy in the stack over many years.
Reduce the cost of change. In 2018, Segment hit a new record: our users enabled 86,551 new destinations over the course of the year. Assuming a 40-hour work week, our customers were turning on a new tool every 1.4 minutes of every workday. To better understand how impactful this was, we surveyed 100 of our customers. On average, they told us it took 100 engineering hours to connect a new tool, not to mention the 21 hours of recurring maintenance per month. This might sound like a long time to just add a few hundred lines of code, but to fully QA, test, and ship to production, the time quickly adds up. Throughout 2018, Segment saved engineers more than 4,000 engineering years worth of implementation time. We recommend you use an analytics platform to abstract data collection from integration.
Choosing the right stack