Opening black boxes: How customer data provides the key [Podcast]

Madelyn Mullen on August 17th 2020

This is Segment’s Data Council series, where members share stories about using Segment to work with customer data within Enterprise companies. To make sure you don’t miss an episode, subscribe on iTunesSpotify, or your favorite podcast player. You can also read a lightly edited transcript of the conversation below.

Imagine what it would be like if your product managers had an overview of support tickets, billing issues, sales interactions, and your users’ clickstreams—all of which live in a unified data warehouse. It would be the Holy Grail of data management. Instead, most companies silo datasets in black boxes visible only to the relevant teams and leadership. If it were the other way around, your company could spot issues earlier and deal with them efficiently, without losing time by running questions up the flagpole and waiting for them to come back down.

Vishal Rana, Director of Product and former VP of Customer Success at Segment, has observed firsthand how good data leads to organizational success. With over a decade of helping enterprises grow, Vishal Rana shares customer insights from that first "aha" moment to when retention experiences get tuned just right.

Before his time in product, Vishal led Customer Success at Segment for three years and Professional Services at Medallia for five years. To see the journey from vision to reality and all the prioritization in between, tune into this episode about how to succeed with customer data—especially with those last-mile activities.


  1. Keep using your favorite tools, and push to get to see all the data that everybody else has collected in a trusted, centralized core of customer data.

  2. Define customer success metrics using very rational observations about your customers’ usage behaviors or brand engagements.

  3. Engagement data without purpose doesn’t serve anyone. Early in the product's lifecycle, define a particular business problem you want to solve with engagement data and collect accordingly.

Audio highlights:

  • “Everybody's trying to transform other people's data into their format so that it can live through the lens of their core.”

  • “Those systems, in their own siloed way, have a treasure trove of information about what a customer is trying to accomplish that isn't in the [event] clickstream. What's great is that with Segment, you can take all of that data and put it into a data warehouse, which—is agnostic to the type of information we're pulling in—and then marry it with that event-stream data. And now, for the first time in most companies' digital maturation, they're able to see everything in one place. Then they can start to marry the information they knew about the customer before they showed up (maybe in their CRM) and add that context to engagement that [the customers] are having with someone else [at the company].”


Read the transcript:

Madelyn Mullen: I'm Madelyn Mullen, part of the Segment Product Marketing team. I'm joined by Vishal Rana, a Segment Product Leader who focuses on our enterprise customers. Prior to his product role, Vishal led Customer Success at Segment for three years and professional services at Medallia for five. At Medallia, he focused on using data to create customer experiences across global brands. Welcome, Vishal! 

Vishal Rana: Hello. It’s great to be here!

Madelyn: How do you define customer success?

Vishal: Defining customer success is different for a lot of companies. But generally it’s about making sure customers get value out of your products and services and making sure they come back for more and are telling their friends how much they love working with you. Those are all the characteristics that most customer success leaders measure themselves on. It’s retention, advocacy, and expansion, right? Customers coming back to buy more. To measure beyond those outcome metrics, you ask about the things individual customers are doing in their interactions with you so that you can predict those outcomes. And that's where it gets really gnarly and where getting into these deep insights is really important. 

Finding the first light

Madelyn Mullen: Thinking about prediction and measuring retention, what role does data play? 

Vishal Rana: For retention, data's really important. Because if you're thinking about B2B SaaS like we do at Segment, you're usually an annual contract. You're trying to figure out, over the course of the year, the things the customer should do over that period of time. By the time we'd come back and say, "Hey customer, would you like to continue working with us?” it's a non-event. They decided, "Of course. I've gotten so much value from working with you that we're of course gonna stick around." 

You can expect it in a company like Segment. We are measuring just about everything we can to understand the interactions you're having with our company. There's the obvious stuff, where we measure your engagement with our product. You would expect us to do that. But we're also measuring the engagement you have with our team. How often are you talking to your CSMs and your salespeople? How often are you writing and tickets and things like that? What about the other interactions around billing? We're measuring all of that stuff and trying to get a full picture of all the interaction points and then trying to determine which of those are predictive.

We’re also spending energy directing our time and effort to make sure we're doing the most positive activities towards an individual customer. Data's really central both in understanding retention, where we're trying to make sure customers are doing positive things that we know are related to retention. It's also related to our expansion motions. When customers are using a particular part of the product a lot, it’s usually a signal to us that there's a high value here and we should spend more time talking to that customer. We do use all of the signals we can get from all across the Segment application and interactions with individuals to predict what we think is going to happen and to course-correct when necessary.

Madelyn: Before you have all of that information, what might be some of the early success with Segment? 

Vishal: If you think about a customer in their broad journey with Segment, the core job we do for most companies is to move data from one place to another. Early success is making that first connection from a source of information where a customer interacts with you to the destination, which is why we formally call them Source and Destination.

The Source is where customers interact with you. The Destination is a [marketing or analytics] tool or some other application where you need that information for someone to do their job. We've got a pretty well-defined path that we see most companies go through when they implement us. The first thing most people do is that they implement our libraries and SDKs and send data out to AdwordsFacebook, and Google Analytics.

(Take a quick tutorial on creating a Javascript web source and Google Analytics destination.)

Those three things are pretty common. What this lets them do is spend money on ads, see what their traffic is, and get initial feedback on what's working or not working on their website or in their app. That's usually the "aha" moment for most people.

It's not a particularly novel solution, right? Just about everyone has done this before they had Segment. The thing they learn here is that they can do this all in a few hours instead of a few days. And that's what's really powerful about those initial successes: “I've gone through this motion that I've probably done before, and it's painstaking. But I did it again with Segment, and it was pretty easy.”

Madelyn: Customers realize that quick time-to-value, and we're able to see that on our end as well. What does it look like from a long-term success perspective seeing the customer data?

Vishal: Zooming out, when I think about this as a Customer Success person, every industry has got to figure out what their retention moment or their long-term success moment is. And those are usually different metrics. A lot of us talk about the "aha" moment as the "first light." When do we first realize that this thing is working? When you think about long-term success, it's when the person feels like they've gotten significantly more value than what the application or the product that they've bought costs—or the expense to get the thing working.

For different organizations that measurement of this varies quite a bit. I'll use an example in finance: I used to lead the financial services vertical at Medallia, and for most banks we worked with, the key for them was they needed three accounts and direct deposit.

That was pretty standard in the market. You needed a checking account, savings account, something else like a credit card or a mortgage or some other auto loans—and direct deposit. Once you had those things hooked up at a bank, they knew they had you. Some exorbitantly high number of people did not leave a bank if they had gotten to that level of engagement.

At that point, the costs of switching and moving to something else—in addition to setting it all up somewhere else—were way too high. People wouldn't get to that level of engagement with you if they didn't like what you were doing, right? So there's some predictive measurement there, but there's also a backcasting. You're buying more things, because you actually are aligned with the services this company's providing.

That's an example of a company that thinks about what long-term success looks like. And the key here is understanding what [long-term success] is for your business. And usually these are pretty rational things. These aren't surprising, right? These are your core value drivers, where you’ve got a relationship across multiple products at the bank.

Starting with what matters

Madelyn: For customers defining rational success metrics, what are some common milestones on this journey using customer data?

Vishal: For Segment, we can enable a number of different use cases for a company. The first one we talked about was spending money on advertising and analytics. Everybody does those things, and they're pretty standard.

But eventually we find that companies start to move on to more advanced use cases. They're trying to do more than they could with those few tools. And they usually want to do some messaging. Maybe it's an e-commerce company, and they want to figure out how to recover someone who's abandoned the cart. Or maybe somebody has taken a bunch of steps, but they want you to make a recommendation to them. In those cases, you need some sort of messaging platform where you need more advanced analytics. Those usually become the next set of applications you turn on, and we can observe that.

Once you've set up the first set of advertising and analytics cases, we start to see people add messaging tools, and we start to see them add additional analytics cases. [Segment CTO and Co-Founder] Calvin French-Owen put out an excellent blog that shows you the progression of companies through the technology stack that we have.

But one of the watershed moments for us, honestly, is when somebody starts to integrate sets of data that aren't naturally part of what we call “the events stream.”

Zoom with margin

Example of a Segment events stream. Read more in the Growth of Stacks 2019 blog to see the most popular customer data types integrated into the event stream.

Until now, most of the things we were talking about naturally understood that a user was clicking on things, and we were trying to capture that information and put it into a tool to watch that customer journey live. What we really see—when customers start to add other sources of data that don't naturally marry with event data (like a CRM, a billing system, a customer support system)—is that all of these sources of information are engaged outside of this click-through behavior.

They're engaging with a salesperson, in the case of the CRM. Or they're engaging with our Finance or Billing team in the billing system. They're engaging with our Customer Success or Support team through our support system. And those systems, in their own siloed way, have a treasure trove of information about what a customer is trying to accomplish that isn't in the [event] clickstream. With Segment, you can take all of that data and put it into a data warehouse—which is agnostic to the type of information we're pulling in—and then marry it with that events-stream data. 

And now, for the first time in most companies' digital maturation, they're able to see everything in one place. Then they can start to marry the information they knew about the customer before they showed up (maybe in their CRM) and add that context to engagement that the customers are having with someone else at the company.

That starts to create this point of connection for a fair number of disparate technologies into a unified platform. Not only is this information together for the first time ever, but we now start to hear customers who have really engaged with Segment start saying, "Hey, no new systems are allowed to not be on this platform, because the network effect of being together with the rest is information is so high.”

It's so powerful that half the value of having a new tool is that it can consume and send its data out to all the other tools. For us, that’s this real moment where a customer says: "Yep, I've got it in this ecosystem, and I have the ability to add and remove tools as I need them. I can move all the information those tools generate to wherever it needs to be.” And that is the real value moment for Segment: where people really get what a customer data platform can do for them.

Together for the first time

Madelyn: Vishal, over the past 10 years, have you seen these definitions of success using customer data change? 

Vishal: When we think about what you get out of using customer data early on, companies really are saying, "Hey, let's measure everything and try and push these volume metrics, like the amount of people who land on your site or the number of actions they're taking.” They’re not super sophisticated in what they're measuring.

What we've seen change over time is—as we're able to collect richer information and have more confidence in what signal that information is providing—people are getting much more advanced in how they're actually implementing their actions afterwards.

A good example is that we've got a large media company just measuring the number of people watching a particular clip at a particular time. Whether or not they watched an ad, that helped them understand roughly how their revenue metrics were doing as people looked at their clips. They made the money through serving advertisements. And as they started to collect this data and get a more robust picture of who was watching which ads at which time and which calls to action (like an email or a push notification), were driving them to actually watch this content.

What they started to do was super interesting. They would use different types of content to retire the quotas they had on the types of people who were watching an individual piece of an advertisement. If you think about a rerun, that will capture a fanatical audience over and over again. They would run that to fulfill the requirements for something, and then they'd take their blockbuster show and they put a higher-paying ad into that. 

That seems super simple as a very high level. Put your best paying ads in your blockbuster content, and your lower paying ads in the others. What you could actually do outside of the blockbuster question, which is the easy one, is just drive much better yield in these middle-performing pieces of content, because you knew who was going to show up to watch. So you could get really targeted audiences to watch a very particular piece of content. We went from just trying to drive more eyeballs early on to really rationalizing where we put eyeballs.

The only way you get to that is if you really, really trust your data. This is a large Fortune 500 media company, and when they migrated to Segment, they spent a bunch of time parity checking and making sure they could trust the information that was flowing through, because they already had a bunch of pipelines measuring activity like this. They spent a lot of time parallel tracking Segment information against the other systems they had and making sure that they believed it.

Once they had gotten to that point, they spent a bunch more time inside tools like our Protocols and Personas applications. They used Protocols to make sure they were capturing the right information and that there was less noise. They really relied on the signal that was coming out. And they used Personas to really think through to unify the information they had, so they could get better signals.

(Take the Protocols course and Personas course on Segment University.)

Because—once I've got confidence in the information I'm collecting and I've enriched it with all this other context I might have about this user—there's now a really powerful signal that I can use with machine-learning algorithms and other predictive models to do even more powerful success outcomes. It’s not just saying, “I've got lots of engagement,” but instead, “I have a particular type of engagement, which fulfills a particular business problem that I'm trying to solve.” 

The maturity from these blunt metrics to these more advanced metrics really shifts, because a broad, complete view of your customer is what enables you to zero in on these focused, business-outcome metrics. You know so many more things about this individual. So when you use the data from that rich context, you can actually focus on a very specific business problem versus the blunt problem of more users, more customers, more of something.

There's an insurance company that uses Personas so that the support agents can see the click path before someone gets on the phone. They can help pick up where that person left off in the flow and then also understand the other products they might have with the insurance company. That's a much faster sequence. It reduces the number of things that agents need to do when they are on the phone, because they're able to see one unified profile, and it exists in the application they use. There's one system that gets all of the data, and instead of what a typical support or contact center experiences, where agents have 15 different tabs open at the same time to see every individual piece of information, which makes that person super unhappy at their job, because they're just constantly multitasking but less effective. So this is a really helpful way both to add data they wouldn't normally get (which is the clickstream from the website) but also to give them the unified profile of this user by pulling information from billing and marketing and other stuff.

Companies engage with our data-governance and cleanliness products to make sure the information flowing through is absolutely right and their dashboards are reliable for all of the audiences they're trying to serve—especially executives. Once we've gotten there, Marketing and Success usually start to look at what's going on with this customer data platform. Marketing is trying to get even more information into their campaigns so they can target individual customers more accurately. And Customer Success is trying to get more context by asking: "Hey, what was the journey the customer was on until they engaged with me? What else can I know about what this customer is trying to do so that I can serve them better?" 

Connected context drives customer data platform adoption

Madelyn: How does the organization's data maturity influence the roles that would get involved to create this roadmap, to take your customer success metrics, and make them granular and enable experimentation?

Vishal: We definitely see an evolution of roles, and Segment often enters an organization from one area. Oftentimes it's Product or Engineering who pull us into that primary use case we talked about before. It's that first light of saying, "Hey, I've got to move data around, and it takes time, and it's annoying to connect all these things."

What we learn is that the vast majority of the value we create by solving this problem lives inside other organizations that are either analytics or marketing or even sales or success. Those teams start to realize that they're getting so much more context about their users when they use their tools, because the company is on Segment. And that means those parts of the company start engaging with whichever team owns Segment.

We definitely see this migration, and as companies get more mature, they're enabling use cases for these different organizations. But they're all on the back of this one centralized core of information, so we aren't looking at individual data silos.

The key here is that you've got a support organization that's probably operating inside their support software, and you've got a marketing organization that's operating inside their marketing tools. Just thinking about those two organizations: All the data that's sitting inside whatever tool you use is kept in one place. [You can see] who wrote in, how long it took us to respond, whether we get an SLA, what products they have. And then you've got another area of marketing where we are looking at campaign effectiveness and all this information about targeting. 

Everybody's trying to put the rest of the data together, but they feel like it's native to them. We see it everywhere. That's what everybody's trying to do. And what generally is happening in that world is everybody's trying to transform the other people's data into their format so that it can live through the lens of the thing they think of as their core.

Segment has a different view on it, which is we just want the data to be everywhere. We have an agnostic view. This data should be able to be consumed by all of these tools, so that folks in Sales who are sitting in the CRM system can see all of the data, the Support people inside Zendesk can see all of the data they want, and the Finance team who's trying to debug a billing problem can see all of the data. They don't want to log in to all these other tools or see data in some other format, right? That's the big difference of using something like Segment. 

The maturity really comes when marketing can use the information and context from Support and Success and use the clickstream context to send a call to action to a customer to move them into a more engaged state with us. And that's the evolution we start to see: These teams start to take data and find that something that happened with Support was actually an opportunity for the folks in Marketing or something that happened in the click path is actually super important for the Support team to understand. 

You get to see inside these black boxes. All these individual monolithic tools and teams that operate on their own – and only trust the data that they've created – start to trust everybody else's data and pull all of that context and richness into their own tools so that the Support folks keep working inside Zendesk or Salesforce Customer Cloud or whatever other tool they're using. They get to keep using their tool, but they get to see all the data everybody else has collected. That's really powerful. 

And you see other companies trying to solve this problem. There are these large suites that go out and buy these other tools, because they're trying to promise the story that they’re a CRM, and they’re launching other capabilities, and they want everybody to operate within their ecosystem, because they want to drive more eyeballs to the CRM. And they want to standardize all data into whatever is the dominant system of record.

The big difference with Segment is that customer maturity is about enabling these other workflows, and we have a strong opinion that the broad ecosystem of tools that are designed for the workflows of an individual team are actually the right place for data to live. The real problem is not, "How do I pull this data from one system into the other and push everybody into one system?" It is, "Give everybody a system they own and want to do their work in, and just give them all the data they need."

Trust and talk to other parts of your organization

Madelyn: What makes seeing inside the black box so difficult for organizations? 

Vishal: The reason this is so hard is because you're trying to get a bunch of parts of an organization that don't actually trust or talk to each other to trust and talk to each other. 

Most companies are still run by executives asking other executives what they think, and those teams have to burrow inside of their own orgs to get context. What you have are a bunch of siloed opinions being aggregated at the top and making major choices. We see that in a lot of organizations we work with, where we spend a lot of time trying to convince a team to pay attention and to trust the data another team has.

You need the directive from your leadership that this is important, but also you need these early successes. We don't see companies jump to the end and do the most advanced use cases off the bat. They go through this natural path of building use cases on top of each other, because each time you complete a foundational use case, you enable the whole of the company to trust the data that everyone is sharing more and more and more.

Eventually, you get to the point where the reports you're able to create and the data you're able to consume at various parts of the company are trusted ubiquitously. Everybody's looking in their own system, but they're seeing the data of everyone else, and they are able to make choices individually based off of the full context the company has about their customers. They make optimized choices with the full dataset versus execs being the only people who have access to the most complete view, because they have the aggregated rollups of all of the different teams. That's the major unlock, right? 

The first version is one team trying to solve the data problem. They suck in the data from everywhere. The next version of it is, “Okay, we've got five silos of teams that are sucking in all of their data, and really, the executive team is the only team that has the full view.” The Holy Grail at the end is when every team has a full view of data down to the individual analyst or product manager. Just imagine a product manager sitting inside your organization who sees support tickets, billing issues, sales interactions, events-stream clicks, and all the data in the data warehouse. They’re able to decide about their particular feature and prioritize, across all of those information points, the most important thing they could do. They don't need the VP of Product or the CEO to come tell them: "Hey, you know what? We've got fire over here. Can you build X?" They already know the fire's here, and they knew weeks ago. It didn't take escalating it all the way to the CEO for the product managers to know there was a problem. That's what we're all trying to get to. 


Madelyn: To wrap up this discussion, what are three takeaways you'd like to leave with the listeners?

Vishal: The first one is this: Right now, you don't know what you don't know. If you're in this world, or if you’re back a couple of steps from what I just talked about, and you're a product manager sitting inside of a team trying to figure out what you should do next, you probably don't even know there's a problem somewhere in Support unless you go talk to that person. And they may be in another country than you are.

Just getting this first level of information up and fluid starts to open everybody's eyes open to the fact that there's a whole bunch of information out there that they’re not even thinking about. If they actually go out and seek it, they can understand more of the context. 

The second thing is trying to get to the promise of digital experiences. You’ve got to go through a path to get to mass customization and mass individualization. There's a lot of grunt work involved in getting all of your information right and surfacing it to the right place so that you can act on it. Lots of people are thinking: “What's our last step? What do you see and do in your engagement with a website or with an app?” But in order for that experience to be really high quality, you've got to do all this work ahead of time to get the pop-up to surface a Support agent who knows everything about what you've been doing until that point. That's the work that lots of people are underestimating. They think it's easy, but that's what we spend all of our time doing. 

The last piece is prioritizing where you spend your time. If you create a really glossy experience on the front end but don't give it any richness or context so that the actual content of what you've delivered is novel and useful, then why wrap it up in a pretty bow and deliver it in a flashy experience while your user is clicking around the site?

How annoying is it when you're trying to shop online, and you're getting these recommendations that don't make sense? When you're in a really context-full experience where you're getting recommendations for things that make sense for what you just clicked through or relates to a need you have, you're really surprised.

But those could be kind of janky, right? The way they're surfaced to you doesn't need to be beautiful, but it needs to have real content. It needs to have the context of what this company knows about you and what you've given to them through your experiences and engagement. Do the hard work first.

Madelyn: Vishal, thank you very much for sharing those takeaways. Your insights around connecting customer data across the organization to create powerful experiences are very valuable for our listeners. Thank you very much for joining us.

Vishal: You're welcome.

The State of Personalization

Our annual look at how attitudes, preferences, and experiences with personalization have evolved over the past year.

Related articles

Madelyn Mullen on August 17th 2020

Simply put, data governance leads to better automation. Listen to this Data Council episode to hear how Arjun Grama grew his customer data wrangling techniques to transform product lines at IBM and raise the bar on growth KPIs at Anheuser-Busch InBev.

Madelyn Mullen on August 17th 2020

Your business growth depends on empowering every team with good data. Introducing the Segment Data Council, a series of interviews with seasoned customer data experts who know how to build bridges across the organization and empower teams.

Madelyn Mullen on August 17th 2020

What does it take for a data driven business case to excite stakeholders across an organization? Tune in to this Data Council episode for an insider perspective from Kurt Williams, Global Director of Customer Products at Anheuser-Busch InBev.

Madelyn Mullen on August 17th 2020

Simply put, data governance leads to better automation. Listen to this Data Council episode to hear how Arjun Grama grew his customer data wrangling techniques to transform product lines at IBM and raise the bar on growth KPIs at Anheuser-Busch InBev.

Madelyn Mullen on August 17th 2020

Your business growth depends on empowering every team with good data. Introducing the Segment Data Council, a series of interviews with seasoned customer data experts who know how to build bridges across the organization and empower teams.

Madelyn Mullen on August 17th 2020

What does it take for a data driven business case to excite stakeholders across an organization? Tune in to this Data Council episode for an insider perspective from Kurt Williams, Global Director of Customer Products at Anheuser-Busch InBev.

Become a data expert.

Get the latest articles on all things data, product, and growth delivered straight to your inbox.