Introducing Delivery Overview: Gain full visibility into your data pipeline

What questions do you have before you purchase a car?

Maybe you’re curious about when it was made. How many prior owners has it had? Does it have any accident history? Before you make a decision, you seek data. 

The curiosity around your data pipeline isn’t dissimilar. You may have questions like, what is the extent of the drop/spike of events in this connection? Which parts of the pipeline are responsible for the drop/spike? What is the real reason for the drop/spike? Can I see samples of the impacted events? Can I take action on altering the outcome of discarded events?

When it comes to your data pipeline, it is important to understand how the outcome occurred as you look to make a decision based on that outcome.

With a lack of data, you might be left feeling like Guy Pierce’s character in 2000’s Memento, wondering how the heck your data ended up the way it did.

With our newest feature, Delivery Overview, you can answer all of these questions (and more) with a full picture of the data you send into Segment. Delivery Overview gives you the insight to understand exactly how data is processed in your CDP with complete visibility of every stage of event delivery to your Destinations.

Historically, our Event Delivery feature helped customers understand questions like how many events made it to the intended destination, or what errors occurred at the destination that prevented events from being ingested properly. 

Delivery Overview expands on this functionality by enabling you to transparently inspect each step of Segment’s processing mechanisms that our data passes through. The new pipeline view is central to this new experience, and provides insights into each “stage” data navigates en route to a destination:

• Successfully received

  • Events that Segment ingested from your source

• Failed on ingest

  • Events that were received by Segment, but were dropped due to internal data validation rules or issues with Source Functions

• Filtered at source

  • Events that were discarded due to schema settings or Protocols tracking plans

• Filtered at destination

  • Events discarded due to Destination Filters, Filtering in the Integrations object by the customer, per source schema integration filters, and mapping rules established in an Actions integration

• Failed Delivery

  • Events that have been discarded due to errors at the destination or unmet destination requirements or issues with Destination Functions

• Successful delivery

  • Events that were successfully delivered to the destination

For each one of these areas, we can view a breakdown of the events that were impacted:

Because you have complete visibility into each step of event ingress, processing, and egress from Segment’s ecosystem, Delivery Overview enables self service troubleshooting by making it easier to diagnose and debug any issues should event delivery fail.

Taking our new debugging capabilities to the next level, you can keep your eyes peeled for things like the change rate of any given step, compared to its preceding stats from the last prior time period in question (ie; is this drastically more traffic than I usually receive from this source?). You can also explore samples of the payloads for the discarded events in question.

From our end, we continuously strive to provide remedies to any discard that a user can have control over. When you encounter a discarded event today, you’ll either be routed to the correct documentation or receive a human-readable description that addresses what happened. In the future, we’ll enhance those remedies by directly linking you to the proper control/setting to address the error whenever possible.

The beta launch for Delivery Overview will fully support classic Streaming Destinations along with Source and Destination Functions. In the near future, Delivery Overview will also support Actions, Reverse ETL, and Unify & Engage connections. We’ll also be working to improve alerting features within Segment to best leverage the information that Delivery Overview exposes to the user.

For more information about Delivery Overview, please visit our documentation. We’re excited to bring you this game-changing feature that gives you the confidence that the right data gets to the right destinations with a comprehensive, transparent, and debuggable event delivery pipeline.