Data Lakes Sync History and Health

The Segment Data Lakes sync history and health tabs generate real-time information about data syncs so you can monitor the health and performance of your data lakes. These tools provide monitoring and debugging capabilities within the Data Lakes UI, so you can identify and proactively address data sync or data pipeline failures.

Sync history

The Sync History table shows detailed information about the latest 100 syncs to the data lake. The table includes the following fields:

Selecting a row in the Sync History table opens a sidebar showing the number of rows from each collection that synced.

To access the Sync History page from the Segment app, open the My Destinations page and select the data lake. On the data lakes Settings page, select the Sync History tab.

Health

The health tab provides an overview of the rows that synced to your data lake both today and each day for the last 30 days.

The bar chart, ‘Daily Synced Rows,’ shows an overview of the rows synced for each of the last 30 days. Hovering over a date shows the number of rows that were synced for that day. Selecting a date from the bar chart opens the Daily Row Volume table, which provides a breakdown of which collections synced, how many rows from each collection synced, and the percentage of all synced rows from each collection.

The Daily Row Volume table contains the following information:

• Collections: The name of each collection of properties synced to the data lake
• Rows: The number of rows synced from each collection
• % of Total: The percentage of the total number of rows synced that each collection represents

Above the Daily Row Volume table is an overview of the total syncs for the current day, showing the number of rows synced, the number of collections that synced, and the current date.

To access the Sync history page from the Segment app, open the My Destinations page and select the data lake. On the data lakes settings page, select the Health tab.

Data Lakes reports FAQ

How long is a data point available?

The health tab shows an aggregate view of the last 30 days worth of data, while the sync history retains the last 100 syncs.

How do sync history and health compare?

The sync history feature shows detailed information about the most recent 100 syncs to a data lake, while the health tab shows just the number of rows synced to the data lake over the last 30 days.

What timezone is the time and date information in?

All dates and times on the sync history and health pages are in the user’s local time.

When does the data update?

The sync data for both reports updates in real time.

When do syncs occur?

Syncs occur approximately every two hours. Users cannot choose how frequently the data lake syncs.