We've improved notifications for ingestion issues

To do anything valuable with Tinybird, you first have to get data into it, whether from a third-party source like Kafka or S3, or using the Tinybird Events API. Because data ingestion is the analytical foundation of everything we offer, we’re hyper-focused on making sure it’s a reliable, observable process.

While you have access to all of the raw observability data through Tinybird’s Service Data Sources, we realized a simple feature might help improve basic observability...

You now get an email for data ingestion issues

If ingestion fails completely, resulting in data loss you’ll get an email like the one below letting you know about it.

Likewise, you’ll get a similar email if data ends up in quarantine:

If you want to manage who gets these emails, you can do so in the Member configuration of the Workspace:

Add your own observability layer with datasources_ops_log

Every Tinybird workspace includes the Service Data Source datasources_ops_log with information that you can use to build your own observability functions.

The email alerts you now get are built on top of this datasource, so feel free to expand these alerts into your own observability platforms whether that’s email, Slack, Datadog, or… anything.

Related: How to monitor Tinybird with Datadog using vector.dev

For example, here’s a simple snippet of SQL to get, for the workspace, all append events in the last week that sent rows to quarantine:

As always, you can easily publish these results as an Endpoint that your observability services can call.

Let us know if this helps!

If you have any questions about these notifications, or ideas about how we can improve observability features, please let us know in Slack.