The Tinybird DynamoDB Connector is now GA

The Tinybird DynamoDB Connector is now GA and ready for production use in all Workspaces.

Last month, the DynamoDB Connector hatched into public beta, and quickly broke the record for fastest adoption of a Connector launch yet. We knew DynamoDB was popular, but the reception was beyond what we expected!

“Tinybird's DynamoDB Connector has proved to be easy to set up, reliable and low-latency, while providing cross partition queries, with unlimited indexing, at sub second speeds for our massive dataset. It perfectly complements DynamoDB and is saving us money by moving optimizations and large queries to Tinybird.”
- Carl Thorner, CTO and Co-founder of Zembula

Within one week, Tinybird users connected over 750 DynamoDB tables, captured 1.3TB worth of changes and published API Endpoints supporting >5 million requests.

We’ve gathered feedback, crunched numbers, and built a raft of improvements in the month since. From bug fixes, to improved observability, and performance enhancements, the Connector has improved in every way possible, and is reliably powering production use cases at scale for over 25 customers.

What is the DynamoDB Connector?

DynamoDB is a powerful OLTP database for fast, fresh and frequent access to transactional data. By pairing DynamoDB with Tinybird, applications can serve fast, fresh and frequent access to analytical data without your teams having to build and maintain complex, external ETLs. The Connector is fully managed, and maintains a stateful copy of your DynamoDB table in Tinybird in real-time. Now, you can use DynamoDB to support transactional workloads, and offload complex aggregating and filtering queries to Tinybird. Best part? You can publish every query in Tinybird as a dynamic, scalable REST API, making it trivial to integrate into your application.

Read more about how the DynamoDB Connector works.

Changes Since Public Beta

Since the public beta release, our customers have provided us with feedback to make this connector even better. Here are some highlights: 

• Better scalability. We completely re-engineered how Tinybird handles DynamoDB Streams shards to support significantly higher scale change streams. 
• Improved reliability. For historical loads, we made improvements when importing massive DynamoDB tables and for high-throughput change streams, we now have better handling of shard partition changes.
• Support for `New and old` stream types. We've enabled more complex change comparison logic in Pipes.
• Better error reporting. It is now easier to know why the connector isn’t working for common cases like missing JSONPaths in Data Source schemas or misconfigured AWS permissions.
• Support for Array(Nullable(Date)) type and nullable values in Map types Map(*,Nullable()). In cases where there are nullable subtypes, we’ve fixed some bugs where it was causing failures.

Get Started with Tinybird

If you’re working with DynamoDB and are looking to augment your transactional use cases with real-time, user-facing analytics, try pairing it with Tinybird. You can aggregate DynamoDB data and run complex queries while not compromising on concurrency, latency and speed. Sign up here and try it today.

For more details about the DynamoDB Connector, refer to the documentation.

If you have any questions, please join our Slack Community.