Engineering Excellence How to safely cancel a database query Cancelling a query from a UI client is more nuanced than it might seem. Here's how we implemented safe KILL QUERY operations in Tinybird.
Engineering Excellence Behind the scenes of Tinybird's big frontend refactor We recently refactored the Tinybird frontend. Take a look behind the scenes at how we started, the choices we made, and how we arrived at a faster and simpler app deployment.
Engineering Excellence We rebuilt our docs from scratch. It was worth it. Building modern docs begins with a modern tech stack. Here’s how we rebuilt our documentation and what it enables us to do for our customers.
Engineering Excellence Iterating terabyte-sized ClickHouse®️ tables in production ClickHouse schema migrations can be challenging even on batch systems. But when you're streaming 100s of MB/s, it's a whole different beast. Here's how we make schema changes on a large ClickHouse table deployed across many clusters while streaming… without missing a bit.
Engineering Excellence Resolving a year-long ClickHouse®️ lock contention Here's how we fixed a problem that had stumped us for over a year and saw a 5x performance improvement in the process.
Engineering Excellence Killing the ProcessPoolExecutor A story about the Python GIL, threading, and fixing code that already works.
Engineering Excellence Adding JOIN support for parallel replicas on ClickHouse®️ We recently introduced a pull request to ClickHouse that enables simple JOIN support for parallel replicas on ClickHouse. The solution may be simple and naive, but the ceiling for performance on distributed queries just got WAY higher.
Engineering Excellence 7 strategies we're using to reduce cloud infrastructure costs in 2023 Cloud infrastructure costs are going up for everyone, but expectations to perform haven't receded. Here are 7 strategies that we use at Tinybird to keep our costs low as we innovate new ways to help our customers achieve more with less.
Engineering Excellence Building a faster CI pipeline with Turborepo and pnpm We made 3 big changes that have shortened our frontend CI execution time despite a massive increase in code size and test coverage.
Engineering Excellence How we cut our CI pipeline execution time in half Our GitLab CI pipeline was slow and flaky. Here’s how we used telemetry data, real-time analytics, and smart k8s configuration to fix it.
Engineering Excellence Horizontally scaling Kafka consumers with rendezvous hashing How we used rendezvous hashing to horizontally scale Kafka consumers to support hundreds of concurrent topics with fewer connections, thus lowering our infrastructure costs.
Engineering Excellence Splitting CSV files at 3GB/s Splitting CSV files is a must when dealing with large, potentially larger than RAM, files. But how fast can it be?
Engineering Excellence Performance and Kafka compression The **unmodified** support message we sent to one of our clients outlining potential performance gains through Kafka compression
Engineering Excellence How we processed 12 trillion rows during Black Friday In this post we explain the data architecture, infrastructure and how we scale our real-time analytics service with ClickHouse.
Engineering Excellence How We Handle Technical Incidents and Service Disruptions Part of providing an outstanding level of service is to react quickly and professionally when things go wrong. This is an account of a recent service disruption and how we handled it.