---
title: "Good architecture isn’t always designed upfront"
author: "Luis César"
date: 2026-08-24
canonical_url: https://bonsai.io/blog/good-architecture-isnt-always-designed-upfront/
license: CC BY-SA 4.0
license_url: https://creativecommons.org/licenses/by-sa/4.0/
copyright: Bonsai.io, 2026
---

Good architecture isn’t always designed. It’s discovered.

Quick definition before I get into it: I think of “good architecture” as an architecture that satisfies the system’s performance/scalability/availability constraints at an acceptable cost.

Over the past few weeks I worked on a feature called [Query Logs](https://bonsai.io/blog/announcing-query-logs/) with Nick, the CEO at Bonsai. Going into the project I expected to learn about mostly scalability, since this is a feature that needs to process large amounts of data.

And while I did, I ended up learning something bigger: how to design software in a different way.

## The problem

At a high level, the Query Logs feature answers a simple question: What are your customers searching for?

If your product has a search bar (it most likely has), there is no better signal for understanding your customers than the queries they type in.

The challenge is scale. Some of our customers generate millions of search queries every day. Across thousands of customers, we’re processing on the order of billions of queries daily.

Collecting that data wasn’t the hard part. Processing it efficiently was. That became the central challenge of this project.

## We built the wrong architecture

One thing I enjoyed about this project is that we didn’t spend weeks trying to design the perfect architecture before writing code. This is how we work at Bonsai: build something quickly, learn from it, then iterate.

The prototype evolved into a fairly sophisticated pipeline. At the time, it seemed like a perfectly reasonable direction. Until reality hit.

Large customers would generate so many query logs that our workers’ memory would simply blow up while processing them.

To work around that, we introduced limits on how many query logs we’d process per customer.

Technically, that fixed the memory problem. Practically, it broke the product. It was just a very bad bandaid.

If a customer generates millions of queries and we only provide insights about a tiny fraction of them, what’s the value?

So we kept iterating. We made some improvements here and there that help with memory. But the interesting part was about to happen.

While working on improvements, we also started building a backfill tool to import historical data before releasing the feature. Since it was a one-time utility, we bypassed most of the existing pipeline and designed a stripped down version of it. It was simply the easiest thing that could work.

## We benchmarked it

We benchmarked the backfilling tool. It was important, since we were about to backfill a massive amount of data. But before discussing results, an aside:

One of my favorite parts of this project was how we used AI during the benchmarking process.

Instead of writing benchmarking infrastructure, we asked Claude to do the tedious stuff for us.

We described what we wanted to measure, which scenarios to test, and Claude handled the implementation. That completely changed the dynamic of the work.

When pairing with someone much more experienced than me, like Nick, a good chunk of the conversation used to revolve around syntax, APIs, libraries, and implementation details. Because that’s what you needed to discuss when manually writing code.

This time, we didn’t have to.

Nick and I spent almost the entire time discussing the important stuff:

-   What are we measuring?
-   Why are we measuring it?
-   What hypothesis are we trying to validate?

As someone learning from a more experienced engineer, that was incredibly valuable. Instead of following code, I could follow his thinking.

Now back to the benchmark results.

The original pipeline processed roughly 600 events per second. The simplified backfill approach processed roughly 90,000 events per second. About 150× faster.

So yeah… that number made us question the entire thing. You just can’t ignore those gains.

![Bar chart of events processed per second. The original pipeline, the one we designed, reaches 600. The backfill pipeline, the one we discovered, reaches 90,000 — 150 times faster.](https://bonsai.io/processed_images/pipeline-benchmark.c1b9a36b757df4da.webp)

## We created the environment for discovery

The backfill architecture (the thing we originally built as a supporting tool) suddenly looked like it could be the real thing. After a few tweaks, we promoted it to the main production pipeline behind Query Logs, and discarded the initial architecture we’d built.

At first it felt like serendipity. But after reflecting while writing this post, it wasn’t.

We created the conditions for it: We built prototypes. We measured them. We questioned our assumptions. And we were willing to change what needed to be changed.

Because we did those things, we came to realize that our original design wasn’t the best solution. Otherwise, how could we have arrived at the final architecture?

![Two panels. What we designed: a sophisticated pipeline of interconnected stages, where workers run out of memory and per-customer limits act as a band-aid — it fixed the memory and broke the product. What we discovered: one pass end to end, from events to storage to insights, at about 90,000 events per second.](https://bonsai.io/processed_images/discovered-architecture.ab960b6a7844145e.webp)

## What I’ll take into every project

Before this project, I thought architecture was mostly about coming up with the right design upfront.

That’s still important. But now I know you also need a process that lets good ideas emerge.

We weren’t trying to prove our design was correct. We were trying to learn.

-   Sometimes that meant building a prototype.
-   Sometimes it meant benchmarking.
-   Sometimes it meant throwing away code we’d already written. (way easier with AI now)

If I had to summarize everything I learned into three lessons, they’d be these:

-   Create an environment where discovery is possible. If you don’t prototype, measure, or experiment, you’ll never discover a better solution than the one you imagined on day one.
-   Don’t become emotionally attached to your architecture. If the evidence points to a simpler solution, be willing to throw away your own work. (and I’ll guarantee it will happen… more than once)
-   Benchmark everything that matters. None of this would’ve happened without measuring.

Good architecture isn’t always something you design upfront. Sometimes, it’s something you discover.

_Copyright ©️ Bonsai.io, 2026 · By Luis César · Originally published at https://bonsai.io/blog/good-architecture-isnt-always-designed-upfront/ · CC BY-SA 4.0_