As experienced search engineers, we often see problems that require a philosophical approach. I've assembled some kōans that reflect this. What is a kōan you may ask? Here is a concise introduction from Wikipedia:
“
The popular Western understanding sees kōan as referring to an unanswerable question or a meaningless or absurd statement. However, in Zen practice, a kōan is not meaningless, and not a riddle or a puzzle. Teachers do expect students to present an appropriate response when asked about a kōan. According to Hori, a central theme of many kōan is the 'identity of opposites'
”
These are all inspired by actual experiences I've run into in my 15 years working with search engines, relevance, and AI. While content on our contemporary internet is often quickly consumed, I wrote these as pieces to meditate on individually. You may find one or two that stick with you, finding its way into long term contemplation.
The Kōans:
A manager and engineer were reviewing search quality.
"When I search for this query, the results are irrelevant" said the manager.
The engineer reviewed the data for the past year and replied "but no customer has yet searched that query"
A search reliability engineer noticed that many new queries took over a minute to return results from the engine, overwhelming the platform.
The SRE asked the product owner "do you need to run these queries?"
The product owner replied, "we are testing with the customer."
A relevance engineer asked two subject matter experts "for this query, is this document relevant or not?"
One expert replied "yes", the other expert replied "no".
An AI engineer was testing prompts for summary generation.
The metric was at a decent baseline with room for improvement.
The engineer added a single comma to the 800 token prompt template, and the metric improved.
The engineer then added another comma, and the metric regressed.
A search engineer noticed that a 32GB Java heap was approaching saturation.
The engineer increased the heap to 48GB, and the heap saturated.
A search team began a long migration to a new engine.
Along the way, an executive asked if the customers will be happier.
The search team replied "we don’t know, but we will be".
A subject matter expert pointed out that during development of the RAG project, the results became obscured along with their meticulously researched content.
The product director countered that the customers liked the summaries.
Two engineers were tasked to separately prototype with new search engines and each prepare a demo.
They came back a month later, showing their work.
"Which do you like better?" they asked the team.
A loyal customer used the search one day, and noticed irrelevant results had crept into many searches.
They opened a support ticket. Soon later they received a reply: "We added several new content sources to our corpus".
A full stack engineer on a small team noticed one day that the company website search was not working.
There was no indication for how long it was broken, and nobody had told them.
A search team completed their upgrade from lexical to vector search after great effort, with relevance significantly improved.
The following day, they received a support ticket from a major customer: "keywords in result snippets are not highlighted anymore".
Over happy hour on a Friday, an engineer realized an elegant solution to a complex search problem.
They returned to work on Monday to a full backlog of unrelated support tickets, the solution forgotten by Wednesday.
A search engineer was working late into the evening on a particularly hard relevance problem, when the engineer's child entered the room.
"Why are you working so late?" the child asked.
The engineer replied "well, the right documents are not returning for search and I need to fix it by tomorrow".
To which the child replied, "can’t you just tell the computer to do it?".
Ready to take a closer look at Bonsai?
Find out if Bonsai is a good fit for you in just 15 minutes.
Learn how a managed service works and why it’s valuable to dev teams
You won’t be pressured or used in any manipulative sales tactics
We’ll get a deep understanding of your current tech stack and needs
Get all the information you need to decide to continue exploring Bonsai services
