Slow search engine? CPU is probably your biggest bottleneck

Your website visitors don’t have much patience for slow load speeds. Most users will leave if they type something into your search engine and have to wait too long for results. Which can result in a poor user experience for site visitors and potentially losing customers for your business.

This highlights a few questions:

• What causes a slow search engine experience?
• What are your best tactics for improving search engine speed?

From our experience working with thousands of search deployments at Bonsai, we believe search engine speed bottlenecks almost always come down to just one thing: Your CPU speed. In this article, we’ll cover why CPU speed is the most common modern culprit, as well as how to improve your speed if you determine that this is indeed your search vulnerability.

Let’s dive in.

How fast does your search engine need to be?

None of the work we do at Bonsai matters if search is slow for the end user. In other words improving search speed can be the number one way to improve user experience — even more important than relevancy. 

Because even if your search query doesn’t pull up relevant searches, you can at least run multiple searches on a fast search engine and receive the results you’re looking for. On the other hand, if results are relevant but slow, a user might leave the search tool anyway because it’s a bad user experience.

So, how quickly does your search engine need to retrieve information to keep searchers engaged and happy?

Your target should be under 100 milliseconds. For the search engine user, this feels like they’re receiving their information instantaneously, which improves the general appeal of your entire website. Approaching 1 second, people begin to notice the wait. If search results take longer than one second, your system had better be doing a lot of other valuable work to keep readers on the page — because when it doesn’t feel instantaneous, users have very little patience for standard value. 

Of course, you can go much faster than 100 milliseconds. On the Bonsai platform, 99% of search queries respond in under 100 milliseconds, with about 50% responding below 10ms.

Why CPU is the most likely culprit slowing down your information processing

In our experience, the most common bottleneck that’ll slow down an otherwise well-tuned search experience is CPU. This is the case for two primary reasons.

1. Bottleneck in the writing of data: To build a better search experience, you need to build a better index. And to do that, you need a lot of CPU. Indexing commonly involves splitting, moving, and parsing massive strings of natural language text. That requires a lot of CPU and other system resources as a tradeoff to make searches faster or more complex.
2. Complexity in your searches: Your CPU is used to calculate scores, run comparisons, and perform sorting — things a lot of people don’t automatically think of when they imagine search engines. That’s why the tension between the computation requirements of building the index and the computational requirements of running the search is a common factor in slowing down your CPU.

Some of the highest level and most expensive search engine features in Elasticsearch require a lot from your CPU. So when you think about it in that light, it makes sense that these big responsibilities often cause some of the biggest slowdowns for the user. 

So, how do you deal with it?

What can you do to improve search engine speeds significantly?

Reducing the speed of your search engine often comes down to optimizing your index to put less toll on your CPU. For example, you can use cleaner indexes with less redundancy. If there’s something not delivering value in the interface, consider getting rid of it.

The goal is to require less work from the CPU.

One of the best methods to “do less work” with your CPU is to run fewer searches that you don’t actually use. For example, if you’re doing an auto-completing search, as the user types, they’re seeing a gradual updating list of potential responses to their query. On the surface, this is a nice user interface. People get real time feedback about what they’re typing.

The problem arises when auto-complete does unnecessary work. In other words, autocomplete can be taken to impractical extremes.

If every key press generates a search, for example, that puts an unnecessary toll on the CPU in the form of thrown-out searches. Think about it: People aren’t looking at the auto-complete suggestions after every key press.

Instead, you can auto-complete searches when the user pauses typing. This can be a pause to think about the right word, which is the perfect time to suggest the most likely searches based on your index, anyway. The user is looking for an answer at the same time your autocomplete is handing options to them. And by waiting for these pauses, you require less CPU, which means a faster (and better) user experience all around.

A faster search experience is the best search experience

As we discussed earlier in the article, speed is often more important than relevancy when it comes to providing a good search user experience. If you can provide results at 100 milliseconds or faster, web visitors will enjoy an experience that feels instantaneous. 

A slow search experience, on the other hand, can result in higher website bounce rates, fewer sales for your business, and a negative user experience. All of these cost your business money.

If you’re experiencing slow Elasticsearch speeds, it’s most often a result of CPU, but it’s good to run a diagnostic test like the ones mentioned above, just to be sure.

Have additional questions? Want to learn more about Elasticsearch? Schedule a consultation with a member of our team.