4 Ways to Reduce Costs and Improve Performance with Elasticsearch

What can your team do to get better performance out of Elasticsearch, while at the same time reducing costs?

Hardware evolves quickly. Software is updated regularly. There are almost always new ways to optimize your search engine to improve the user experience. In this article, we’ll investigate 4 ways to improve Elasticsearch performance—and hopefully save you a lot of money along the way. Let’s dive in.

1. Pick newer instance types (time for new hardware)

Hardware continually gets better. As a rule of thumb, you’re always getting a better price per performance with newer hardware. If you’ve used the same hardware for a few years, one of the most straightforward ways to improve your performance and reduce costs is simply to replace your current hardware with modern versions. 

Consider Intel. Every eighteen months or so Intel produces a new chip that is between 5-20% better than the previous version. If your hardware is three to five years old, you can see significant performance improvements and cost reductions by simply swapping out the old hardware for new ones. This requires very little time (and no downtime), if you know what you’re doing. It is a relatively simple way to instantly improve performance and price.

2. Update to the latest version of Elasticsearch

In databases, newer software is almost always faster than previous versions. If you’re on version 1 or 2 of Elasticsearch, you can get up to 7 (the latest version). 

Updating to the latest version means getting the newest bells and whistles, while generally raising the speed, security, and performance of your entire system. There isn’t much more to add here. If you’d like to learn about the latest version of Elasticsearch, read about version 7.16 here.

3. Query tuning

One of Bonsai’s clients recently realized they were storing old data that wasn’t being surfaced regularly. We encouraged the customer to divide the data into two sections: recent and archived. This simple index change turned out to be a major cost saving for the company. The company went from $13,000 to $14,000 per month in server cost down to about $6,000 or $7,000, reducing their costs by half.

Query tuning isn’t a quick win like the other two. This point basically means: become good at using Elasticsearch.

At the heart of every good search engine is a strong index. The process of query tuning means improving the underlying index to make searches faster, more relevant, and more user-friendly. The better your index, the less hard it has to work to populate the best answers. 

This type of tuning begins with the user. For example, if your users only search for content that’s been updated or created in the past ninety days, then you’re working with a highly bloated system by storing nine years of data in a single index. Like in the story above, it would be better to separate old from new data to improve the search experience and performance.

This is a more advanced and specialized optimization tactic. By the time you’re doing deep query tuning, you hopefully already have a good Java engineer on your team. This isn’t a DIY project.

4. Get discounts on AWS

We saved the easiest win on this list for last. Many companies are afraid to sign up for long-term contracts with their preferred cloud storage provider. Usually, this discount is ignored when companies lack the confidence to commit to something for the long term. They’re concerned that things at their organization will change and they’ll end up overpaying for services or getting locked into a contract. More often, these companies simply overpay for the exact same storage services.

If you know that your search needs won’t change significantly in the next 12 months, you can receive an instant discount through Amazon Savings Plan on AWS or Google’s Committed Use program.

The best savings plan? Improve your index regularly

Rather than selecting one or two items from this list, the best way to improve cost and performance is by implementing all four points. By regularly updating your hardware and software, fine-tuning your index, and using your expected spend to garner discounts, you’ll maintain a cost-effective and higher-quality search experience for the long term. 

Elasticsearch is an ever-evolving tool. As technology improves and you learn more about how people use your search engine, you will continue to find new opportunities to save money and create a search experience that’s better for all.