Comparison of Elasticsearch Ruby Gems

If you’ve ever wanted to supercharge your app’s search capabilities, there’s no better tool to use than Elasticsearch. Integrating Elasticsearch into a Ruby on Rails app is a fairly straightforward process once you’ve selected a framework.

The framework you choose will act as an interface between your app and Elasticsearch. This choice should not be made lightly; while all frameworks can generally perform the same tasks, they have dramatically different approaches for doing so.

In this post, we’re going to explore the differences between the three most popular Elasticsearch frameworks for Rails apps: Searchkick, Elasticsearch Rails, and Chewy.

What’s the Difference?

Under the hood, all three of these gems are leveraging the Elasticsearch Ruby gem. Elasticsearch Rails adds some Rails-specific niceties like ActiveRecord integration and convenience methods like search and index to various models. The Searchkick and Chewy gems do more or less the same, albeit in slightly different ways.

So what are the key differences to developers?

Data Modeling

There are a number of ways to organize your data in a cluster. The Index-per-Application paradigm is where all of your application’s data is stored in a single index. Queries then use fields to filter and cache results as needed. Chewy uses this paradigm.

Alternatively, the Index-per-Model paradigm has data logically separated by model into its own Elasticsearch index. Elasticsearch Rails and Searchkick use this paradigm.

So which approach is “best?” Well, suppose you have a Rails application and want to store data about users, posts and comments. This table compares how the each paradigm handles the data separation:

<table>
<thead>
<tr>
<th>Paradigm</th>
<th>Index-Per-Application</th>
<th>Index-Per-Model</th>
</tr>
</thead>
<tbody>
<tr>
<td>Gem</td>
<td>Chewy</td>
<td>Elasticsearch Rails, Searchkick</td>
</tr>
<tr>
<td>Models</td>
<td>User, Post, and Comment</td>
<td>User, Post, and Comment</td>
</tr>
<tr>
<td>Elasticsearch Indices</td>
<td>1</td>
<td>3</td>
</tr>
<tr>
<td>Mapping</td>
<td>One mapping for all models</td>
<td>Each model has its own mapping</td>
</tr>
<tr>
<td>Settings</td>
<td>Master settings for all models</td>
<td>Each model can have its own settings</td>
</tr>
<tr>
<td>Queries (default)</td>
<td>Performed against all documents. Searching only documents of a specific type possible with filters.</td>
<td>Performed against only documents associated with its mapped model. Searching against all documents can be done by explicitly naming which indices to search.</td>
</tr>
<tr>
<td>Pros</td>
<td>Simplifies the indexed models by having the configurations mainly in one file separate from the models</td>
<td>Logic and settings can be tuned on a per-index basis. No cross-contamination of relevancy scoring caused by other indices</td>
</tr>
<tr>
<td>Cons</td>
<td>A single mapping can have field name collisions and lead to sparse documents, which has performance implications. Relevancy scores for a type could be influenced by data in a different type</td>
<td>Can be complicated to manage when to perform a search against one index or several</td>
</tr>
<tr>
<td>Good Fit For…</td>
<td>Applications where Elasticsearch needs to be able to search from many models at once. Applications where related objects need to be denormalized (updates to one object affect one or more related objects). Examples: Media catalog with different types of media (TV, movies, music, books); Blog with tags, where changes to a tag should be reflected on all entries with the tag.</td>
<td>Applications where Elasticsearch really only needs to cover one model, or where search is logically separated. Applications where Elasticsearch is used to query time-series data (logs, metrics, etc). Examples: Online store with many products; A restaurant recommendation app, where searching for users is a different interface from searching for nearby restaurants.</td>
</tr>
</tbody>
</table>

‍

Coding

Let’s explore the code of the previously mentioned gem paradigms with their User, Post, and Comment models. I will omit creating routes, views, and getting data into Elasticsearch.

An example setup for Elasticsearch Rails

‍

Model-observing code:

<pre><code>require 'elasticsearch/model'
‍
class User < ApplicationRecord
 include Elasticsearch::Model
 include Elasticsearch::Model::Callbacks
 settings index: { number_of_shards: 1 }
end

class Post < ApplicationRecord
 include Elasticsearch::Model
 include Elasticsearch::Model::Callbacks
 settings index: { number_of_shards: 1 }
end

class Comment < ApplicationRecord
 include Elasticsearch::Model
 include Elasticsearch::Model::Callbacks
 settings index: { number_of_shards: 1 }
end
</pre></code>

A Search Controller contains:

<pre><code>class SearchController< ApplicationController
 def search_all
  # This will search all models that have `include Elasticsearch::Model`
  @results = Elasticsearch::Model.search(params[:q]).records
 end

 def search_users
  @users = User.search(params[:q]).records
 end

 def search_posts
  @posts = Post.search(params[:q]).records
 end

  def search_comments
  @comments = Comment.search(params[:q]).records
 end
end

</pre></code>

An example setup for Searchkick

Model-observing code:

<pre><code>class User < ApplicationRecord
 searchkick
end

class Post < ApplicationRecord
 searchkick
end

class Comment < ApplicationRecord
 searchkick
end

A Search Controller contains:

<pre><code>class SearchController< ApplicationController
 def search_all
# This will search all models that include `searchkick`
  @results = Searchkick.search(params[:q]), models: [User, Post, Comment]
 end

 def search_users
  @users = User.search(params[:q])
 end

 def search_posts
  @posts = Post.search(params[:q])
 end

  def search_comments
  @comments = Comment.search(params[:q])
 end
end

</pre></code>

‍

An example setup for Chewy

Model-observing code:

<pre><code>class User < ApplicationRecord
  update_index('users') { self }
end

class Post < ApplicationRecord
  update_index('users') { users }
end

class Comment < ApplicationRecord
  update_index('users') { users }
end

</pre></code>

‍

Index definition created in app/chewy/users_index.rb:

<pre><code>class UsersIndex < Chewy::Index
  index_scope User.active.includes(:post, :comment)
  field :first_name, :last_name
  field :post do
     field :title
     field :description
  field :comment do
     field :description
end

</pre></code>

‍

Users Controller contains:

<pre><code>def search
 @results = UsersIndex.query(query_string: { fields: [:first_name, :last_name, ...], query:params[:q], default_operator: 'and' })
end

</pre></code>

‍

Features Comparison

<table>
<thead>
<tr>
<th align="left">Features</th>
<th align="center">Elasticsearch Rails</th>
<th align="center">Searchkick</th>
<th align="center">Chewy</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">Reindex without downtime</td>
<td align="center"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
</tr>
<tr>
<td align="left">Stemming</td>
<td align="center"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"></td>
</tr>
<tr>
<td align="left">Special characters like ñ</td>
<td align="center"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"></td>
</tr>
<tr>
<td align="left">Catch Misspellings</td>
<td align="center"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"></td>
</tr>
<tr>
<td align="left">Personalize results for each user</td>
<td align="center"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"></td>
</tr>
<tr>
<td align="left">Autocomplete</td>
<td align="center"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"></td>
</tr>
<tr>
<td align="left">“Did you mean” suggestions</td>
<td align="center"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"></td>
</tr>
<tr>
<td align="left">ActiveRecord support</td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
</tr>
<tr>
<td align="left">Mongoid support</td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"></td>
</tr>
<tr>
<td align="left">Supports ES 7.x</td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
</tr>
<tr>
<td align="left">Supports ES 6.x</td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
</tr>
<tr>
<td align="left">Supports ES 5.x</td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
</tr>
<tr>
<td align="left">Supports ES 2.x</td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"></td>
</tr>
<tr>
<td align="left">Supports ES 1.x</td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"></td>
</tr>
<tr>
<td align="left">Provides Rake tasks</td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
<td align="center"><img draggable="false" role="img" class="emoji" alt="✅" src="https://s.w.org/images/core/emoji/14.0.0/svg/2705.svg"></td>
</tr>
<tr>
<td align="left">Licensing</td>
<td align="center">Apache 2 License; Starting in 2021, <a href="https://www.elastic.co/blog/elastic-license-v2" target="_blank" rel="nofollow noopener">Server Side Public License (SSPL)</a></td>
<td align="center">MIT License</td>
<td align="center">MIT License</td>
</tr>
</tbody>
</table>

‍

Which one should you choose?

Searchkick is easy to set up and ideal if you don’t need fine grained control over how your app searches. It has many out-of-the-box features to get searching quickly. Stemming, spell checking, personalized search results, autocomplete, and “Did you mean” suggestions are some of the features you can expect. True, you can build these yourself with Elasticsearch-Rails and Chewy, but it will take some extra work (and time to fix bugs!).

Chewy logically separates its index from model code, resulting in leaner models. Chewy is built for some niche use cases, so it’s not ideal for everyone.

If you are already familiar with the inner workings of Elasticsearch and want the most hands on approach to managing search features (like stemming, spell checking, reindexing, autocomplete, and more, then the Elasticsearch Rails gem may be for you. It allows you to manage the details of how your search works, and implement your own opinions on the process.

What are your experiences with these gems? How did they compare with different app use cases? Share your thoughts on Twitter: @bonsaisearch.