This error indicates that the request body to Elasticsearch exceeded the limits of the Bonsai proxy layer. This can be caused by a few things:
- A request larger than 40MB. Elasticsearch's Query DSL can be fairly verbose JSON, particularly when queries are complex. The 40MB cap is meant to be a safety mechanism to prevent runaway queries from overwhelming the routing layer, while still being an order of magnitude higher than 99.9% of request bodies.
- Indexing too many documents at once. The Elasticsearch Bulk API allows applications to index groups of documents in a single request. Sending a single batch of millions of documents could easily trigger the HTTP 413 message.
- Lots of request headers. Metadata about a request can be passed to Elasticsearch in the form of request headers. Bonsai allows up to 16KB for request headers; this should be enough for whatever CORS and content-type specification needs to occur. Note that the TLS and authentication headers in the request are not counted towards this limit.
- Indexing large files. When Elasticsearch indexes a rich text file like a PDF or Word document, it converts the file into a Base64 string to compress it for transit. Still, it's possible that this string is longer than 40MB, which could trigger the HTTP 413 error.
If you're seeing this error, check that your queries are sane and not 40MB of flattened JSON. Ensure you're not explicitly sending lots of headers to your cluster.
If you're seeing this message during bulk indexing, then decrease your batch sizes by half and try again. Repeat until you can reindex without receiving an HTTP 413.
Finally, if it is indeed a large file causing the problem, then the odds are good that metadata and media in the file are resulting in its huge size. You may need to use a some file editing tool to remove the media (images, movies, sounds) and possibly the metadata from the file and then try again. If the files are user-submitted, consider capping the file size your users are able to upload.
Customers who are unable to change their application to accommodate smaller request bodies or payloads should reach out to us. We we lift this limit for customers on Business and Enterprise plans, subject to some caveats on performance.
