Text fields are not optimised for operations that require per-document field data

Having switched from Elasticsearch to Opensearch, my application now fails to run a simple query with:

"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [status] in order to load field data by uninverting the inverted index. Note that this can use significant memory."

There's a question concerning the same error at Searchkick / Elasticsearch Error: Please use a keyword field instead. Alternatively, set fielddata=true on [name], but there the problem was only affecting tests and I only get the problem (so far) in development mode.

Here's the query being run:

::Record.search(q ? q : "*", where: where_clause, fields: fields, match: :word_middle, per_page: max_per_page(per_page) || 30, page: page || 1, order: sort_clause, aggs: aggs, misspellings: {below: 5} 

If I take out aggs then the search is fine, but they're essential for the application. Removing :status from the list of aggregation fields causes the error to name the next field in the array as the problem. So, I presumably need to specify the correct type for each field used in aggregations. But how?

The Searchkick docs suggest this example under "Advanced Mapping" ():

class Product < ApplicationRecord searchkick mappings: { properties: { name: {type: "keyword"} } } end 

So, I tried this:

# in models/Record.rb mapping_properties = {} aggregation_fields.each do |af| mapping_properties[af] = { type: 'keyword' } end searchkick mappings: { properties: mapping_properties } 

But, the same problem persists. I also tried something similar to that shown in the linked post, e.g.

mappings: { properties: { name: { type: "text", fielddata: true, fields: { keyword: { type: "keyword" } } } } } 

...but similarly without luck.

Can anyone suggest how this might be fixed?

3 Answers

The immediate issue was dealt with by changing all the fields used for aggregations, so rather than:

aggs = %w(field1 field2 field3 ...) 

...in the above search query. I used:

aggs = %w(field1.keyword field2.keyword field3.keyword ...) 

If you have a Tomcat running with Elastic Search while you delete your index it will trigger this error.

Stop your tomcat, again delete your index and you're good to go.

Hope this helps someone

This was not related to using Opensearch. Instead, the method of indexing used caused the indexes to be generated incorrectly.

See: Searchkick memory leak