By default in Elasticsearch, if a shard is not searched for 30 seconds it stops refreshing. The next call triggers refresh and refresh runs before the call.

When search queries are paused for a certain period of time, this causes Elasticsearch search queries to take too long time to run for the first time.

To prevent this, the following two settings can be changed:

1) Changing the value that specifies how long a shard cannot receive calls or requests until the call is considered idle.

index.search.idle.after=30sn
CODE
  • When this value is increased, the time required to pass between two queries for the refresh operation increases.
  • It just postpones this problem for a little longer term.
  • If it is a system that is frequently questioned, it may be considered.


2) Changing the value that specifies how often the refresh operation will be performed, which makes the latest changes made in the index visible for search.

index.refresh_interval=1sn
CODE
  • When this value is increased, automatic refresh is made at certain intervals.
  • This results in query results that may contain outdated data for a certain period of time, 
  • This consumes more system resources.
  • If queries are not made frequently, but the querying needs to be high-performance, it is important to adjust it.
  • Fine-tuning this refresh time varies depending on your needs.

Setting up Index Setting

Index-based settings cannot be made in the "elasticsearch.yml" file.

Index-based settings can be changed either by using the index template or by using the index setting API.

Setting API Usage

How to set this setting using the API for all indexes is shown below:

curl -X PUT "<ELASTICSEARCH_IP>:9200/_all/_settings?preserve_existing=true&ignore_unavailable=true" --header 'Content-Type:application/json' -d'{ "index" : {"refresh_interval" : "5s"}}'
CODE

How to set this setting using the API for specified index is shown below:

curl -X PUT "<ELASTICSEARCH_IP>:9200/apinizer-log-apiproxy-<INDEX_KEY>/_settings?preserve_existing=true&ignore_unavailable=true" --header 'Content-Type:application/json' -d'{ "index" : {"refresh_interval" : "5s"}}'
CODE

Index Template Usage

How to set this setting using the Index Template for specified index is shown below:

{
    "data_stream": {
    },
    "index_patterns": [
        "<INDEX_PATTERN_NAME>"
    ],
    "template": {
        "mappings": { ... },
        "settings": {
            "index": {
                "lifecycle": {
                    "name": "<ILM_POLICY_NAME>"
                },
                "number_of_replicas": "0",
                "number_of_shards": "1",
                "refresh_interval": "5sn"
            }
        }
    }
}
CODE