When the mapping type value of the data stored in Elasticsearch needs to be changed (See: Apinizer 2022.12.01), reindexing the old indexes is necessary to update their mapping types for accurate search results.

There won't be any issues with the new indexes as they will use the new mapping type.

This document explains how to perform the reindexing process in the case of a mapping type change.

You can access the current indexes and mapping types of Apinizer from this page.


Things To Do

1) To Display Field Type in Mapping:

curl --location --request GET 'http://<ELASTICSEARCH_IP>:9200/apinizer-log-apiproxy-<XXXX>/_field_caps?fields=hr1ra'
POWERSHELL


2) Creating a new template:

Note: In Apinizer, this process was done with code in version 2022.12.01, to make this edit manually, please see the following address:

https://www.elastic.co/guide/en/elasticsearch/reference/7.9/indices-templates-v1.html


3) In order for the new template to be valid, the criteria of ILM must be changed from the screen and the service traffic must continue to flow in the background in order to switch to the new index. To reflect this immediately, lowering the poll_interval with the following code will speed things up:

 curl --location --request PUT 'http://ELASTICSEARCHIP:9200/_cluster/settings' \
--header 'Content-Type: application/json' \
--data-raw '{"persistent":{"indices.lifecycle.poll_interval":"10s"}}'
POWERSHELL


4) After the change is reflected and the new index is created, the ILM and poll_interval must be reinstated:

curl --location --request PUT 'http://ELASTICSEARCHIP:9200/_cluster/settings' \
--header 'Content-Type: application/json' \
--data-raw '{"persistent":{"indices.lifecycle.poll_interval":null}}'
POWERSHELL


5) With the reindex operation, the index remaining in the old mapping type in the datastream is moved to another index as "reindexed":
(New mapping will be used in the reindex. Since it will be taken from the new mapping template, the name of the new index must contain "apinizer-log-apiproxy-<env name:test>-" for the template to be valid)

curl --location --request POST 'http://ELASTICSEARCHIP:9200/_reindex' \
--header 'Content-Type: application/json' \
--data-raw '{
    "conflicts": "proceed",
    "source": {
        "index": ".ds-apinizer-log-apiproxy-test-000001"
    },
    "dest": {
        "index": ".ds-apinizer-log-apiproxy-test-000001-reindexed",
        "op_type": "create"
    }
}'
POWERSHELL


6) Deleting the remaining index in the old mapping type:

curl --location --request DELETE 'http://ELASTICSEARCHIP:9200/.ds-apinizer-log-apiproxy-test-000001'
POWERSHELL


If you create the new index with a different name than the current system, you need to continue with the following steps.

The following steps continue assuming that the above nomenclature does not include ".ds".


7)The name of the data moved to the index named "apinizer-log-apiproxy-test-000001-reindexed" must be changed in order to be displayed on the manager.

# Putting the new index in read-only mode:

 curl --location --request PUT 'http://ELASTICSEARCHIP:9200/apinizer-log-apiproxy-test-000001-reindexed/_settings' \
--header 'Content-Type: application/json' \
--data-raw '{
  "settings": {
    "index.blocks.write": "true"
  }
}'
POWERSHELL


# Cloning the new index to the original name and removing it from read-only mode:

 curl --location --request PUT 'http://ELASTICSEARCHIP:9200/apinizer-log-apiproxy-test-000001-reindexed/_clone/.ds-apinizer-log-apiproxy-test-000001' \
--header 'Content-Type: application/json' \
--data-raw '{
  "settings": {
    "index.blocks.write": null 
  }
}'
POWERSHELL


# Wait until the target index is green:

 GET /_cluster/health/target_index?wait_for_status=green&timeout=30s
POWERSHELL


# If this process, which normally takes place quickly, takes a long time, check if there is a problem with the following commands:

GET /_cat/indices/target_index
GET /_cat/recovery/target_index
GET /_cluster/allocation/explain
POWERSHELL


# The old index is deleted:

curl --location --request DELETE 'http://ELASTICSEARCHIP:9200/apinizer-log-apiproxy-test-000001-reindexed'
POWERSHELL


# If we did reindex, the ilm would be automatically captured and added due to the template, but when we clone it, only the name was changed and all other settings were changed to the new one.
# When we check ilm we will see that it is empty:

curl -X GET "http://ELASTICSEARCHIP:9200/.ds-apinizer-log-apiproxy-test-000001/_ilm/explain?pretty"
POWERSHELL


# So it is necessary to add ilm to the new index:

curl -X PUT "http://ELASTICSEARCHIP:9200/.ds-apinizer-log-apiproxy-test-000001/_settings?pretty" -H 'Content-Type: application/json' -d'
{
  "index": {
    "lifecycle": {
      "name": "apinizer-log-ilm-policy-test"
    }
  }
}
'
POWERSHELL

Detailed information for the reindex process: https://www.elastic.co/guide/en/elasticsearch/reference/7.9/docs-reindex.html