Commonly Used Commands in Elasticsearch
Checking Indexes
Check Document Count
curl -X GET "<ELASTICSEARCH_IP>:9200/<INDEX_KEY>/_doc/_count"
BASH
Check Index with Where Clause
curl -X GET "<ELASTICSEARCH_IP>:9200/*/_search?pretty=true&q=apiGatewayName:KPS+XYS"
CODE
curl -X GET "<ELASTICSEARCH_IP>:9200/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must": [
{
"match": {
"apiGatewayName": "TEST KPS GW"
}
},
{
"range": {
"created": {
"gte": "now-7d/d",
"lt": "now-5d/d"
}}}]}}}'
CODE
curl -X GET "<ELASTICSEARCH_IP>:9200/_search?pretty" -H 'Content-Type: application/json' -d'
{
"_source": ["contentType"],
"size": 50,
"query": {
"match_all": {}
}}'
CODE
Finding Requests That Have the Different CID From Response
curl -X GET "<ELASTICSEARCH_IP>:9200/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{
"script": {
"script": {
"source": "doc['headerRequestFromClient.APINIZER-CORRELATION-ID.keyword'].value != doc['headerResponseToClient.APINIZER-CORRELATION-ID.keyword'].value",
"lang": "painless"
}}},
{
"range": {
"created": {
"gte": "2021-06-28T16:30:32.000"
}}},
{
"term":{
"instanceId":2
}}]}}}'
CODE
Finding Request on Specific Interval
curl -X GET "<ELASTICSEARCH_IP>:9200/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"filter": [
{
"match": {
"api": "26"
}},
{
"range": {
"created": {
"gte": "2020-06-08T15:08:00.000",
"lte": "2020-06-08T15:12:00.000"
}}}]}},
"aggs" : {
"reqs_over_time" : {
"date_histogram" : {
"field" : "created",
"interval" : "1s"
}}}}'
CODE
Find Documents by Correlation ID
curl -X GET "<ELASTICSEARCH_IP>:9200/_search?pretty" -H 'Content-Type: application/json' -d' { "query" : { "match":{ "aci": "c3d8523e-e3ac-497b-ac7a-76853198c239" }}}'
BASH
Delete by Index Name
curl -X DELETE "<ELASTICSEARCH_IP>:9200/<INDEX_KEY>"
BASH
Deleting Indexes with Given Word
curl -X DELETE "<ELASTICSEARCH_IP>:9200/*metric*"
BASH
Changing Elasticsearch Cluster's Read_Only Status
curl -X PUT "<ELASTICSEARCH_IP>:9200/_all/_settings?wait_for_completion=false" -H "Content-Type: application/json" -d'
{
"index.blocks.read_only_allow_delete": null,
"index.blocks.write": null
}'
BASH
For only one index:
curl -X PUT "<ELASTICSEARCH_IP>:9200/<INDEX_KEY>/_settings?pretty" -H 'Content-Type: application/json' -d'
{
"index.blocks.read_only_allow_delete": null,
"index.blocks.write": null
}'
BASH
Deleting Logs Up to a Specific Date
curl -X POST "<ELASTICSEARCH_IP>:9200/.ds-apinizer-log-apiproxy-<LOG_KEY>-000*/_update_by_query?pretty" -H 'Content-Type: application/json' -d '
{
"query": {
"bool": {
"filter": [
{
"range": {
"@timestamp": {
"lte": "2024-04-20T00:00:00.000Z"
}
}
}
]
}
},
"script": {
"source": "ctx._source.remove(\"tba\"); ctx._source.remove(\"fbarb\"); ctx._source.remove(\"tcb\")"
}
}'
BASH
To delete by api proxy id value instead of time range, "match": { "api": "64ac03067e8f7400cf4adbdd" } filter can be used instead of "range": { "@timestamp": { "lte": "2024-04-20T00:00:00.000Z" } } filter.
To examine the data structure of Elasticsearch and identify fields to be deleted, visit the following link: API Traffic Log Data Structure.
Switching to New Index with Rollover
http://<ELASTICSEARCH_IP>:9200/apinizer-log-apiproxy-<INDEX_KEY>/_rollover
BASH
Search
Search Documents on Specific Index
curl -X GET "<ELASTICSEARCH_IP>:9200/<INDEX_KEY>/_search?pretty=true&q=*:*"
BASH
Search Documents on Index with Criteria
curl -X GET "<ELASTICSEARCH_IP>:9200/*/_search?pretty=true&q=apiProxyName:Petstore+API"
BASH
Search Documents by Interval
curl -X GET "<ELASTICSEARCH_IP>:9200/_search?pretty" -H 'Content-Type: application/json' -d
'{
"query": {
"bool": {
"must": [
{
"match": {
"apiProxyName": "Petstore API"
}
},
{
"range": {
"created": {
"gte": "now-7d/d",
"lt": "now-5d/d"
}
}
}
]
}
}
}'
BASH
Search with Conditions and Aggregating the Results
curl -X GET "<ELASTICSEARCH_IP>:9200/_search?pretty" -H 'Content-Type: application/json' -d
'{
"query": {
"bool": {
"filter": [
{
"match": {
"api": "26"
}
},
{
"range": {
"created": {
"gte": "2020-06-08T15:08:00.000",
"lte": "2020-06-08T15:12:00.000"
}
}
}
]
}
},
"aggs" : {
"reqs_over_time" : {
"date_histogram" : {
"field" : "created",
"interval" : "1s"
}
}
}
}'
BASH
Search with Limitations
curl -X GET "<ELASTICSEARCH_IP>:9200/_search?pretty" -H 'Content-Type: application/json' -d'
{
"_source": ["contentType"],
"size": 50,
"query": {
"match_all": {}
}
}'
BASH
Update
Update Document
curl -X PUT "<ELASTICSEARCH_IP>:9200/<INDEX_KEY>/doc/1?pretty&pretty" -H 'Content-Type: application/json' -d'
{
"name": "John Doe"
}'
BASH
Deleting Some Key and Values
curl -X POST "<ELASTICSEARCH_IP>:9200/_update_by_query?pretty" -H 'Content-Type: application/json' -d'
{
"script" :
"ctx._source.headerRequestFromClient.remove('header-name-1');
ctx._source.headerRequestToTarget.remove('header-name-2');",
"query": { "match_all": {} }
}
BASH
Deleting Specific Values by Criteria
curl -X POST "<ELASTICSEARCH_IP>:9200/*/_update_by_query?pretty&conflicts=proceed&requests_per_second=200" -H 'Content-Type: application/json' -d'
{
"query": {
"bool" : {
"filter": {
"exists": {
"field": "headerRequestFromClient.user_username"
}
},
"must_not" : {
"term": {
"headerRequestFromClient.user_password": ""
}
}
}
},
"script": "ctx._source.headerRequestFromClient.remove(\"user_password\");"
}
BASH
- Execution Reject error will be prevented by requests_per_second key's value.
- Batch Size is 1000 by default. Waiting time between two request is set by giving 5 (=1000/200)
http://<ELASTICSEARCH_IP>:9200/*/_update_by_query?conflicts=proceed&wait_for_completion=true
{
"script": {
"inline":
"ctx._source.remove('apiGatewayApiMethodId');
ctx._source.remove('bodyRequestToTarget');
ctx._source.remove('bodyResponseFromTarget');
ctx._source.remove('bodyResponseToClient');
ctx._source.remove('headerRequestFromClient');
ctx._source.remove('headerRequestToTarget');
ctx._source.remove('headerResponseFromTarget');
ctx._source.remove('headerResponseToClient');
ctx._source.remove('paramRequestFromClient');
ctx._source.remove('paramRequestToTarget');",
"lang": "painless"
},
"query": {
"bool": {
"must": [
{
"range": {
"created": {
"gte": "2019-02-01T20:03:12.963",
"lte": "2019-04-30T20:03:12.963"
}
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
}
}
BASH
Setting Replica Number
curl -X PUT "<ELASTICSEARCH_IP>:9200/_template/template_genel?pretty" -H 'Content-Type: application/json' -d'
{
"index_patterns": ["*log*", "*metric*", "*db*"],
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
}
}
'
BASH
curl -XPUT '<ELASTICSEARCH_IP>:9200/*/_settings' -H 'Content-Type: application/json' -d'
{
"index" : {
"number_of_replicas" : 0
}
}'
BASH
Shard Allocation
curl -X PUT "<ELASTICSEARCH_IP>:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
{
"transient" : {
"cluster.routing.allocation.enable" : "all"
}
}'
BASH
Shard Limit
http://<ELASTICSEARCH_IP>:9200/_cluster/settings
{
"persistent" : {
"cluster.routing.allocation.total_shards_per_node" : 2000 ,
"cluster.max_shards_per_node":2000
}
}
BASH
Log
Changing Log Level
curl -X PUT "<ELASTICSEARCH_IP>:9200/_cluster/settings" -H 'Content-Type: application/json' -d'{"transient":{"logger._root":"DEBUG"}}'
curl -X PUT "<ELASTICSEARCH_IP>:9200/_cluster/settings" -H 'Content-Type: application/json' -d'{"transient":{"logger._root":"INFO"}}'
BASH
ShowLog Settings
curl -X PUT "<ELASTICSEARCH_IP>:9200/*log*/_settings?pretty" -H 'Content-Type: application/json' -d'
{
"index.search.slowlog.threshold.fetch.trace": "200ms",
"index.search.slowlog.level": "trace"
}'
BASH
Other
_cat APIs
curl -X GET "<ELASTICSEARCH_IP>:9200/_cat/indices/*?v&s=index&pretty"
curl -X GET "<ELASTICSEARCH_IP>:9200/_cat/thread_pool?v&h=id,node_name,ip,name,core,queue,rejected,completed,max"
BASH
_nodes APIs
curl -X GET "<ELASTICSEARCH_IP>:9200/_nodes/os?pretty"
curl -X GET "<ELASTICSEARCH_IP>:9200/_nodes/jvm?pretty"
curl -X GET "<ELASTICSEARCH_IP>:9200/_nodes/thread_pool?pretty"
curl -X GET "<ELASTICSEARCH_IP>:9200/_nodes/stats/process?filter_path=**.max_file_descriptors"
BASH
_cluster APIs
curl -X GET "<ELASTICSEARCH_IP>:9200/_cluster/stats?pretty"
curl -XGET '<ELASTICSEARCH_IP>:9200/_cluster/state?pretty=true' > result.json
BASH
Flush
curl -X POST "<ELASTICSEARCH_IP>:9200/*log*/_flush/synced?pretty"
BASH
Removing the Log Writing Barrier
curl -XPUT 'http://<ELASTICSEARCH_IP>:9200/*log*/_settings' -H 'Content-Type: application/json' -d'{"index": {"blocks": {"read_only_allow_delete": null}}}'
BASH
General Information About Snapshots
curl 'http://<ELASTICSEARCH_IP>:9200/_snapshot?pretty'
BASH
Repository and Snapshot Details
curl 'http://<ELASTICSEARCH_IP>:9200/_slm/policy/apinizer-slm-policy-<INDEX_KEY>?pretty'
BASH
Detailed Examination of the Repository and Snapshot Names Obtained from the Previous Command
curl -XGET "http://<ELASTICSEARCH_IP>:9200/_snapshot/apinizer-repository-<INDEX_KEY>/apinizer-snapshot-<INDEX_KEY>-2023.03.13-m33h8zcpq_if4swyzn0wrq?pretty"
curl -XGET "http://<ELASTICSEARCH_IP>:9200/_snapshot/apinizer-repository-<INDEX_KEY>/apinizer-snapshot-<INDEX_KEY>-2023.03.13-m33h8zcpq_if4swyzn0wrq/_status?pretty"
BASH
Deleting All Settings Related to Snapshots
curl -XDELETE 'http://<ELASTICSEARCH_IP>:9200/_snapshot/apinizer-repository-<INDEX_KEY>?pretty'
BASH