Elasticsearch

This page explains how to configure Elasticsearch connection settings.

The picture below shows the Elasticsearch Connection definition:

The fields used for Elasticsearch Configuration are shown in the table below.

Field	Description
Enabled	Indicates the active state of the setting.
Name	The name of the setting to be made.
Type	The operation to be performed via Elasticsearch API is selected.
Description	Description can be entered.
Authenticate	This field is activated to authenticate.
Username	Username of the Elasticsearch server.
Password	Password of the Elasticsearch server.
Scheme Type	Selected as HTTP or HTTPS.
Host	It is the IP or access address information of the server.
Port	It is the port information of the server.
Socket Reuse Address	Socket reuse address information.
Socket Keep Alive	Socket keep alive information.
IO Threads	IO threads information
Max Connection Per Host	The maximum number of connections per host is entered.
Max Connection Total	The maximum number of connections is entered.
Connection Timeout	The default timeout value for the Elasticsearch Index API for logging API Traffic requests is set to 5 seconds.
Enable to administrate cluster	This field is activated so that Apinizer can perform Index creation, Template creation and Index Lifecycle management operations in the Elasticsearch cluster. If this field is active, other administrative fields also become active; if it is inactive, only the index name value to be read/written is needed.
Index Name	Target Index name.
Template Name	Target Template name.
Number of Replicas	This field describes the default value and function of specifying how many copies of the data will exist. The default value is 1, indicating that only the original data will be retained, with no backup.
Number of Shards	This field explains the number of shards per index. Its default value is set to 1.
Index Refresh Interval	This field describes a parameter that determines how often the index will be updated, in seconds. Its default value is set to 5.

Editing the Apinizer Index Lifecycle

Phases of default lifecycle policies can be edited when enabled to manage the cluster.

An index lifecycle policy allows defining rules for specific actions such as a rollover or force merge within a specified period of an index and automates the process.

Index lifecycle management defines 4 different phases in which indexes can take place:

Hot phase: It is the phase where active querying and writing operations are performed on the index. A new index is created over the pseudo-index name given to very large or very old indexes. It is generally recommended to be stored in SSD space to achieve the best performance requirements.
Warm phase: In this phase, the index can be queried, but the index is in read-only mode. Shards can be reserved for less performing hardware. For faster searches, the number of shards can be reduced and segments can be merged to reduce the number.
Cold phase: Less queries are made on indexes and queries take longer to be answered. It is still too early to delete data. E.g; In the case of security monitoring, old data may be needed to support its investigation. Therefore, hardware with less performance can be used and the number of replicas can be reduced.
Delete phase: When the index is no longer needed, it can be defined when it is safe to delete.

Delete phase, is disabled by default in order to prevent unconsciously log loss.

The picture below shows the Index Lifecycle settings:

The fields used to configure the Index Lifecycle are shown below.

Field		Description
Policy Name		The policy name information entered to configure Index Lifecycle Management.
Hot Phrase	Enable to Warm phase	It is activated to apply the hot phase to the indexes.
	Maximum Age of the Index	Age information of the index in this phase in days (d) is entered.
	Maximum Index Size	The maximum size of the index on the primary shard is entered in gigabytes (gb).
	Maximum Number of Documents	The maximum number of documents in the index. This issue does not include replica shard documents.
Warm Phrase	Enable to Warm phase	It is activated to apply the Warm phase to the indexes.
	Number of Primary Shards for Shrink Index	While applying the shrink action, a new index can be created by reducing the number of primary shards in the index. Therefore, the number of primary shards desired to have indexes that pass this phase is entered.
	Number of Segments for Force Merge	With the force merge action, the number of segments in the shard containing the indexes is reduced. The desired segment number of an index is entered.
	Number of Replica Shard	Replica shard number of indexes can be updated.
Cold Phrase	Enable to Cold phase	It is activated to apply the delete phase to indexes.
	Minimum Age of Cold Phase	The indices enter the cold phase with the minimum age parameter. The index does not enter phase until it is older than the minimum age. The minimum age of the cold phase is entered in days (d).
	Number of Replica Shard	Enter the number of replica shards to which an index will be assigned for the cold phase.
Delete Phrase	Enable to Delete phase	It is activated to apply the delete phase to indexes.
Delete Phrase	Minimum Age of Delete Phase	Indexes enter the delete phase with a minimum age parameter. The index does not enter phase until it is older than the minimum age. The minimum age of the delete phase is entered in days (d).

When the Elasticsearch Cluster definiton is deleted, the relevant indexes, the policies and the templates will not be deleted. However, policies belonging to the cluster's Index Lifecycle Management (ILM) disable all phases. Thus, unnecessary resource consumption is be prevented.

For instance; Supposing any index in the deleted cluster is assigned 5 to the minimum age of the index in the hot phase of the ILM policy. A new index will be created every 5 days. This action is considered necessary in order not to increase the number of indexes and shards in the cluster.

Elasticsearch Monitoring

To monitor the status of the Elasticsearch cluster, the monitoring page can be accessed by clicking the "Monitor" option from the menu at the end of the line you want to monitor in the list of Elasticsearch connections.

Click here for detailed information about Elasticsearch Monitor.

Elasticsearch Rollover

To enable Elasticsearch cluster to automatically create a new index and route new data to this new index, you can access the 'Rollover' option from the menu at the end of the row in the list where Elasticsearch connections are located.