Checks Required Before Starting Installation
1) Operating System Configurations
# Elasticsearch user is created and authorized.
sudo adduser elasticsearch
sudo passwd elasticsearch
sudo usermod -aG wheel elasticsearch
# Switch to the user to continue operations
sudo su - elasticsearch
# It is recommended that the following tools be installed on all servers.
sudo yum update
sudo yum install -y curl wget telnet zip lsof lvm2 net-tools yum-utils bind-utils device-mapper-persistent-data tar
# Firewall is disabled
sudo systemctl stop ufw
sudo systemctl disable ufw
# Firewall is disabled
sudo systemctl stop firewalld
sudo systemctl disable firewalld
# SELinux is disabled to prevent communication problems on servers.
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config
# Swap is disabled and the swap line in /etc/fstab file is deleted to prevent it from restarting.
sudo swapoff -a
sudo vi /etc/fstab
# Then close the vi file (:wq).
2) Elasticsearch Installation
2.1) Operating System Configurations and Elasticsearch Application Installation
sudo vi /etc/security/limits.conf
elasticsearch - nofile 65535
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
sudo sysctl -w vm.swappiness=1
sudo sysctl -w vm.max_map_count=262144
sudo vi /etc/sysctl.conf
vm.max_map_count=262144 elasticsearch
sudo sysctl -p
sudo sysctl vm.max_map_count
2.2) Elasticsearch Installation
sudo mkdir /opt/elasticsearch
cd /opt/elasticsearch
sudo wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.17.10-linux-x86_64.tar.gz
sudo tar -xzf elasticsearch-8.17.10-linux-x86_64.tar.gz
sudo chown -R elasticsearch:elasticsearch /opt/elasticsearch
sudo chmod -Rf 775 /opt/elasticsearch
##At this point, pay attention to where the appropriate disk is mounted or ask system administrators to add the disk to the following path
#The following commands can be used for this check
df -h
lsblk
sudo mkdir /data
sudo mkdir /data/elastic-data/
sudo mkdir /data/elastic-snapdata/
sudo chown -Rf elasticsearch:elasticsearch /data/elastic-*
sudo chmod -Rf 775 /data/elastic-*
2.3) Configuring Elasticsearch Parameters According to Environment
The following parameters must be added by configuring them according to your own environment.
sudo vi /opt/elasticsearch/elasticsearch-8.17.10/config/elasticsearch.yml
ImportantHere, path.data and path.repo addresses must be given as the address of the disk where your log file will be stored in the system, in accordance with the previous item.
If you are setting up Elasticsearch in a Master-Data architecture with High Availability, the following fields must be properly configured in the yaml file: cluster.initial_master_nodes: ["<MASTER_NODE_1_IP>", "<MASTER_NODE_2_IP>", "<MASTER_NODE_3_IP>"]
discovery.seed_hosts: ["<MASTER_NODE_1_IP>", "<MASTER_NODE_2_IP>", "<MASTER_NODE_3_IP>"]
cluster.name: ApinizerEsCluster
node.name: "<NODE_IP>"
network.host: "<NODE_IP>"
http.port: 9200
node.roles: ["master","data"]
cluster.initial_master_nodes: ["<NODE_IP>"]
discovery.seed_hosts: []
path.data: /data/elastic-data/
path.repo: ["/data/elastic-snapdata"]
# Security
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
#xpack.security.http.ssl:
# enabled: true
# keystore.path: certs/elastic-certificates.p12
# truststore.path: certs/elastic-certificates.p12
# Transport SSL:
xpack.security.transport.ssl:
enabled: true
verification_mode: certificate
keystore.path: certs/elastic-certificates.p12
truststore.path: certs/elastic-certificates.p12
# CORS ayarları
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-methods: OPTIONS, HEAD, GET, POST, PUT, DELETE
http.cors.allow-headers: X-Requested-With, X-Auth-Token, Content-Type, Content-Length
You can configure the JVM (Java Virtual Machine) values and other JVM parameters that Elasticsearch will use as follows.
sudo vi /opt/elasticsearch/elasticsearch-8.17.10/config/jvm.options
ImportantHere, you can go up to half of the RAM amount that the operating system has, and this value should not exceed 32GB
2.4) Setting Elasticsearch as Linux Service
sudo chmod -Rf 775 /opt/elasticsearch/elasticsearch-8.17.10/*
sudo vi /etc/systemd/system/elasticsearch.service
[Unit]
Description=Elasticsearch 8.17.10
Wants=network-online.target
After=network-online.target
[Service]
Type=simple
User=elasticsearch
Group=elasticsearch
ExecStart=/opt/elasticsearch/elasticsearch-8.17.10/bin/elasticsearch
Environment=ES_PATH_CONF=/opt/elasticsearch/elasticsearch-8.17.10/config
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
LimitMEMLOCK=infinity
TimeoutStopSec=0
[Install]
WantedBy=multi-user.target
Elasticsearch Master–Data Architecture High Availability (H.A.) Setup InformationWhen setting up a High Availability (H.A.) architecture in an Elasticsearch environment by separating Master and Data nodes, the following steps should be followed:
-
Certificate Creation
TLS/SSL certificates are created only on the initial (primary) Elasticsearch server. The generated certificates are securely copied to all other Elasticsearch servers.
-
Starting the Elasticsearch Service
The Elasticsearch service is started on all nodes, and it is verified that basic cluster communication is established.
-
User Creation
After security is enabled, the necessary system and application users are created (e.g., elastic, kibana, application users).
-
TLS Configuration in YAML File
TLS settings are enabled in the elasticsearch.yml file, and the certificate paths are defined.
-
Restarting Services
All Elasticsearch services are restarted to apply the configuration changes.
Node Role DefinitionsMaster and Data nodes are separated in the elasticsearch.yml file as follows:For Master Node:For Data Node:With this setup:
- Cluster management is handled by the master nodes.
- Data operations and queries run on the data nodes.
- The system becomes highly available (High Availability) and scalable.
The following commands should only be executed during the initial installation:After the initial installation, all operations such as starting, stopping, or restarting Elasticsearch must be performed via systemctl.
# The following command is used to create a certificate. It is not mandatory for this file to be protected with a password (and it is not recommended).
# Adding a password to the .p12 file makes management more difficult, as it requires additional configuration (such as a keystore) for services to restart automatically, and is therefore generally not preferred.
cd /opt/elasticsearch/elasticsearch-8.17.10/
./bin/elasticsearch-certutil ca --days 3650
./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 --days 3650
# The certificates are moved under the config/certs directory after creating it
mkdir config/certs/
mv elastic-certificates.p12 config/certs/
mv elastic-stack-ca.p12 config/certs/
cd /opt/elasticsearch/elasticsearch-8.17.10/config/certs/
openssl pkcs12 -in elastic-certificates.p12 -nokeys -out elastic-certificates.crt
# Before proceeding, make sure that these certificate files have the same permissions and ownership as the Elasticsearch files.
sudo chown -Rf elasticsearch:elasticsearch /opt/elasticsearch
sudo chmod -Rf 775 /opt/elasticsearch
# The following three certificates should be visible under the certs directory:
elastic-certificates.crt # Contains the trusted Root Certificate (CA) that validates this server’s identity, allowing other Elastic Stack components (Apinizer, Kibana, Logstash) to trust the server using this certificate.
elastic-certificates.p12 # Used on the server side to store the private key and server certificate, enabling encryption of incoming TLS/SSL connections and proving the server’s identity.
elastic-stack-ca.p12 # Contains the central Root Certificate Authority (CA) information that provides the trust chain and manages all certificates.
sudo systemctl daemon-reload
sudo systemctl start elasticsearch
sudo systemctl status elasticsearch
sudo systemctl enable elasticsearch
2.5) Built-in user password generation
When the commands are executed, Elasticsearch usernames, certificate details, and other critical information will be displayed on the screen ONE TIME ONLY.It is important to store this information securely.
Elasticsearch’s default built-in username is elastic. However, no password is set for this user by default.
When assigning a password, the passwords of all built-in users will be changed.
cd /opt/elasticsearch/elasticsearch-8.17.10/
# To automatically update the passwords, run the following command in the Elasticsearch directory.
./bin/elasticsearch-setup-passwords auto
# The output will be as follows
Changed password for user elastic
PASSWORD elastic = dd6mjMiemZlAKfOXkUAm
Do not move on to the next step before the password is created.
sudo vi /opt/elasticsearch/elasticsearch-8.17.10/config/elasticsearch.yml
Open the lines related to “xpack.security.http.ssl” and restart Elasticsearch.
xpack.security.http.ssl:
enabled: true
keystore.path: certs/elastic-certificates.p12
truststore.path: certs/elastic-certificates.p12
sudo systemctl restart elasticsearch
This step is necessary to create a new user after Elasticsearch security is enabled. The credentials here can also be defined in Apinizer API Manager.
3) Adding an Elasticsearch Connector to Apinizer
Go to Administration > Connection Management > Elasticsearch page and add the username-password created on the server and the elastic-certificates.crt certificate through the interface, along with the server information.
Apinizer - Apinizer – Elasticsearch Connector Addition Screen
For detailed information, you can refer to the documentation at this link.