Checks Required Before Starting Installation
1) Operating System Configurations
# Elasticsearch user is created and authorized
sudo adduser elasticsearch
sudo usermod -aG sudo elasticsearch
# Switch to the user to continue operations
sudo su - elasticsearch
# It is recommended that the following tools be installed on all servers
sudo apt update
sudo apt install -y curl wget net-tools gnupg2 software-properties-common apt-transport-https ca-certificates
# Firewall is disabled
sudo systemctl stop ufw
sudo systemctl disable ufw
# Swap is disabled and the swap line in /etc/fstab file is deleted to prevent it from restarting
sudo swapoff -a
sudo vi /etc/fstab
# Then close the vi file (:wq)
2) Elasticsearch Installation
2.1) Operating System Configurations and Elasticsearch Application Installation
sudo vi /etc/security/limits.conf
elasticsearch - nofile 65535
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
sudo sysctl -w vm.swappiness=1
sudo sysctl -w vm.max_map_count=262144
sudo vi /etc/sysctl.conf
vm.max_map_count=262144 elasticsearch
sudo sysctl -p
sudo sysctl vm.max_map_count
2.2) Elasticsearch Installation
sudo mkdir /opt/elasticsearch
cd /opt/elasticsearch
sudo wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.17.10-linux-x86_64.tar.gz
sudo tar -xzf elasticsearch-8.17.10-linux-x86_64.tar.gz
sudo chown -R elasticsearch:elasticsearch /opt/elasticsearch
sudo chmod -Rf 775 /opt/elasticsearch
##At this point, pay attention to where the appropriate disk is mounted or ask system administrators to add the disk to the following path
#The following commands can be used for this check
df -h
lsblk
sudo mkdir /data
sudo mkdir /data/elastic-data/
sudo mkdir /data/elastic-snapdata/
sudo chown -Rf elasticsearch:elasticsearch /data/elastic-*
sudo chmod -Rf 775 /data/elastic-*
2.3) Configuring Elasticsearch Parameters According to Environment
The parameters in the configuration file must be added by configuring them according to your own environment.
sudo vi /opt/elasticsearch/elasticsearch-8.17.10/config/elasticsearch.yml
ImportantHere, path.data and path.repo addresses must be given as the address of the disk where your log file will be stored in the system, in accordance with the previous item.
If you are setting up Elasticsearch in a Master-Data architecture with High Availability, the following fields must be properly configured in the yaml file: cluster.initial_master_nodes: ["<MASTER_NODE_1_IP>", "<MASTER_NODE_2_IP>", "<MASTER_NODE_3_IP>"]
discovery.seed_hosts: ["<MASTER_NODE_1_IP>", "<MASTER_NODE_2_IP>", "<MASTER_NODE_3_IP>"]
cluster.name: ApinizerEsCluster
node.name: "<NODE_IP>"
network.host: "<NODE_IP>"
http.port: 9200
node.roles: ["master","data"]
cluster.initial_master_nodes: ["<NODE_IP>"]
discovery.seed_hosts: []
path.data: /data/elastic-data/
path.repo: ["/data/elastic-snapdata"]
# Security
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
#xpack.security.http.ssl:
# enabled: true
# keystore.path: certs/elastic-certificates.p12
# truststore.path: certs/elastic-certificates.p12
# Transport SSL:
xpack.security.transport.ssl:
enabled: true
verification_mode: certificate
keystore.path: certs/elastic-certificates.p12
truststore.path: certs/elastic-certificates.p12
# CORS settings
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-methods: OPTIONS, HEAD, GET, POST, PUT, DELETE
http.cors.allow-headers: X-Requested-With, X-Auth-Token, Content-Type, Content-Length
You can configure the JVM (Java Virtual Machine) values and other JVM parameters that Elasticsearch will use as follows.
sudo vi /opt/elasticsearch/elasticsearch-8.17.10/config/jvm.options
ImportantHere, you can go up to half of the RAM amount that the operating system has, and this value should not exceed 32GB
2.4) Setting Elasticsearch as Linux Service
sudo chmod -Rf 775 /opt/elasticsearch/elasticsearch-8.17.10/*
sudo vi /etc/systemd/system/elasticsearch.service
[Unit]
Description=Elasticsearch 8.17.10
Wants=network-online.target
After=network-online.target
[Service]
Type=simple
User=elasticsearch
Group=elasticsearch
ExecStart=/opt/elasticsearch/elasticsearch-8.17.10/bin/elasticsearch
Environment=ES_PATH_CONF=/opt/elasticsearch/elasticsearch-8.17.10/config
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
LimitMEMLOCK=infinity
TimeoutStopSec=0
[Install]
WantedBy=multi-user.target
Elasticsearch Master–Data Architecture High Availability (H.A.) Installation InformationWhen setting up High Availability (H.A.) architecture by separating Master and Data nodes in the Elasticsearch environment, the following steps should be followed:
-
Certificate Creation
TLS/SSL certificates are created only on the first (primary) Elasticsearch server. The created certificates are securely copied to all other Elasticsearch servers.
-
Starting Elasticsearch Service
Elasticsearch service is started on all nodes and it is verified that basic cluster communication is established.
-
User Creation
After security is activated, necessary system and application users are created. (e.g., elastic, kibana, application users)
-
TLS Configuration in YAML File
TLS settings are activated in the elasticsearch.yml file and certificate paths are defined.
-
Restarting Services
All Elasticsearch services are restarted for configuration changes to take effect.
Node Role DefinitionsMaster and Data nodes are separated in the elasticsearch.yml file as follows:For Master Node:For Data Node:With this structure:
- Cluster management is performed by master nodes.
- Data processing and queries run on data nodes.
- The system becomes highly available (High Availability) and scalable.
The following commands should only be run during initial installation:After initial installation, all operations such as starting, stopping, or restarting Elasticsearch should be done through systemctl.
# The following command is used to create certificates. It is not mandatory to protect this file with a password (Not recommended).
# Adding a password to the p12 file requires additional configuration (Keystore, etc.) for automatic service restarts, which complicates management and is generally not preferred.
cd /opt/elasticsearch/elasticsearch-8.17.10/
./bin/elasticsearch-certutil ca --days 3650
./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 --days 3650
# Certificates are moved to the config/certs directory created
mkdir config/certs/
mv elastic-certificates.p12 config/certs/
mv elastic-stack-ca.p12 config/certs/
cd /opt/elasticsearch/elasticsearch-8.17.10/config/certs/
openssl pkcs12 -in elastic-certificates.p12 -nokeys -out elastic-certificates.crt
# Before proceeding, ensure that these certificate files have the same privileges as the Elasticsearch files.
sudo chown -Rf elasticsearch:elasticsearch /opt/elasticsearch
sudo chmod -Rf 775 /opt/elasticsearch
#The following 3 certificates should be displayed under the certs directory:
elastic-certificates.crt # Contains the trusted Root Certificate (CA) that authenticates this server; thus, other Elastic Stack components (Apinizer, Kibana, Logstash) can trust the server with this certificate.
elastic-certificates.p12 # Used on the server side to encrypt incoming TLS/SSL connections and prove the server's identity by hosting the private key and server certificate.
elastic-stack-ca.p12 # Contains the central Root Certificate Authority (CA) information that provides this entire trust chain and certificate management.
sudo systemctl daemon-reload
sudo systemctl start elasticsearch
sudo systemctl status elasticsearch
sudo systemctl enable elasticsearch
2.5) Creating Built-in User Passwords
When the commands are executed, Elasticsearch username, certificate information, and other critical information will be displayed on the screen ONE TIME ONLY.It is important to store this information securely.
Elasticsearch’s default built-in username is elastic. However, there is no password created for this username.
When assigning passwords, all built-in users’ passwords will be changed.
cd /opt/elasticsearch/elasticsearch-8.17.10/
# Run the following command in the Elasticsearch directory to automatically update passwords.
./bin/elasticsearch-setup-passwords auto
#The output it will give will be as follows
Changed password for user elastic
PASSWORD elastic = dd6mjMiemZlAKfOXkUAm
Do not proceed to the next step before creating the password.
sudo vi /opt/elasticsearch/elasticsearch-8.17.10/config/elasticsearch.yml
The lines related to “xpack.security.http.ssl” are opened and elasticsearch is restarted.
xpack.security.http.ssl:
enabled: true
keystore.path: certs/elastic-certificates.p12
truststore.path: certs/elastic-certificates.p12
sudo systemctl restart elasticsearch
This step is necessary to create a new user after enabling Elasticsearch security. These credentials can also be defined in Apinizer API Manager.
3) Adding Elasticsearch Connector to Apinizer
Go to Administration > Connection Management > Elasticsearch page and add the username-password and elastic-certificates.crt certificate created on the server, along with server information, through the interface.
Apinizer - Elasticsearch Connector addition screen
For detailed information, you can review the documentation at this link.