Thursday, July 18, 2024
HomeHow toHow to Install OpenSearch on Debian 12

How to Install OpenSearch on Debian 12

OpenSearch is an open-source fork of Elasticsearch and Kibana developed by Amazon. It is a real-time, distributed, and scalable search engine that allows for full-text search as well as analytics. It is used to index and search through large amounts of data. It is commonly deployed alongside OpenSearch Dashboards (a fork of Kibana), a powerful data visualization dashboard for OpenSearch. OpenSearch Dashboards allows you to explore the OpenSearch log data, and build dashboards and queries to gain insight into your application.

This tutorial will show you how to install OpenSearch on a Debian 12 server. You will also learn how to secure your installation with custom TLS certificates and enable authentication on OpenSearch.

Prerequisites

  • A server running Debian 12 with a minimum of 4GB of RAM.

  • A non-root user with sudo privileges.

  • The uncomplicated Firewall(UFW) is enabled and running.

  • A Fully Qualified Domain Name (FQDN) like opensearch.example.com pointing to your server. This is beneficial if you want to serve OpenSearch Dashboards via SSL using a proxy server.

  • Everything is updated.

    $ sudo apt update && sudo apt upgrade
    
  • A few essential packages are required for the tutorial and Craft CMS to run. Some of these will already be on your server.

    $ sudo apt install curl wget nano software-properties-common dirmngr apt-transport-https ca-certificates lsb-release debian-archive-keyring gnupg2 ufw unzip -y
    

Step 1 – Configure Firewall

The first step before installing any packages is to configure the firewall to allow HTTP and HTTPS connections.

Check the status of the firewall.

$ sudo ufw status

You should see something like the following.

Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
OpenSSH (v6)               ALLOW       Anywhere (v6)

Allow HTTP and HTTPs ports.

$ sudo ufw allow http
$ sudo ufw allow https

Check the status again to confirm.

$ sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
80/tcp                     ALLOW       Anywhere
443/tcp                    ALLOW       Anywhere
OpenSSH (v6)               ALLOW       Anywhere (v6)
80/tcp (v6)                ALLOW       Anywhere (v6)
443/tcp (v6)               ALLOW       Anywhere (v6)

Step 2 – Prepare System

Before installing OpenSearch, we need to prepare and optimize our system for it. There are several steps involved which include disabling any swap memory if you have it, setting the number of memory maps for your system, and setting up the system hostname.

First, check if you have swap memory enabled.

$ free -m

If you have swap enabled, then you should see a similar output.

               total        used        free      shared  buff/cache   available
Mem:            7945        1083        6492          34         636        6861
Swap:           4242           0        4242

Run the following commands to disable swap space permanently.

$ sudo swapoff -a
$ sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

Check for swap memory again and it should show the following.

               total        used        free      shared  buff/cache   available
Mem:            7945        1113        6449          37         651        6831
Swap:              0           0           0

OpenSearch uses a mapfs directory by default to store its indices. The default operating system limits on nmap counts are too low which can cause out of memory errors.

Check the existing value for the vm.max_map_count variable.

$ sysctl -n vm.max_map_count
65530

To increase the number of memory maps used by each process, run the following command.

$ sudo sysctl -w vm.max_map_count=262144

This will work only till the system is restarted. To make this value permanent, add the line vm.max_map_count=262144 in the /etc/sysctl.conf file using the following command.

$ echo "vm.max_map_count=262144" | sudo tee -a /etc/sysctl.conf

You can make the change permanent by either rebooting the system or running the following command.

$ sudo sysctl -p

The final step is setting up the hostname. Depending upon where you are hosting your server, some cloud companies allow you to set your hostname directly via their control panel. However, it is better to do it via the terminal.

Run the following command to set the hostname.

$ sudo hostnamectl set-hostname opensearch

Add the FQDN (Fully qualified domain name) to the /etc/hosts file. In here, we map the hostname opensearch to the domain name opensearch.example.com and point it to the server IP address. Replace <serverIPaddress> in the command below with the IP address of your server.

$ echo "<serverIPaddress>  opensearch.example.com  opensearch" | sudo tee -a /etc/hosts

Check the hostname and the FQDN using the following commands.

$ hostname
opensearch
$ hostname -f
opensearch.example.com

This finishes the process of optimizing our system to work with OpenSearch.

Step 3 – Install OpenSearch

Import the OpenSearch GPG key used for signing the installation package.

$  curl -o- https://artifacts.opensearch.org/publickeys/opensearch.pgp | sudo gpg --dearmor --batch --yes -o /usr/share/keyrings/opensearch-keyring

Create the OpenSearch APT repository file.

$ echo "deb [signed-by=/usr/share/keyrings/opensearch-keyring] https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable main" | sudo tee /etc/apt/sources.list.d/opensearch-2.x.list

Update the system repository list.

$ sudo apt update

Check all the available versions of OpenSearch available.

$ sudo apt list -a opensearch
Listing... Done
opensearch/stable 2.12.0 amd64
opensearch/stable 2.11.1 amd64
opensearch/stable 2.11.0 amd64
opensearch/stable 2.10.0 amd64
opensearch/stable 2.9.0 amd64
opensearch/stable 2.8.0 amd64
opensearch/stable 2.7.0 amd64
opensearch/stable 2.6.0 amd64
opensearch/stable 2.5.0 amd64

The latest version of OpenSearch v2.12.0 requires you to set the admin demo password before the installation or it will fail.

You can install the latest version of OpenSearch using the following command. We can set the admin demo password using the environment variable in the installation command itself. Choose a strong password.

$ sudo env OPENSEARCH_INITIAL_ADMIN_PASSWORD=<custom-admin-password> apt install opensearch

If you see the following warning during the installation, you can safely ignore it.

......
chown: warning: '.' should be ':': ‘opensearch.opensearch’
chown: warning: '.' should be ':': ‘opensearch.opensearch’
chown: warning: '.' should be ':': ‘opensearch.opensearch’
chown: warning: '.' should be ':': ‘opensearch.opensearch’
chown: warning: '.' should be ':': ‘opensearch.opensearch’
......

Or you can install a specific version.

$ sudo apt install opensearch=2.11.1

Reload the service daemon.

$ sudo systemctl daemon-reload

Enable the OpenSearch service.

$ sudo systemctl enable opensearch

Start the OpenSearch service.

$ sudo systemctl start opensearch

Verify the status of the OpenSearch service.

$ sudo systemctl status opensearch

You should get the following output.

? opensearch.service - OpenSearch
     Loaded: loaded (/lib/systemd/system/opensearch.service; enabled; preset: enabled)
     Active: active (running) since Sat 2024-03-02 06:37:01 UTC; 5s ago
       Docs: https://opensearch.org/
   Main PID: 31109 (java)
      Tasks: 75 (limit: 4652)
     Memory: 1.3G
        CPU: 1min 3.128s
     CGroup: /system.slice/opensearch.service
             ??31109 /usr/share/opensearch/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true...

Step 4 – Test OpenSearch

Before proceeding any further, let us check if the installation is working fine or not. Run the following command to verify if OpenSearch is running as intended.

$ curl -X GET https://localhost:9200 -u 'admin:<custom-admin-password>' --insecure

You should get a similar output.

{
  "name" : "opensearch",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "3oFvYn96St2MOpDP3Sf29g",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.12.0",
    "build_type" : "deb",
    "build_hash" : "2c355ce1a427e4a528778d4054436b5c4b756221",
    "build_date" : "2024-02-20T02:18:31.541484890Z",
    "build_snapshot" : false,
    "lucene_version" : "9.9.2",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

Query the OpenSearch plugins endpoint to see the list of available plugins.

$ curl -X GET https://localhost:9200/_cat/plugins?v -u 'admin:<custom-admin-password>' --insecure

You can also use the following command for the same.

$ /usr/share/opensearch/bin/opensearch-plugin list

You should get a similar output.

name       component                            version
opensearch opensearch-alerting                  2.12.0.0
opensearch opensearch-anomaly-detection         2.12.0.0
opensearch opensearch-asynchronous-search       2.12.0.0
opensearch opensearch-cross-cluster-replication 2.12.0.0
opensearch opensearch-custom-codecs             2.12.0.0
opensearch opensearch-flow-framework            2.12.0.0
opensearch opensearch-geospatial                2.12.0.0
opensearch opensearch-index-management          2.12.0.0
opensearch opensearch-job-scheduler             2.12.0.0
opensearch opensearch-knn                       2.12.0.0
opensearch opensearch-ml                        2.12.0.0
opensearch opensearch-neural-search             2.12.0.0
opensearch opensearch-notifications             2.12.0.0
opensearch opensearch-notifications-core        2.12.0.0
opensearch opensearch-observability             2.12.0.0
opensearch opensearch-performance-analyzer      2.12.0.0
opensearch opensearch-reports-scheduler         2.12.0.0
opensearch opensearch-security                  2.12.0.0
opensearch opensearch-security-analytics        2.12.0.0
opensearch opensearch-skills                    2.12.0.0
opensearch opensearch-sql                       2.12.0.0

You can install additional plugins using the following command.

$ /usr/share/opensearch/bin/opensearch-plugin install <plugin-name>

You can find the list of available OpenSearch plugins on their official website.

Step 5 – Configure OpenSearch

OpenSearch’s default settings are not fit for a production environment. By default, OpenSearch is not bound to a network interface and is unreachable from outside. Furthermore, default user names and passwords are being used which can cause security issues if used directly in a production environment.

OpenSearch stores its configuration in the /etc/opensearch/opensearch.yml file. Open it for editing.

$ sudo nano /etc/opensearch/opensearch.yml

Find the #network.host: 192.168.0.1 line and uncomment it by removing the hash (#) in front of it and changing its value as shown. This binds OpenSearch to all available interfaces. You can also choose to enter a specific IP to bind it to a specific interface.

# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0

Add the following lines at the bottom of the file.

# Unless you have already configured a cluster, you should set
# discovery.type to single-node, or the bootstrap checks will
# fail when you try to start the service.
discovery.type: single-node

# If you previously disabled the Security plugin in opensearch.yml,
# be sure to re-enable it. Otherwise you can skip this setting.
plugins.security.disabled: false

Since we are installing on a single server, we have set discovery.type to single-node. Setting the plugins.security.disabled to false enables OpenSearch security plugin.

Save the file by pressing Ctrl + X and entering Y when prompted.

We also need to specify initial and maximum JVM (Java Virtual Memory) heap sizes. This setting depends on your server’s RAM size. For example, we are using a server with 4GB of RAM so we are setting the heap size to 3GB.

Create and open the heapsize.options in the /etc/opensearch/jvm.options.d directory for editing. It will overwrite the default value set in the /etc/opensearch/jvm.options file.

$ sudo nano /etc/opensearch/jvm.options.d/heapsize.options

Add the following lines to it.

-Xms3g
-Xmx3g

Save the file by pressing Ctrl + X and entering Y when prompted.

Restart the OpenSearch service to implement the changes.

$ sudo systemctl restart opensearch

You can check the current heap size and maximum memory using the following command.

$ curl -sS "https://localhost:9200/_cat/nodes?h=heap*&v" -u 'admin:<custom-admin-password>' --insecure

You should get the following output.

heap.current heap.percent heap.max
     345.5mb           11      3gb

Step 6 – Configure TLS for OpenSearch

Here we will be generating certificates to secure OpenSearch. TLS certificates provide security by allowing clients to confirm the identity of hosts and encrypt traffic between a client and the host. We will generate the following certificates

  1. Root CA Certificates – These are the primary certificates that will sign all other certificates.
  2. Admin certificates – These certificates are used to gain elevated rights for performing administrative tasks related to the Security plugin.
  3. Node certificates – These certificates are used for communication between a node and the client.

All the OpenSearch certificates are stored in the /etc/opensearch directory. The first step is to remove the default certificates set during OpenSearch’s installation.

Switch to the OpenSearch directory.

$ cd /etc/opensearch

Remove the default certificates.

$ sudo rm -f *.pem

Open the OpenSearch configuration file for editing.

$ sudo nano opensearch.yml

Comment out the following code at the bottom of the file by putting a hash in front of each line inside the OpenSearch Security Demo Configuration section as shown below.

######## Start OpenSearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production
#plugins.security.ssl.transport.pemcert_filepath: esnode.pem
#plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
#plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
#plugins.security.ssl.transport.enforce_hostname_verification: false
#plugins.security.ssl.http.enabled: true
#plugins.security.ssl.http.pemcert_filepath: esnode.pem
#plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
#plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
#plugins.security.allow_unsafe_democertificates: true
#plugins.security.allow_default_init_securityindex: true
#plugins.security.authcz.admin_dn: ['CN=kirk,OU=client,O=client,L=test,C=de']
#plugins.security.audit.type: internal_opensearch
#plugins.security.enable_snapshot_restore_privilege: true
#plugins.security.check_snapshot_restore_write_privileges: true
#plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access]
#plugins.security.system_indices.enabled: true
#plugins.security.system_indices.indices: [.plugins-ml-config, .plugins-ml-connector,
#  .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task, .plugins-ml-conversation-meta,
#  .plugins-ml-conversation-interactions, .plugins-ml-memory-meta, .plugins-ml-memory-message,
#  .opendistro-alerting-config, .opendistro-alerting-alert*, .opendistro-anomaly-results*,
#  .opendistro-anomaly-detector*, .opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state,
#  .opendistro-reports-*, .opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability,
#  .ql-datasources, .opendistro-asynchronous-search-response*, .replication-metadata-store,
#  .opensearch-knn-models, .geospatial-ip2geo-data*, .plugins-flow-framework-config,
#  .plugins-flow-framework-templates, .plugins-flow-framework-state]
#node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########

Save the file by pressing Ctrl + X and entering Y when prompted.

Next, create a directory to store certificates inside the /etc/opensearch directory.

$ sudo mkdir /etc/opensearch/certs

Change the permission of the directory to the currently logged-in user to avoid using sudo while generating the certificates.

$ sudo chown $USER:$USER /etc/opensearch/certs -R

Switch to the certificate directory.

$ cd /etc/opensearch/certs

Generate Root Certificates

Generate a private key for the root certificate.

$ openssl genrsa -out root-ca-key.pem 2048

Generate a self-signed root CA certificate. We are using the -subj parameter to provide information that is embedded in the certificate. If you don’t use the -subj flag, you will be asked for all this information.

$ openssl req -new -x509 -sha256 -key root-ca-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=ROOT" -out root-ca.pem -days 730

You can change the information to suit your specific host.

The variables in the -subj parameter mean the following.

C = Country Name (2 letter code), ST = State or Province Name (Full name), L = Locality Name (e.g, city), O = Organization Name, OU = Organizational Unit Name (e.g, section), CN = Common Name (e.g., server FQDN)

Your root certificate is now live.

Generate Admin Certificates

Generate the private key for the Admin certificate using the following command.

$ openssl genrsa -out admin-key-temp.pem 2048

Convert the private key to the PKCS#8 format for use in Java using a PKCS#12 compatible algorithm(3DES).

$ openssl pkcs8 -inform PEM -outform PEM -in admin-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out admin-key.pem

Generate the Admin CSR (Certificate Signing Request) from the private key. Since this certificate is not tied to any hosts and is used for authenticating elevated access, we are using a value of A for the Common Name (CN).

$ openssl req -new -key admin-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=A" -out admin.csr

Sign the Admin CSR with the root certificate and private key you just created.

$ openssl x509 -req -in admin.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out admin.pem -days 730
Certificate request self-signature ok
subject=C = CA, ST = ONTARIO, L = TORONTO, O = ORG, OU = UNIT, CN = A

Generate Node Certificates

Generate a private key for the node certificate.

$ openssl genrsa -out node1-key-temp.pem 2048

Convert the node private key to the PKCS#8 format.

$ openssl pkcs8 -inform PEM -outform PEM -in node1-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out node1-key.pem

Create a new CSR for the Node certificate from the private key. Make sure to change the value of CN variable with the hostname of your node. This certificate is tied to your hosts and it should match with the hostname or the IP address of the node.

$ openssl req -new -key node1-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=opensearch.example.com" -out node1.csr

Create a SAN (Subject Alternative Name) extension file that contains the node hostname or FQDN or IP address.

$ sh -c 'echo subjectAltName=DNS:opensearch.example.com > node1.ext'

Sign the node certificate CSR with the root certificate and private key you just created.

$ openssl x509 -req -in node1.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out node1.pem -days 730 -extfile node1.ext
Certificate request self-signature ok
subject=C = CA, ST = ONTARIO, L = TORONTO, O = ORG, OU = UNIT, CN = opensearch.example.com

Preparing Certificates for Installation

Remove any temporary certificate, CSR, and the SAN extension file.

$ rm -f *temp.pem *csr *ext

Next, we will install the root CA certificate on the server. For that, convert the root CA certificate to the .crt format.

$ openssl x509 -outform der -in root-ca.pem -out root-ca.crt

Copy the root-ca.crt file to the /usr/local/share/ca-certificates/ directory.

$ sudo cp root-ca.crt /usr/local/share/ca-certificates/

Add the root CA certificate to your Debian server using the following command.

$ sudo update-ca-certificates

You should see the following output ( 1 added) confirming that the root CA certificate is added to your system.

Updating certificates in /etc/ssl/certs...
rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
rehash: warning: skipping root-ca.pem,it does not contain exactly one certificate or CRL
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

Change the ownership of the directory back to the opensearch user.

$ sudo chown opensearch:opensearch /etc/opensearch/certs -R

Change the permission on the directory to 0700.

$ sudo chmod 0700 /etc/opensearch/certs

Change the permission on the certificates to 0600.

$ sudo chmod 0600 /etc/opensearch/certs/{admin-key,admin,node1-key,node1,root-ca-key,root-ca}.pem
$ sudo chmod 0600 /etc/opensearch/certs/root-ca.crt

Step 7 – Install TLS to OpenSearch

Now that all the certificates are generated and set up, it is time to install and add the certificates to your OpenSearch installation.

The first step is to create a bash script that will add all the created certificates and TLS security plugin settings to the OpenSearch configuration file (/etc/opensearch/opensearch.yml).

Create a file named add-opensearch-tls.sh and open it for editing.

$ cd ~
$ nano add-opensearch-tls.sh

Paste the following code in it.

#! /bin/bash

# Before running this script, make sure to replace the CN in the 
# node's distinguished name with a real DNS A record.

echo "plugins.security.ssl.transport.pemcert_filepath: /etc/opensearch/certs/node1.pem" | sudo tee -a /etc/opensearch/opensearch.yml
echo "plugins.security.ssl.transport.pemkey_filepath: /etc/opensearch/certs/node1-key.pem" | sudo tee -a /etc/opensearch/opensearch.yml
echo "plugins.security.ssl.transport.pemtrustedcas_filepath: /etc/opensearch/certs/root-ca.pem" | sudo tee -a /etc/opensearch/opensearch.yml
echo "plugins.security.ssl.http.enabled: true" | sudo tee -a /etc/opensearch/opensearch.yml
echo "plugins.security.ssl.http.pemcert_filepath: /etc/opensearch/certs/node1.pem" | sudo tee -a /etc/opensearch/opensearch.yml
echo "plugins.security.ssl.http.pemkey_filepath: /etc/opensearch/certs/node1-key.pem" | sudo tee -a /etc/opensearch/opensearch.yml
echo "plugins.security.ssl.http.pemtrustedcas_filepath: /etc/opensearch/certs/root-ca.pem" | sudo tee -a /etc/opensearch/opensearch.yml
echo "plugins.security.allow_default_init_securityindex: true" | sudo tee -a /etc/opensearch/opensearch.yml
echo "plugins.security.authcz.admin_dn:" | sudo tee -a /etc/opensearch/opensearch.yml
echo "  - 'CN=A,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'" | sudo tee -a /etc/opensearch/opensearch.yml
echo "plugins.security.nodes_dn:" | sudo tee -a /etc/opensearch/opensearch.yml
echo "  - 'CN=opensearch.example.com,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'" | sudo tee -a /etc/opensearch/opensearch.yml
echo "plugins.security.audit.type: internal_opensearch" | sudo tee -a /etc/opensearch/opensearch.yml
echo "plugins.security.enable_snapshot_restore_privilege: true" | sudo tee -a /etc/opensearch/opensearch.yml
echo "plugins.security.check_snapshot_restore_write_privileges: true" | sudo tee -a /etc/opensearch/opensearch.yml
echo "plugins.security.restapi.roles_enabled: [\"all_access\", \"security_rest_api_access\"]" | sudo tee -a /etc/opensearch/opensearch.yml

Save the file by pressing Ctrl + X and entering Y when prompted.

Make the file executable.

$ chmod +x add-opensearch-tls.sh

Run the script.

$ ./add-opensearch-tls.sh

You should get the following output.

plugins.security.ssl.transport.pemcert_filepath: /etc/opensearch/certs/node1.pem
plugins.security.ssl.transport.pemkey_filepath: /etc/opensearch/certs/node1-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: /etc/opensearch/certs/root-ca.pem
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: /etc/opensearch/certs/node1.pem
plugins.security.ssl.http.pemkey_filepath: /etc/opensearch/certs/node1-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: /etc/opensearch/certs/root-ca.pem
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - 'CN=A,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
plugins.security.nodes_dn:
  - 'CN=opensearch.example.com,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]

Open the file /etc/opensearch/opensearch.yml file to check whether your settings have been applied.

$ sudo nano /etc/opensearch/opensearch.yml

Scroll to the bottom of the file and you should see the following.

# If you previously disabled the Security plugin in opensearch.yml,
# be sure to re-enable it. Otherwise you can skip this setting.
plugins.security.disabled: false
....
plugins.security.ssl.transport.pemcert_filepath: /etc/opensearch/certs/node1.pem
plugins.security.ssl.transport.pemkey_filepath: /etc/opensearch/certs/node1-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: /etc/opensearch/certs/root-ca.pem
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: /etc/opensearch/certs/node1.pem
plugins.security.ssl.http.pemkey_filepath: /etc/opensearch/certs/node1-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: /etc/opensearch/certs/root-ca.pem
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - 'CN=A,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
plugins.security.nodes_dn:
  - 'CN=opensearch.example.com,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]

Once you are satisfied, save the file by pressing Ctrl + X.

Step 8 – Creating OpenSearch User

In this step, we will remove all the demo users except for the admin user and replace the default admin password using a script.

Switch to the OpenSearch security plugins tools directory.

$ cd /usr/share/opensearch/plugins/opensearch-security/tools

Run the hash.sh script to generate a new password.

$ ./hash

The script will fail and you will get the following error because the path to the Java executable (JDK) has not been defined.

**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
WARNING: nor OPENSEARCH_JAVA_HOME nor JAVA_HOME is set, will use 
./hash.sh: line 35: java: command not found

Run the command again by declaring the environment variable OPENSEARCH_JAVA_HOME with the location of the JDK.

$ OPENSEARCH_JAVA_HOME=/usr/share/opensearch/jdk ./hash.sh

You will get a similar output.

**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
[Password:]
$2y$12$4NL0LqEnN4FXaUdQQC.gOupeCQT2IuMo9gN0b5d3nxdMVbLX088f2

It will generate the password hash which we will need later.

Open the OpenSearch User configuration file internal_users.yml in the /etc/opensearch/opensearch-security/ directory for editing.

$ sudo nano /etc/opensearch/opensearch-security/internal_users.yml

Comment out all the demo users by adding hash in front of them as shown.

---
# This is the internal user database
# The hash value is a bcrypt hash and can be generated with plugin/tools/hash.sh

_meta:
  type: "internalusers"
  config_version: 2

# Define your internal users here

## Demo users

#admin:
#  hash: "$2y$12$Twy84uxOX0SQ1QMQ68og8O/i1MvqWItt/phZX3DiJMjpBoT9u9KX6"
#  reserved: true
#  backend_roles:
#  - "admin"
#  description: "Demo admin user"

#anomalyadmin:

Add the admin user with the hash you generated above as shown.

---
# This is the internal user database
# The hash value is a bcrypt hash and can be generated with plugin/tools/hash.sh

_meta:
  type: "internalusers"
  config_version: 2

# Define your internal users here

admin:
  hash: "$2y$12$4NL0LqEnN4FXaUdQQC.gOupeCQT2IuMo9gN0b5d3nxdMVbLX088f2"
  reserved: true
  backend_roles:
  - "admin"
  description: "Admin user"

## Demo users

#admin:

Once finished, save the file by pressing Ctrl + X and entering Y when prompted.

Now that the TLS certificates are installed and the demo users removed or assigned new passwords, it is time to apply the configuration changes. To apply the changes, we will invoke the securityadmin.sh script inside the /usr/share/opensearch/plugins/opensearch-security/tools directory. The script requires the OpenSearch service to run.

Restart the OpenSearch service.

$ sudo systemctl restart opensearch

Switch to the /usr/share/opensearch/plugins/opensearch-security/tools directory.

$ cd /usr/share/opensearch/plugins/opensearch-security/tools

Run the script using the following command.

$ sudo OPENSEARCH_JAVA_HOME=/usr/share/opensearch/jdk ./securityadmin.sh -h opensearch.nspeaks.com -p 9200 -cd /etc/opensearch/opensearch-security/ -cacert /etc/opensearch/certs/root-ca.pem -cert /etc/opensearch/certs/admin.pem -key /etc/opensearch/certs/admin-key.pem -icl -nhnv

The security plugin stores its configuration – including users, permissions, roles, and backend settings in a system index on the OpenSearch cluster. Storing these settings in the cluster allows you to change settings without restarting the cluster and eliminates the need to edit the configuration file on every node. This is accomplished by running the securityadmin.sh script.

  • The -cd option specifies where the security plugin configuration files are located.
  • The -icl (--ignore-clustername) option tells the security plugin to upload the configuration regardless of the cluster name. You can also specify the cluster name instead of using the -cn (--clustername) option.
  • Since the certificates are self-signed, we use the -nhnv (--disable-host-name-verification) option.
  • The --cacert, --cert, and --key options define the location of the root certificate, the admin certificate, and the private key for the admin certificate. If the private key has a password, you can define it using the -keypass option.

You can learn more about the security plugin configuration in the OpenSearch documentation.

You should have gotten a similar output as shown below with the above command.

**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
Security Admin v7
Will connect to opensearch:9200 ... done
Connected as "CN=A,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA"
OpenSearch Version: 2.12.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: opensearch
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index already exists, so we do not need to create one.
Populate config from /etc/opensearch/opensearch-security/
Will update '/config' with /etc/opensearch/opensearch-security/config.yml 
   SUCC: Configuration for 'config' created or updated
Will update '/roles' with /etc/opensearch/opensearch-security/roles.yml 
   SUCC: Configuration for 'roles' created or updated
Will update '/rolesmapping' with /etc/opensearch/opensearch-security/roles_mapping.yml 
   SUCC: Configuration for 'rolesmapping' created or updated
Will update '/internalusers' with /etc/opensearch/opensearch-security/internal_users.yml 
   SUCC: Configuration for 'internalusers' created or updated
Will update '/actiongroups' with /etc/opensearch/opensearch-security/action_groups.yml 
   SUCC: Configuration for 'actiongroups' created or updated
Will update '/tenants' with /etc/opensearch/opensearch-security/tenants.yml 
   SUCC: Configuration for 'tenants' created or updated
Will update '/nodesdn' with /etc/opensearch/opensearch-security/nodes_dn.yml 
   SUCC: Configuration for 'nodesdn' created or updated
Will update '/whitelist' with /etc/opensearch/opensearch-security/whitelist.yml 
   SUCC: Configuration for 'whitelist' created or updated
Will update '/audit' with /etc/opensearch/opensearch-security/audit.yml 
   SUCC: Configuration for 'audit' created or updated
Will update '/allowlist' with /etc/opensearch/opensearch-security/allowlist.yml 
   SUCC: Configuration for 'allowlist' created or updated
SUCC: Expected 10 config types for node {"updated_config_types":["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"],"updated_config_size":10,"message":null} is 10 (["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"]) due to: null
Done with success

Verify the new credentials using the following command.

During the previous test, we directed requests to localhost. Now that the TLS certificates are set up and the new certificates refer to the host’s actual DNS record, the requests to localhost will fail the CN check, and the certificate will be considered invalid. Therefore, add your actual DNS record in the command below to verify.

$ curl https://your.host.address:9200 -u admin:yournewpassword -k

You should see the following response.

{
  "name" : "opensearch",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "3oFvYn96St2MOpDP3Sf29g",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.12.0",
    "build_type" : "deb",
    "build_hash" : "2c355ce1a427e4a528778d4054436b5c4b756221",
    "build_date" : "2024-02-20T02:18:31.541484890Z",
    "build_snapshot" : false,
    "lucene_version" : "9.9.2",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

Step 9 – Install OpenSearch Dashboards

Since we installed OpenSearch already, we can skip the step of importing the OpenSearch GPG key.

To install OpenSearch dashboards, run the following command to create an APT repository for it.

$ echo "deb [signed-by=/usr/share/keyrings/opensearch-keyring] https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/2.x/apt stable main" | sudo tee /etc/apt/sources.list.d/opensearch-dashboards-2.x.list

Update the system repository list.

$ sudo apt update

List all the available versions of OpenSearch Dashboards.

$ sudo apt list -a opensearch-dashboards

You should see a similar output.

Listing... Done
opensearch-dashboards/stable 2.12.0 amd64
opensearch-dashboards/stable 2.11.1 amd64
opensearch-dashboards/stable 2.11.0 amd64
opensearch-dashboards/stable 2.10.0 amd64
opensearch-dashboards/stable 2.9.0 amd64
opensearch-dashboards/stable 2.8.0 amd64
opensearch-dashboards/stable 2.7.0 amd64
opensearch-dashboards/stable 2.6.0 amd64
opensearch-dashboards/stable 2.5.0 amd64

Install the latest version of OpenSearch Dashboard.

$ sudo apt install opensearch-dashboards

Or install a specific version of OpenSearch Dashboard.

$ sudo apt install opensearch-dashboards=2.12.0

The version of OpenSearch dashboards and OpenSearch should be the same otherwise you can face multiple issues.

Reload the service daemon.

$ sudo systemctl daemon-reload

Enable the OpenSearch Dashboards service.

$ sudo systemctl enable opensearch-dashboards

Start the OpenSearch Dashboards service.

$ sudo systemctl start opensearch-dashboards

Verify the status of the OpenSearch Dashboards service.

$ sudo systemctl status opensearch-dashboards

You should see the following output.

? opensearch-dashboards.service - "OpenSearch Dashboards"
     Loaded: loaded (/lib/systemd/system/opensearch-dashboards.service; enabled; preset: enabled)
     Active: active (running) since Sat 2024-03-02 16:01:25 UTC; 4s ago
   Main PID: 7098 (node)
      Tasks: 11 (limit: 9484)
     Memory: 175.4M
        CPU: 5.389s
     CGroup: /system.slice/opensearch-dashboards.service
             ??7098 /usr/share/opensearch-dashboards/node/bin/node /usr/share/opensearch-dashboards/src/cli/dist

Mar 02 16:01:25 opensearch systemd[1]: Started opensearch-dashboards.service - "OpenSearch Dashboards".

Step 10 – Configure OpenSearch Dashboards

OpenSearch Dashboards stores its configuration in the /etc/opensearch-dashboards/opensearch_dashboards.yml file. Open it for editing.

$ sudo nano /etc/opensearch-dashboards/opensearch_dashboards.yml

Find the line # server.host: "localhost" and uncomment it by removing the hash in front of it and changing its value as shown.

# Specifies the address to which the OpenSearch Dashboards server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: 0.0.0.0

Setting the value of server.host to 0.0.0.0 binds OpenSearch Dashboard to any external IP which makes it accessible over the World Wide Web. If you want to serve the Dashboards over SSL using a reverse proxy, then leave this value unchanged.

Scroll down to find the following section.

opensearch.hosts: [https://localhost:9200]
opensearch.ssl.verificationMode: none
opensearch.username: kibanaserver
opensearch.password: kibanaserver
opensearch.requestHeadersWhitelist: [authorization, securitytenant]

Change the values as configured for our OpenSearch installation.

opensearch.hosts: [https://opensearch:9200]
opensearch.ssl.verificationMode: none
opensearch.username: admin
opensearch.password: <custom-admin-password>
opensearch.requestHeadersWhitelist: [authorization, securitytenant]

Once finished, save the file by pressing Ctrl + X and entering Y when prompted.

Restart the OpenDashboards service.

$  sudo systemctl restart opensearch-dashboards

Open port 5601 in the UFW firewall.

$ sudo ufw allow 5601

Open the URL http://<yourserverIP>:5601 or http://opensearch.example.com:5601 to launch the OpenSearch Dashboards website.

If you are content accessing OpenSearch Dashboards over HTTP and with a port number in the URL, you can stop here. However, it is better to serve Dashboards over SSL. For this, we will install Nginx as a reverse proxy and serve SSL over it.

Step 11 – Install Nginx

Debian 12 ships with an older version of Nginx. You need to download the official Nginx repository to install the latest version.

Import Nginx’s signing key.

$ curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor \
    | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null

Add the repository for Nginx’s mainline version.

$ echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] \
http://nginx.org/packages/mainline/debian `lsb_release -cs` nginx" \
    | sudo tee /etc/apt/sources.list.d/nginx.list

Update the system repositories.

$ sudo apt update

Install Nginx.

$ sudo apt install nginx

Verify the installation. On Debian systems, the following command will only work with sudo.

$ sudo nginx -v
nginx version: nginx/1.25.4

Start the Nginx server.

$ sudo systemctl start nginx

Check the service status.

$ sudo systemctl status nginx
? nginx.service - nginx - high performance web server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; preset: enabled)
     Active: active (running) since Sat 2024-03-02 12:53:57 UTC; 4s ago
       Docs: https://nginx.org/en/docs/
    Process: 3976 ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf (code=exited, status=0/SUCCESS)
   Main PID: 3977 (nginx)
      Tasks: 3 (limit: 9484)
     Memory: 2.7M
        CPU: 9ms
     CGroup: /system.slice/nginx.service
             ??3977 "nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf"
             ??3978 "nginx: worker process"
             ??3979 "nginx: worker process"

Mar 02 12:53:57 opensearch systemd[1]: Starting nginx.service - nginx - high performance web server...
Mar 02 12:53:57 opensearch systemd[1]: Started nginx.service - nginx - high performance web server.

Step 12 – Install SSL

We need to install Certbot to generate the SSL certificate. You can either install Certbot using Debian’s repository or grab the latest version using the Snapd tool. We will be using the Snapd version.

Debian 12 comes doesn’t come with Snapd installed. Install Snapd package.

$ sudo apt install snapd

Run the following commands to ensure that your version of Snapd is up to date.

$ sudo snap install core && sudo snap refresh core

Install Certbot.

$ sudo snap install --classic certbot

Use the following command to ensure that the Certbot command can be run by creating a symbolic link to the /usr/bin directory.

$ sudo ln -s /snap/bin/certbot /usr/bin/certbot

Verify if Certbot is functioning correctly.

$ certbot --version
certbot 2.9.0

Run the following command to generate an SSL Certificate.

$ sudo certbot certonly --nginx --agree-tos --no-eff-email --staple-ocsp --preferred-challenges http -m [email protected] -d craftcms.example.com

The above command will download a certificate to the /etc/letsencrypt/live/craftcms.example.com directory on your server.

Generate a Diffie-Hellman group certificate.

$ sudo openssl dhparam -dsaparam -out /etc/ssl/certs/dhparam.pem 4096

Check the Certbot renewal scheduler service.

$ sudo systemctl list-timers

You will find snap.certbot.renew.service as one of the services scheduled to run.

NEXT                        LEFT           LAST                        PASSED    UNIT                         ACTIVATES                   ---------------------------------------------------------------------------------------------------------------------------------------  
Sat 2024-03-02 16:25:00 UTC 3h 27min left  -                           -         snap.certbot.renew.timer     snap.certbot.renew.service
Sat 2024-03-02 23:00:51 UTC 10h left       Sat 2024-03-02 06:29:02 UTC 6h ago    apt-daily.timer              apt-daily.service
Sun 2024-03-03 00:00:00 UTC 11h left       -                           -         dpkg-db-backup.timer         dpkg-db-backup.service
.....

Do a dry run of the process to check whether the SSL renewal is working fine.

$ sudo certbot renew --dry-run

If you see no errors, you are all set. Your certificate will renew automatically.

Step 13 – Configure Nginx

Open the file /etc/nginx/nginx.conf for editing.

$ sudo nano /etc/nginx/nginx.conf

Add the following line before the line include /etc/nginx/conf.d/*.conf;.

server_names_hash_bucket_size 64;

Save the file by pressing Ctrl + X and entering Y when prompted.

Create and open the file /etc/nginx/conf.d/opensearch.conf for editing.

$ sudo nano /etc/nginx/conf.d/opensearch.conf

Paste the following code in it. Replace opensearch.example.com with your domain name.

server {

    listen 443 ssl;
    listen [::]:443 ssl;

    http2 on;
    http3 on;
    quic_retry on;

    server_name opensearch.example.com;

    access_log  /var/log/nginx/opensearch.access.log;
    error_log   /var/log/nginx/opensearch.error.log;

    ssl_certificate      /etc/letsencrypt/live/opensearch.example.com/fullchain.pem;
    ssl_certificate_key  /etc/letsencrypt/live/opensearch.example.com/privkey.pem;
    ssl_trusted_certificate /etc/letsencrypt/live/opensearch.example.com/chain.pem;

    ssl_session_timeout  5m;
    ssl_session_cache shared:MozSSL:10m;
    ssl_session_tickets off;
    ssl_early_data on;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
    ssl_ecdh_curve X25519:prime256v1:secp384r1:secp521r1;
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_dhparam /etc/ssl/certs/dhparam.pem;

    location / {
        proxy_pass http://localhost:5601;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
# enforce HTTPS
server {
    listen 80;
    listen [::]:80;
    server_name  opensearch.example.com;
    return 301   https://$host$request_uri;
}

Save the file by pressing Ctrl + X and entering Y when prompted.

Verify your Nginx configuration.

$ sudo nginx -t

Restart the Nginx server.

$ sudo systemctl restart nginx

Visit https://opensearch.example.com in your browser to launch OpenSearch Dashboards.

Step 14 – Access and use OpenSearch Dashboards

If you used Nginx, then access the Dashboards via the URL https://opensearch.example.com or use http://opensearch.example.com:5601 and you should get the login page.

Enter admin and the custom admin password you configured for OpenSearch in step 4 and click the Log in button to proceed.

Click the Add data button to start adding data directly or click the Explore on my own link if you want to browse first. We will browse first and it should take you to a popup that talks about OpenSearch Dashboards’ dark mode.

Click the Dismiss button to proceed. Next, you will be asked to choose a tenant.

Tenants in OpenSearch Dashboards are spaces for saving index patterns, visualizations, dashboards, and other OpenSearch Dashboards objects. Each user can create multiple tenants for multiple uses. Tenants are useful for safely sharing your work with other OpenSearch Dashboards’ users. OpenSearch gives you the option of choosing between three options.

  • Global – This tenant is shared between every OpenSearch Dashboards user.
  • Private – This tenant is exclusive to each user and can’t be shared.
  • Custom – Administrators can create custom tenants and assign them to specific roles. Once created, you can use these tenants to provide spaces to a specific set of users.

For now, we will stick with the default option of a Private tenant. Click the Confirm button to proceed. You will be greeted with the following page.

Next, we will verify the connection between OpenSearch and the OpenSearch Dashboards.

Click the Dev Tools button on the top right and you will be shown a mini popup about the console UI for the OpenSearch Dashboards.

Click the Dismiss button, input the query GET / in the console on the left, and click the small play button in the middle. You should see the following output on the right.

This confirms that OpenSearch Dashboards is receiving data from OpenSearch and both are working fine. You can start adding data from here on.

Conclusion

This concludes the tutorial on installing OpenSearch on a Debian 12 server. If you have any questions, post them in the comments below.


RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here


Most Popular