This is the support thread for open search. I needed this application to get may graylog up and running. 

 

OpenSearch is a scalable, flexible, and extensible open-source software suite for search, analytics, and observability applications derived from Elasticsearch 7.10.2 and Kibana 7.10.2 and licensed under Apache 2.0. It consists of a search engine daemon (OpenSearch) and a user interface for data administration and visualizations (OpenSearch Dashboards).

Make sure Opensearch has access rights to the "Log Data" folder. Permissions may need to be fixed with integrated Unraid File Manager.

Steps below MUST be used in order to get OpenSearch working correctly. Check support thread for more information.

        1. Install CA User Scripts
        2. Create a new script named "set_max_map_count"
        3. Contents of script as follows:

                     #!/bin/bash
                     sysctl -w vm.max_map_count=262144

        4. Set script schedule to At Startup of Array

 

I am happy to help if you have any problem setting this up. 

Edited May 8, 20242 yr by Richy1989

I have set this up several times, but no amount of configurations on logging agents ever make their way to the OpenSearch stack. 

Any tips?

How do you fix permissions for data log folder? I set owner to nobody user but OS is using opensearch user 

On 8/6/2024 at 7:02 AM, nekromantik said:

How do you fix permissions for data log folder? I set owner to nobody user but OS is using opensearch user 

The container comes with a default data directory of:

 

 /mnt/user/database/opensearch/log_data

 

In default Unraid configuration, I'd recommend changing this to ensure it's in the default docker location, updating the the Log Data configuration option:

/mnt/user/appdata/opensearch/log_data

 

Whether you do this or not - the fix for the permissions is :

 

chown 1000:1000 <path to data log folder>

 

so using the updated Log Data option:

chown 1000:1000 /mnt/user/appdata/opensearch/log_data

 

or for the default:

 

chown 1000:1000 /mnt/user/database/opensearch/log_data

 

The easiest way to find out how to fix this if you have this problem is to check what user id (uid) and group id (gid) are being used :

 

docker exec opensearch id

 

This will run the id command within the container then exit, printing out details of the user the container is running as:

➜  docker exec opensearch id
uid=1000(opensearch) gid=1000(opensearch) groups=1000(opensearch)

On 5/8/2024 at 7:39 PM, TQ said:

I have set this up several times, but no amount of configurations on logging agents ever make their way to the OpenSearch stack. 

Any tips?

For whatever reason OpenSearch is not configured to automatically create indexes, hence your data will try and send and not find template definition that it needs to be in OpenSearch, and so fails as it doesn't know how to store the data.

 

You can luckily solve this pretty easily.

 

Either update the settings for the Docker container in the Unraid UI and change:

 

action.auto_create_index from false to true

 

and restart the container after saving changes and that should do it.

 

---------

More complicated and unnecessary ways: 

 

If you want to do it under the hood opensearch.yml file and set :

 

action.auto_create_index: "true"

 

or after install OpenSearch Dashboards goto DevTools and put the following in 

 

PUT _cluster/settings
{
  "persistent": {
    "action.auto_create_index": "true"
  }
}

 

(You can confirm current settings using the following to validate that auto_create_index: "false"):
 

GET _cluster/settings?include_defaults=true

 

Or do it via curl:
 

curl -k -u "admin:YOUR_PASSWORD" -X GET "http://YOUR_OPENSEARCH_HOST:9200/_cluster/settings?include_defaults=true"

 

or set it via curl:

 

curl -k -u "admin:YOUR_PASSWORD" -X PUT "http://YOUR_OPENSEARCH_HOST:9200/_cluster/settings" \
  -H 'Content-Type: application/json' \
  -d '{
    "persistent": {
      "action.auto_create_index": "true"
    }
  }'

Edited March 27, 20251 yr by niavasha