[Support] d8sychain - Elasticsearch


Recommended Posts

This Elasticsearch docker version is built from the official Elasticsearch docker with minimal changes tailoring it for easier use on Unraid and

for the purpose for use with Nextcloud.

 

*Minimal support will be provided for this docker.

99% of this docker is based from the Elasticsearch official docker https://hub.docker.com/_/elasticsearch with very minimal charges made.

Example of provided support:

  1. Requesting that the base image get bumped to a newer version of Elasticsearch and tested before releasing
  2. Help with installation incase something in the directions are not clearly understood.

 

Installation

Directions below MUST be used in order to get Elasticsearch 5 and above working correctly.

Directions:

1. Install CA User Scripts

2. Create a new script named vm.max_map_count

3. Contents of script as follows:

#!/bin/bash 

sysctl -w vm.max_map_count=262144

4. Set script schedule to At Startup of Array.

5. Add the docker container.

 

If you are using this for Nextcloud then continue...

The ingest-attachment plugin is added by default at the 1st start of the container

 

6. Browse to the Admin UI in Nextcloud and config according https://github.com/nextcloud/fulltextsearch/wiki/Basic-Installation.

7. Browse to the Unraid docker tab, click the nextcloud docker icon then Console or Open a SSH Terminal (like PuTTY) and login in to your Unraid server and bash into the container docker exec -it nextcloud /bin/bash.

If you are using linuxserver.io's Nextcloud docker the user is abc otherwise the default is www-data and the path to occ is /config/www/nextcloud/

8. Enter:

nextcloud default 

sudo -u www-data php <path/to/nextcloud install>/occ fulltextsearch:index

linuxserver.io nextcloud docker 

sudo -u abc php /config/www/nextcloud/occ fulltextsearch:index

9. And wait... This can take a while depending on the number of files in your nextcloud server.


 

The following is an excerpt from https://github.com/nextcloud/fulltextsearch/wiki/Commands

fulltextsearch:index

This is the main command of the app, which is used to index your content. You will need to use this command to index your files at least once.

To start your first index:

./occ fulltextsearch:index

Options can be pushed using JSON:

user/users: user/array of users that will be indexed.

provider/providers: provider/array of providers that will be used to retrieve content of the Nextcloud.

path: location, can be a file or a folder. Used by the Files content provider only. Limit the index to the files from this location. If the file or folder does not exist, the index will be over.

paused: true/false. If set to true, index will start paused.

errors: reset Reset the errors

./occ fulltextsearch:index "{\"user\": \"cult\", \"providers\":[\"files\", \"test\"]}"

 

Adding and removing plugins

In the docker volume host path, the default is

/mnt/user/appdata/elasticsearch

contains file my-plugins.txt

Edit this file, adding and/or removing needed plugins, and then restart the container.

A built-in script will check this file at container start and compare it to the currently installed plugins and install and/or remove plugins accordingly.

Link to comment
  • 4 months later...
  • 5 months later...
  • 5 months later...

Hi, using the default settings, any idea what would be causing permission related issues?

 

mkdir: cannot create directory '/config/data': Permission denied
mkdir: cannot create directory '/config/logs': Permission denied
mkdir: cannot create directory '/config/jvm.options.d': Permission denied
chown: changing ownership of '/config': Operation not permitted
cp: cannot create regular file '/config/elasticsearch.yml': Permission denied
cp: cannot create regular file '/config/jvm.options': Permission denied
cp: cannot create regular file '/config/log4j2.properties': Permission denied
cp: cannot create regular file '/config/my-plugins.txt': Permission denied
comm: /config/my-plugins.txt: No such file or directory
comm: /config/my-plugins.txt: No such file or directory
Exception in thread "main" java.nio.file.NoSuchFileException: /config/jvm.options
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:375)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:426)
        at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420)
        at java.base/java.nio.file.Files.newInputStream(Files.java:160)
        at org.elasticsearch.tools.launchers.JvmOptionsParser.readJvmOptionsFiles(JvmOptionsParser.java:167)
        at org.elasticsearch.tools.launchers.JvmOptionsParser.jvmOptions(JvmOptionsParser.java:128)
        at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:95)

** Press ANY KEY to close this window ** 

 

  • Upvote 2
Link to comment
  • 2 months later...

I have the same problem and so far no solution. Can anyone help? Frustrating trying to run elasticsearch with unraid.

 

mkdir: cannot create directory '/config/logs': Permission denied
mkdir: cannot create directory '/config/jvm.options.d': Permission denied
chown: changing ownership of '/config/data': Operation not permitted
chown: changing ownership of '/config': Operation not permitted
cp: cannot create regular file '/config/elasticsearch.yml': Permission denied
cp: cannot create regular file '/config/jvm.options': Permission denied
cp: cannot create regular file '/config/log4j2.properties': Permission denied
cp: cannot create regular file '/config/my-plugins.txt': Permission denied
comm: /config/my-plugins.txt: No such file or directory
comm: /config/my-plugins.txt: No such file or directory
Exception in thread "main" java.nio.file.NoSuchFileException: /config/jvm.options
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:375)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:426)
        at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420)
        at java.base/java.nio.file.Files.newInputStream(Files.java:160)
        at org.elasticsearch.tools.launchers.JvmOptionsParser.readJvmOptionsFiles(JvmOptionsParser.java:167)
        at org.elasticsearch.tools.launchers.JvmOptionsParser.jvmOptions(JvmOptionsParser.java:128)
        at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:95)

** Press ANY KEY to close this window ** 

 

Link to comment
On 7/30/2022 at 8:32 PM, AyslanAlves said:

I have the same problem and so far no solution. Can anyone help? Frustrating trying to run elasticsearch with unraid.

 

mkdir: cannot create directory '/config/logs': Permission denied
mkdir: cannot create directory '/config/jvm.options.d': Permission denied
chown: changing ownership of '/config/data': Operation not permitted
chown: changing ownership of '/config': Operation not permitted
cp: cannot create regular file '/config/elasticsearch.yml': Permission denied
cp: cannot create regular file '/config/jvm.options': Permission denied
cp: cannot create regular file '/config/log4j2.properties': Permission denied
cp: cannot create regular file '/config/my-plugins.txt': Permission denied
comm: /config/my-plugins.txt: No such file or directory
comm: /config/my-plugins.txt: No such file or directory
Exception in thread "main" java.nio.file.NoSuchFileException: /config/jvm.options
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:375)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:426)
        at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420)
        at java.base/java.nio.file.Files.newInputStream(Files.java:160)
        at org.elasticsearch.tools.launchers.JvmOptionsParser.readJvmOptionsFiles(JvmOptionsParser.java:167)
        at org.elasticsearch.tools.launchers.JvmOptionsParser.jvmOptions(JvmOptionsParser.java:128)
        at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:95)

** Press ANY KEY to close this window ** 

 

 

 

Also the same issue.  

Link to comment
  • 2 weeks later...
  • 2 weeks later...
  • 1 month later...
On 5/27/2022 at 9:32 PM, wolfNZ said:

Hi, using the default settings, any idea what would be causing permission related issues?

 

mkdir: cannot create directory '/config/data': Permission denied
mkdir: cannot create directory '/config/logs': Permission denied
mkdir: cannot create directory '/config/jvm.options.d': Permission denied
chown: changing ownership of '/config': Operation not permitted
cp: cannot create regular file '/config/elasticsearch.yml': Permission denied
cp: cannot create regular file '/config/jvm.options': Permission denied
cp: cannot create regular file '/config/log4j2.properties': Permission denied
cp: cannot create regular file '/config/my-plugins.txt': Permission denied
comm: /config/my-plugins.txt: No such file or directory
comm: /config/my-plugins.txt: No such file or directory
Exception in thread "main" java.nio.file.NoSuchFileException: /config/jvm.options
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:375)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:426)
        at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420)
        at java.base/java.nio.file.Files.newInputStream(Files.java:160)
        at org.elasticsearch.tools.launchers.JvmOptionsParser.readJvmOptionsFiles(JvmOptionsParser.java:167)
        at org.elasticsearch.tools.launchers.JvmOptionsParser.jvmOptions(JvmOptionsParser.java:128)
        at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:95)

** Press ANY KEY to close this window ** 

 

Change the permissions on the appdata folder
 

chmod 777 ./elasticsearch/

 

  • Like 2
Link to comment
  • 1 month later...
  • 4 months later...

Running v6.12.0-rc4.1

 

Tested both elasticsearch dockers available in community applications for versions 6.6.2 and 7.10.2.

 

Up to this point everything appears to be working. When i run ./occ fulltextsearch:test at /var/www/html/, in the Nextcloud v26.0.1 container i receive the error:

 

.Testing your current setup:  
Creating mocked content provider. ok  
Testing mocked provider: get indexable documents. (2 items) ok  
Loading search platform. (Elasticsearch) ok  
Testing search platform. fail
In ProductCheckTrait.php line 32:
                                                                                                      
  The client noticed that the server is not Elasticsearch and we do not support this unknown product  
                                                                                                      

fulltextsearch:test [--output [OUTPUT]] [-j|--json] [-d|--platform_delay PLATFORM_DELAY]

 

Is this a known issue? Or does this have anything to do with licensing / forking the project? I have not tried a more recent version of the elasticsearch yet.

 

Thank you.

Edited by chris1259
  • Upvote 2
Link to comment
  • 2 months later...

https://github.com/jemalloc/jemalloc/issues/1328

# WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.

 

where is sysctl.conf ? I couldn't find it in the container.

Link to comment
  • 4 weeks later...
On 5/13/2023 at 3:36 AM, chris1259 said:

Running v6.12.0-rc4.1

 

Tested both elasticsearch dockers available in community applications for versions 6.6.2 and 7.10.2.

 

Up to this point everything appears to be working. When i run ./occ fulltextsearch:test in the Nextcloud v26.0.1 container i receive the error:

 

.Testing your current setup:  
Creating mocked content provider. ok  
Testing mocked provider: get indexable documents. (2 items) ok  
Loading search platform. (Elasticsearch) ok  
Testing search platform. fail
In ProductCheckTrait.php line 32:
                                                                                                      
  The client noticed that the server is not Elasticsearch and we do not support this unknown product  
                                                                                                      

fulltextsearch:test [--output [OUTPUT]] [-j|--json] [-d|--platform_delay PLATFORM_DELAY]

 

Is this a known issue? Or does this have anything to do with licensing / forking the project? I have not tried a more recent version of the elasticsearch yet.

 

Thank you.

 

Got the same issue on my unRAID System.
Is there any solution for this problem?

Link to comment
On 8/11/2023 at 11:10 AM, pkoasne said:

https://github.com/jemalloc/jemalloc/issues/1328

# WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.

 

where is sysctl.conf ? I couldn't find it in the container.

Could this be referring to unRaid's OS and not the container?

 

Open terminal in unRaid

cp /etc/sysctl.conf /etc/sysctl.backup

nano /etc/sysctl.conf

paste vm.overcommit_memory = 1

save and reboot

Link to comment
  • 1 month later...
  • 1 month later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.