[Support] FoxxMD - elasticsearch


Recommended Posts

On 9/15/2020 at 2:50 PM, rob_robot said:

For those that are stuck like me on the ingest-attachment plugin issue:

 

You need to stop the elasticsearch docker and restart it after you have executed the command to install the plugin so it gets loaded into elasticsearch. 

 

Here my steps:

1.) get elasticsearch docker (7.9.1 works) do a clean install (delete old elasticsearch in /mnt/user/appdata/)

 

2.) Download the full text search packages in nextcloud app store (at least 3 packages)

 

3.) Configure your Nextcloud search platform to "Elasticsearch" and address of Servlet to: http://YOUR_IP:9200/ 

It needs to be configured to the port of the REST API

 

4.) Install the plugin for elasticsearch, by either opening a console inside the elasticsearch docker and type /usr/share/elasticsearch/bin/elasticsearch-plugin install --batch ingest-attachment

OR 

through the User scripts Unraid plugin as stated above. 

 

5.) Restart the elasticsearch container

 

6.) Test everything by opening a new shell in the nextcloud container then navigate to the occ directory (/var/www/html) and type 

./occ fulltextsearch:test

 

If everything is ok, then you can continue with the index: ./occ fulltextsearch:index

 

 

This worked great with Nextcloud 20.0.2 and ElasticSearch 7.9.3. Do I need to re-index manually each time files are added? To get OCR working for PDF files I installed Tesseract OCR. Will that start running OCR automatically on all files once enable or needs to be run manually?

Link to comment
  • 3 months later...
On 10/25/2020 at 11:21 AM, Geck0 said:

Okay,

So for those following after me. I'm not very experienced with docker, but I pulled down the container manually and by doing so changed the environment variables, changing the heap size to 2g.

I changed the second set of ES_JAVA_OPTS (in red) from Xms512mb -Xmx512mb to 2g instead. However, I've just noticed that there are two ES_JAVA_OPTS entries, the other one I've marked in yellow. This was a copy and paste from the original container install. No matter what I did I couldn't get the heap size to change from 512mb to any other size by setting ES_JAVA_OPTS as a variable in the CA settings for the container or via jvm.options inside the container itself.

 

Anybody shed light on this? I would like to just pull the container down without all this phaff.

 

I also had the same heap size problem with elasticsearch. It can be solved by editing the docker, then switching to "Advanced view" and edit the EXTRA PARAMETERS line. 

Here an example to switch from 512MB to 4G of heap size:

-e "ES_JAVA_OPTS"="-Xms4g -Xmx4g" --ulimit nofile=262144:262144

 

Actual heap size can be checked by opening up a console inside the docker and then running following command: 

curl -sS  "localhost:9200/_cat/nodes?h=heap*&v"

 

Heap.max should then show 4GB instead of 512 MB.

Edited by rob_robot
Link to comment
  • 1 month later...
  • 5 months later...

Hi guys,

 

I did everything according to this tutorial and crawling seems to be working. But: I get an Error in the ES Docker Log, claiming:

 

{"type": "server", "timestamp": "2021-10-20T07:37:59,873+02:00", "level": "WARN", "component": "o.e.c.c.ClusterFormationFailureHelper", "cluster.name": "docker-cluster", "node.name": "911179884b65", "message": "master not discovered or elected yet, an election requires a node with id [tJO09zgcQSOaJvZadHyMXQ], have discovered [{911179884b65}{tJO09zgcQSOaJvZadHyMXQ}{FzZZFb76SSu6dgTTsjkWJw}{172.17.0.2}{172.17.0.2:9300}{dilmrt}{ml.machine_memory=67047288832, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}] which is a quorum; discovery will continue using [] from hosts providers and [{911179884b65}{tJO09zgcQSOaJvZadHyMXQ}{FzZZFb76SSu6dgTTsjkWJw}{172.17.0.2}{172.17.0.2:9300}{dilmrt}{ml.machine_memory=67047288832, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}] from last-known cluster state; node term 3, last-accepted version 48 in term 3", "cluster.uuid": "JqY853ThR_uPDSn3mURqJA", "node.id": "tJO09zgcQSOaJvZadHyMXQ" }

 

I guess what ES is saying is it is looking for a "master", but can't find a node with that ID. Question is: why would it search for that? How do I configure that and where does that specific ID come from? The same time that error occured I also get errors in FSCrawler, saying the directory he just crawled for about half an hour suddenly does not exist anymore.

 

04:24:27,517 [33mWARN [m [f.p.e.c.f.FsParserAbstract] Error while crawling /mnt/user/public: /mnt/user/public doesn't exists.

 

Can anybody make sense of this and maybe even help me fix it?

 

Thanks a lot for your help in advance!

 

Greetings from Germany,

Flo

Link to comment
  • 3 months later...

One other thing I noticed with this docker is the contain had a high writable size after getting high docker container utilization warnings.  I ended up adding in another contain path map /usr/share/elasticsearch/logs  TO  /mnt/user/appdata/elasticsearch/logs/ to push this large folder out of the container.  Does anyone know if there is a downside to this?  Not sure why a folder in the GB size was not in the docker template unless maybe created some issues?

Link to comment
  • 1 month later...

I can install the docker with no issues, however the docker stops by itself after a few seconds with the log as follows:

(standard install, no changes in template or values)

 

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2022-03-15T15:31:21,997][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [unknown] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Failed to create node environment
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.6.2.jar:6.6.2]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.6.2.jar:6.6.2]
Caused by: java.lang.IllegalStateException: Failed to create node environment
        at org.elasticsearch.node.Node.<init>(Node.java:298) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.2.jar:6.6.2]
        ... 6 more
Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:385) ~[?:?]
        at java.nio.file.Files.createDirectory(Files.java:689) ~[?:?]
        at java.nio.file.Files.createAndCheckIsDirectory(Files.java:796) ~[?:?]
        at java.nio.file.Files.createDirectories(Files.java:782) ~[?:?]
        at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:270) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:203) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:267) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.node.Node.<init>(Node.java:295) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.2.jar:6.6.2]
        ... 6 more
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2022-03-15T15:44:58,649][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [unknown] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Failed to create node environment
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.6.2.jar:6.6.2]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.6.2.jar:6.6.2]
Caused by: java.lang.IllegalStateException: Failed to create node environment
        at org.elasticsearch.node.Node.<init>(Node.java:298) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.2.jar:6.6.2]
        ... 6 more
Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:385) ~[?:?]
        at java.nio.file.Files.createDirectory(Files.java:689) ~[?:?]
        at java.nio.file.Files.createAndCheckIsDirectory(Files.java:796) ~[?:?]
        at java.nio.file.Files.createDirectories(Files.java:782) ~[?:?]
        at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:270) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:203) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:267) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.node.Node.<init>(Node.java:295) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.2.jar:6.6.2]
        ... 6 more

** Press ANY KEY to close this window ** 

 

I have the impression it has to do with user rights but unfortunately not tech savvy enough to solve it:)

Does anybody have an idea how to solve this?

Link to comment
  • 2 months later...
On 3/15/2022 at 3:49 PM, CvT said:

I can install the docker with no issues, however the docker stops by itself after a few seconds with the log as follows:

(standard install, no changes in template or values)

 

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2022-03-15T15:31:21,997][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [unknown] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Failed to create node environment
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.6.2.jar:6.6.2]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.6.2.jar:6.6.2]
Caused by: java.lang.IllegalStateException: Failed to create node environment
        at org.elasticsearch.node.Node.<init>(Node.java:298) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.2.jar:6.6.2]
        ... 6 more
Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:385) ~[?:?]
        at java.nio.file.Files.createDirectory(Files.java:689) ~[?:?]
        at java.nio.file.Files.createAndCheckIsDirectory(Files.java:796) ~[?:?]
        at java.nio.file.Files.createDirectories(Files.java:782) ~[?:?]
        at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:270) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:203) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:267) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.node.Node.<init>(Node.java:295) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.2.jar:6.6.2]
        ... 6 more
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2022-03-15T15:44:58,649][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [unknown] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Failed to create node environment
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.6.2.jar:6.6.2]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.6.2.jar:6.6.2]
Caused by: java.lang.IllegalStateException: Failed to create node environment
        at org.elasticsearch.node.Node.<init>(Node.java:298) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.2.jar:6.6.2]
        ... 6 more
Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:385) ~[?:?]
        at java.nio.file.Files.createDirectory(Files.java:689) ~[?:?]
        at java.nio.file.Files.createAndCheckIsDirectory(Files.java:796) ~[?:?]
        at java.nio.file.Files.createDirectories(Files.java:782) ~[?:?]
        at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:270) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:203) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:267) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.node.Node.<init>(Node.java:295) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.2.jar:6.6.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.2.jar:6.6.2]
        ... 6 more

** Press ANY KEY to close this window ** 

 

I have the impression it has to do with user rights but unfortunately not tech savvy enough to solve it:)

Does anybody have an idea how to solve this?

Probably not the best way but I ssh’d in and created the node folder and did a chown nobody:users

  • Upvote 1
Link to comment
  • 2 weeks later...

Hi, Im try to set up this container with the latest version 8.2.2. I'm getting the following error, looks like it's permission related but I have no idea why the default template wouldn't have the required permissions?

 

 

 

ElasticsearchUncaughtExceptionHandler","elasticsearch.node.name":"b046ed08a40e","elasticsearch.cluster.name":"docker-cluster","error.type":"org.elasticsearch.bootstrap.StartupException","error.message":"java.lang.IllegalStateException: failed to obtain node locks, tried [/usr/share/elasticsearch/data]; maybe these locations are not writable or multiple nodes were started on the same data path?","error.stack_trace":"java.lang.IllegalStateException: failed to obtain node locks, tried [/usr/share/elasticsearch/data]; maybe these locations are not writable or multiple nodes were started on the same data path?\nLikely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/data/node.lock\n\tat java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)\n\tat java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)\n\tat java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)\n\tat java.base/sun.nio.fs.UnixPath.toRealPath(UnixPath.java:825)\n\tat org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:94)\n\tat org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:43)\n\tat org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:44)\n\tat org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:222)\n\tat org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:197)\n\tat org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:276)\n\tat org.elasticsearch.node.Node.<init>(Node.java:404)\n\tat org.elasticsearch.node.Node.<init>(Node.java:291)\n\tat org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234)\n\tat org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234)\n\tat org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:358)\n\tat org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166)\n\tat org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157)\n\tat org.elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:81)\n\tat org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)\n\tat org.elasticsearch.cli.Command.main(Command.java:77)\n\tat org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122)\n\tat org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80)\nFor complete error details, refer to the log at /usr/share/elasticsearch/logs/docker-cluster.log\n"}
uncaught exception in thread [main]
java.lang.IllegalStateException: failed to obtain node locks, tried [/usr/share/elasticsearch/data]; maybe these locations are not writable or multiple nodes were started on the same data path?
Likely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/data/node.lock
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixPath.toRealPath(UnixPath.java:825)
        at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:94)
        at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:43)
        at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:44)
        at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:222)
        at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:197)
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:276)
        at org.elasticsearch.node.Node.<init>(Node.java:404)
        at org.elasticsearch.node.Node.<init>(Node.java:291)
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234)
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234)
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:358)
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166)
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157)
        at org.elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:81)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
        at org.elasticsearch.cli.Command.main(Command.java:77)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80)
For complete error details, refer to the log at /usr/share/elasticsearch/logs/docker-cluster.log

** Press ANY KEY to close this window ** 

 

Edited by wolfNZ
  • Upvote 1
Link to comment

Ok, well I tried the alternative elasticsearch template by d8sychain and got it working by adding this environment variable. Not sure what the implications are, clearly something weird going on with my permissions. It didn't work with this template.

 

image.png.4a2ddd0775eba1f48944e1edb78eaff3.png

Edited by wolfNZ
Link to comment
  • 1 month later...

I was just trying to get this to work on UnRaid 6.10.3 - new config.  I got the access denied error trying create the node environment.  After searching the web I saw someone change the path to /var/lib/elasticsearch/data and it worked so I edited the docker image and changed that path for only the container path but left /mnt/user/appdata for host path and default value.  It seems to be working now but I confess I don't know if this is a viable (supportable) config.  Can anyone validate if it makes sense that I should have to do this - possibly due to new defaults or something in 6.10.3?

Link to comment
1 hour ago, Rich Drinan said:

Is this app/thread abandoned?

If its the one from FoxMD, then not abandoned as its the official container for elasticsearch and they are the primary support for their own container.  Did you run the command (and script) as detailed in the description?

Link to comment
4 hours ago, Squid said:

If its the one from FoxMD, then not abandoned as its the official container for elasticsearch and they are the primary support for their own container.  Did you run the command (and script) as detailed in the description?

Thanks - I did install user scripts and created the script as instructed.  I actually got the container to run but by changing the container path as I listed above - which was to get around a permissions issue similar to the other two people that had permission errors.  I don't know what the ramifications are of my changing to /var/usr/~.  Since at least the last 3 people {starting with @pinion in March) had the same issue with no commentary on the "right" way to fix it, I was wondering if this thread wasn't monitored - no offense intended though.

Link to comment
On 7/27/2022 at 8:55 PM, Rich Drinan said:

Thanks - I did install user scripts and created the script as instructed.  I actually got the container to run but by changing the container path as I listed above - which was to get around a permissions issue similar to the other two people that had permission errors.  I don't know what the ramifications are of my changing to /var/usr/~.  Since at least the last 3 people {starting with @pinion in March) had the same issue with no commentary on the "right" way to fix it, I was wondering if this thread wasn't monitored - no offense intended though.

I didn't have to change anything in the Docker container path. I had to go into my appdata/elasticsearch share and change the owner to Nobody and also the group/other permissions to r/w (I used the Dynamix file plugin). The owner was root before. Not sure if this is also correct as I am pretty new to configuring docker/unraid outside of the usual templates. I think this has to do with the 6.10.3 permissions though.

Link to comment
On 5/27/2022 at 8:39 PM, wolfNZ said:

Ok, well I tried the alternative elasticsearch template by d8sychain and got it working by adding this environment variable. Not sure what the implications are, clearly something weird going on with my permissions. It didn't work with this template.

 

image.png.4a2ddd0775eba1f48944e1edb78eaff3.png

@wolfNZ Can you explain how you added this variable to your elastic search container?

Link to comment
  • 3 weeks later...

Just found this thread after I fixed my elasticsearch container.
I didn't know about the TAKE_FILE_OWNERSHIP Variable, will have to check this later.
I just went to the /mnt/user/appdata/ folder,

 

chown -R nobody:users elasticsearch/ 

because it was root:root and

 

chmod 777- R elasticsearch/

(i know bad, will be fixed later).


Variable ELASTIC_PASSWORD didn't work for me.

Had to access the console of that container and

 

cd /usr/share/elasticsearch/bin

./elasticsearch-reset-password -u elastic

gave me a new password.

 

Then I was able to configure my Zenarmor OPNsense Plugin, httpS:xxx.xxx.xxx.xxx:9200 worked then as expected.

  • Thanks 3
Link to comment
  • 1 month later...
  • 2 months later...
  • 2 weeks later...

Has anyone had any luck upgrading Elasticsearch using this template? I was able to get it updated to v7.17.8 by simply changing the Repository: field to: elasticsearch:7.17.8. Although this successfully updated the image, I am now being inundated with warnings in the logs. Specifically:

 

2022-12-29 08:17:33,366 WARN : org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RestClient - request [GET http://192.168.100.69:9200/_cluster/health?master_timeout=60s&level=cluster&timeout=60s&local=true] returned 1 warnings: [299 Elasticsearch-7.17.8-120eabe1c8a0cb2ae87cffc109a5b65d213e9df1 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-minimal-setup.html to enable security."]

 

I am new to Docker, so I am not exactly sure how to correct this, but if I figure it out, I will post it back here.

Edited by irishjd
Link to comment

Me again... so, even though the warning is being displayed in the graylog log file, this is actually a elasticsearch issue. I believe I know how to fix it kind of. To get rid of this warning, I need to add a couple of lines to the elasticsearch.yml configuration file. The problem is, this file is not located in persistent storage (i.e. /appdata/), as it is located at: /usr/share/elasticsearch/config/elasticsearch.yml. Can anyone tell me how I can make persistent changes to this file? Specifically, I need to add these two lines:

 

xpack.security.enabled: true

discovery.type: single-node

Edited by irishjd
Link to comment
  • 1 month later...
On 8/23/2022 at 1:20 PM, buyenne said:

Just found this thread after I fixed my elasticsearch container.
I didn't know about the TAKE_FILE_OWNERSHIP Variable, will have to check this later.
I just went to the /mnt/user/appdata/ folder,

 

chown -R nobody:users elasticsearch/ 

because it was root:root and

 

chmod 777- R elasticsearch/

(i know bad, will be fixed later).


Variable ELASTIC_PASSWORD didn't work for me.

Had to access the console of that container and

 

cd /usr/share/elasticsearch/bin

./elasticsearch-reset-password -u elastic

gave me a new password.

 

Then I was able to configure my Zenarmor OPNsense Plugin, httpS:xxx.xxx.xxx.xxx:9200 worked then as expected.

 

seems like "./elasticsearch-reset-password -u elastic" command no longer works as its missing from that dir.

is there any work around for this?

Link to comment
  • 2 months later...

First, thank you to @muwahhid and @rob_robot as their posts were very helpful.

 

Running v6.12.0-rc4.1

 

Tested both elasticsearch dockers available in community applications for versions 6.6.2 and 7.10.2.

 

Up to this point everything appears to be working. When i run ./occ fulltextsearch:test in the Nextcloud v26.0.1 container i receive the error:

 

.Testing your current setup:  
Creating mocked content provider. ok  
Testing mocked provider: get indexable documents. (2 items) ok  
Loading search platform. (Elasticsearch) ok  
Testing search platform. fail
In ProductCheckTrait.php line 32:
                                                                                                      
  The client noticed that the server is not Elasticsearch and we do not support this unknown product  
                                                                                                      

fulltextsearch:test [--output [OUTPUT]] [-j|--json] [-d|--platform_delay PLATFORM_DELAY]

 

Is this a known issue? Or does this have anything to do with licensing / forking the project? I have not tried a more recent version of the elasticsearch yet.

 

Thank you.

Link to comment
  • 3 months later...

Just wanted to let others know how to make this template work with Elasticsearch 8.9:

Not necessary changes, but I wanted them:

 

My complete template:

<?xml version="1.0"?>
<Container version="2">
  <Name>elasticsearch</Name>
  <Repository>docker.elastic.co/elasticsearch/elasticsearch:8.9.0</Repository>
  <Registry>https://hub.docker.com/_/elasticsearch</Registry>
  <Network>bridge</Network>
  <MyIP/>
  <Shell>bash</Shell>
  <Privileged>false</Privileged>
  <Support>https://forums.unraid.net/topic/79125-support-foxxmd-elasticsearch</Support>
  <Project/>
  <Overview>Elasticsearch with instructions for installation on unraid. &#xD;
&#xD;
        Directions below MUST be used in order to get Elasticsearch 5 and above working correctly. Check support thread for more information.&#xD;
&#xD;
        Directions:&#xD;
        1. Install CA User Scripts&#xD;
        2. Create a new script named vm.max_map_count&#xD;
        3. Contents of script as follows: &#xD;
&#xD;
        #!/bin/bash&#xD;
        sysctl -w vm.max_map_count=262144&#xD;
&#xD;
        4. Set script schedule to At Startup of Array&#xD;
&#xD;
Adapted version by Luca for elasticsearch 8.x.x</Overview>
  <Category>Other: Tools:Utilities</Category>
  <WebUI>http://[IP]:[PORT:9200]/</WebUI>
  <TemplateURL>https://raw.githubusercontent.com/FoxxMD/unraid-docker-templates/master/foxxmd/elasticsearch.xml</TemplateURL>
  <Icon>https://github.com/FoxxMD/unraid-docker-templates/raw/master/elasticsearch.png</Icon>
  <ExtraParams>--ulimit nofile=262144:262144 --ulimit memlock=-1:-1 --cap-add=IPC_LOCK  --user 99 --group-add=100 --group-add=0</ExtraParams>
  <PostArgs/>
  <CPUset/>
  <DateInstalled>1692281389</DateInstalled>
  <DonateText/>
  <DonateLink/>
  <Requires/>
  <Config Name="Data" Target="/usr/share/elasticsearch/data" Default="/mnt/user/appdata/elasticsearch/data" Mode="rw" Description="Directory where ES data is persisted" Type="Path" Display="always" Required="true" Mask="false">/mnt/user/appdata/elasticsearch/data</Config>
  <Config Name="REST API Port" Target="9200" Default="9200" Mode="tcp" Description="Host port exposed for REST HTTP interface." Type="Port" Display="always" Required="true" Mask="false">9200</Config>
  <Config Name="Nodes Communication Port" Target="9300" Default="9300" Mode="tcp" Description="Host port exposed for node communication." Type="Port" Display="always" Required="true" Mask="false">9300</Config>
  <Config Name="enforce SSL/login" Target="xpack.security.enabled" Default="false" Mode="" Description="" Type="Variable" Display="always" Required="false" Mask="false">false</Config>
  <Config Name="Plugins" Target="/usr/share/elasticsearch/plugins" Default="/mnt/user/appdata/elasticsearch/plugins" Mode="rw" Description="Install plugins through the console of the container. eg. bin/elasticsearch-plugin install [plugin_name]" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/elasticsearch/plugins</Config>
  <Config Name="Logs" Target="/usr/share/elasticsearch/logs" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/appdata/elasticsearch/logs</Config>
  <Config Name="ES_JAVA_OPTS" Target="ES_JAVA_OPTS" Default="-Xms512m -Xmx512m" Mode="" Description="https://www.elastic.co/guide/en/elasticsearch/reference/8.9/advanced-configuration.html#set-jvm-heap-size" Type="Variable" Display="advanced-hide" Required="false" Mask="false">-Xms512m -Xmx512m</Config>
  <Config Name="Discovery Type" Target="discovery.type" Default="single-node" Mode="" Description="" Type="Variable" Display="advanced-hide" Required="true" Mask="false">single-node</Config>
</Container>

 

Edited by lucahammer
fix group-add
  • Thanks 2
Link to comment

image.thumb.png.7f88d68a40988040555a6cd245e7ca23.png

 

I am using unraid 6.12.1, elasticsearch:8.8.1;

 

Followed @lucahammer, @buyenne suggestions, added `TAKE_FILE_OWNERSHIP` and `xpack.security.enabled`; But it don't work for me ...

 

{
  "@timestamp": "2023-08-22T08:52:24.691Z",
  "log.level": "ERROR",
  "message": "fatal exception while booting Elasticsearch",
  "ecs.version": "1.2.0",
  "service.name": "ES_ECS",
  "event.dataset": "elasticsearch.server",
  "process.thread.name": "main",
  "log.logger": "org.elasticsearch.bootstrap.Elasticsearch",
  "elasticsearch.node.name": "0c2a2d6fb5b6",
  "elasticsearch.cluster.name": "docker-cluster",
  "error.type": "java.lang.IllegalStateException",
  "error.message": "failed to obtain node locks, tried [/usr/share/elasticsearch/data]; maybe these locations are not writable or multiple nodes were started on the same data path?",
  "error.stack_trace": "java.lang.IllegalStateException: failed to obtain node locks, tried [/usr/share/elasticsearch/data]; maybe these locations are not writable or multiple nodes were started on the same data path?\n\tat [email protected]/org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:291)\n\tat [email protected]/org.elasticsearch.node.Node.<init>(Node.java:483)\n\tat [email protected]/org.elasticsearch.node.Node.<init>(Node.java:327)\n\tat [email protected]/org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:216)\n\tat [email protected]/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:216)\n\tat [email protected]/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:67)\nCaused by: java.io.IOException: failed to obtain lock on /usr/share/elasticsearch/data\n\tat [email protected]/org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:236)\n\tat [email protected]/org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:204)\n\tat [email protected]/org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:283)\n\t... 5 more\nCaused by: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/data/node.lock\n\tat java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)\n\tat java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)\n\tat java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)\n\tat java.base/sun.nio.fs.UnixPath.toRealPath(UnixPath.java:833)\n\tat [email protected]/org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:94)\n\tat [email protected]/org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:43)\n\tat [email protected]/org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:44)\n\tat [email protected]/org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:229)\n\t... 7 more\n\tSuppressed: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/node.lock\n\t\tat java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)\n\t\tat java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)\n\t\tat java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)\n\t\tat java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:261)\n\t\tat java.base/java.nio.file.Files.newByteChannel(Files.java:379)\n\t\tat java.base/java.nio.file.Files.createFile(Files.java:657)\n\t\tat [email protected]/org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:84)\n\t\t... 10 more\n"
}

 

Error seems to suggest file permission issue.

 

It worked after I change the ownership and permission for the data mount

 

chown -R nobody:users elasticsearch/
chmod 777 -R elasticsearch/

 

Edited by shawnngtq
  • Upvote 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.