surfshack66 Posted September 14, 2018 Share Posted September 14, 2018 Hi - I'm interested in using this docker image, but ran into a few issues trying to start it. https://hub.docker.com/r/sebp/elk/ Has anyone else successfully implemented this docker? Has anyone tried installing three separate containers of Elasticsearch, Logstash, and Kibana? Looking forward to the responses because having this log server would be awesome to complement Grafana! EDIT: Changed the topic from "Request ELK Stack" to "Elastic Stack Setup", since Beats has been added to the stack. Link to comment
surfshack66 Posted September 18, 2018 Author Share Posted September 18, 2018 Looks like Elasticsearch is failing to start. Quote ErrorWarningSystemArrayLogin * Starting periodic command scheduler cron...done.* Starting Elasticsearch Server...done.waiting for Elasticsearch to be up (1/30)waiting for Elasticsearch to be up (2/30)waiting for Elasticsearch to be up (3/30)waiting for Elasticsearch to be up (4/30)waiting for Elasticsearch to be up (5/30)waiting for Elasticsearch to be up (6/30)waiting for Elasticsearch to be up (7/30)waiting for Elasticsearch to be up (8/30)waiting for Elasticsearch to be up (9/30)waiting for Elasticsearch to be up (10/30)waiting for Elasticsearch to be up (11/30)waiting for Elasticsearch to be up (12/30)waiting for Elasticsearch to be up (13/30)waiting for Elasticsearch to be up (14/30)waiting for Elasticsearch to be up (15/30)waiting for Elasticsearch to be up (16/30)waiting for Elasticsearch to be up (17/30)waiting for Elasticsearch to be up (18/30)waiting for Elasticsearch to be up (19/30)waiting for Elasticsearch to be up (20/30)waiting for Elasticsearch to be up (21/30)waiting for Elasticsearch to be up (22/30)waiting for Elasticsearch to be up (23/30)waiting for Elasticsearch to be up (24/30)waiting for Elasticsearch to be up (25/30)waiting for Elasticsearch to be up (26/30)waiting for Elasticsearch to be up (27/30)waiting for Elasticsearch to be up (28/30)waiting for Elasticsearch to be up (29/30)waiting for Elasticsearch to be up (30/30)Couln't start Elasticsearch. Exiting.Elasticsearch log follows below.[2018-09-13T09:37:55,954][INFO ][o.e.n.Node ] [] initializing ...[2018-09-13T09:37:56,025][INFO ][o.e.e.NodeEnvironment ] [o7wlA7C] using [1] data paths, mounts [[/var/lib/elasticsearch (shfs)]], net usable_space [72gb], net total_space [238.4gb], types [fuse.shfs][2018-09-13T09:37:56,025][INFO ][o.e.e.NodeEnvironment ] [o7wlA7C] heap size [989.8mb], compressed ordinary object pointers [true][2018-09-13T09:37:56,027][INFO ][o.e.n.Node ] [o7wlA7C] node name derived from node ID [o7wlA7CsSem_dAXjTL3sTA]; set [node.name] to override[2018-09-13T09:37:56,027][INFO ][o.e.n.Node ] [o7wlA7C] version[6.4.0], pid[94], build[default/tar/595516e/2018-08-17T23:18:47.308994Z], OS[Linux/4.14.49-unRAID/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_181/25.181-b13][2018-09-13T09:37:56,027][INFO ][o.e.n.Node ] [o7wlA7C] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.2ABInCTu, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.enforce.bootstrap.checks=true, -Des.path.home=/opt/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=tar][2018-09-13T09:37:57,742][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [aggs-matrix-stats][2018-09-13T09:37:57,742][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [analysis-common][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [ingest-common][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [lang-expression][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [lang-mustache][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [lang-painless][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [mapper-extras][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [parent-join][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [percolator][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [rank-eval][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [reindex][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [repository-url][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [transport-netty4][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [tribe][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-core][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-deprecation][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-graph][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-logstash][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-ml][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-monitoring][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-rollup][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-security][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-sql][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-upgrade][2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-watcher][2018-09-13T09:37:57,744][INFO ][o.e.p.PluginsService ] [o7wlA7C] no plugins loaded[2018-09-13T09:38:01,189][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/121] [Main.cc@109] controller (64 bit): Version 6.4.0 (Build cf8246175efff5) Copyright (c) 2018 Elasticsearch BV[2018-09-13T09:38:01,474][DEBUG][o.e.a.ActionModule ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security[2018-09-13T09:38:01,655][INFO ][o.e.d.DiscoveryModule ] [o7wlA7C] using discovery type [zen][2018-09-13T09:38:02,229][INFO ][o.e.n.Node ] [o7wlA7C] initialized[2018-09-13T09:38:02,229][INFO ][o.e.n.Node ] [o7wlA7C] starting ...[2018-09-13T09:38:02,334][INFO ][o.e.t.TransportService ] [o7wlA7C] publish_address {172.17.0.7:9300}, bound_addresses {0.0.0.0:9300}[2018-09-13T09:38:02,347][INFO ][o.e.b.BootstrapChecks ] [o7wlA7C] bound or publishing to a non-loopback address, enforcing bootstrap checks[2018-09-13T09:38:02,350][ERROR][o.e.b.Bootstrap ] [o7wlA7C] node validation exception[2] bootstrap checks failed[1]: max file descriptors [40960] for elasticsearch process is too low, increase to at least [65536][2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144][2018-09-13T09:38:02,353][INFO ][o.e.n.Node ] [o7wlA7C] stopping ...[2018-09-13T09:38:02,406][INFO ][o.e.n.Node ] [o7wlA7C] stopped[2018-09-13T09:38:02,406][INFO ][o.e.n.Node ] [o7wlA7C] closing ...[2018-09-13T09:38:02,413][INFO ][o.e.n.Node ] [o7wlA7C] closed[2018-09-13T09:38:02,414][INFO ][o.e.x.m.j.p.NativeController] Native controller process has stopped - no new native processes can be started Two reasons for failing that I see are.. 1. Max file descriptors need to increase 2. Max virtual memory areas need to increase Supposedly, this is a fix for the second issue, but not sure if I want to be messing with unraids max_map_count. Quote A limit on mmap counts equal to 262,144 or more !! This is the most frequent reason for Elasticsearch failing to start since Elasticsearch version 5 was released. On Linux, use sysctl vm.max_map_count on the host to view the current value, and see Elasticsearch's documentation on virtual memory for guidance on how to change this value. Note that the limits must be changed on the host; they cannot be changed from within a container. Anyone from Linuxserver.io interested in making an unraid compatible container for this? Link to comment
surfshack66 Posted September 18, 2018 Author Share Posted September 18, 2018 Found this recent thread referencing the ELK docker. Tried the script @Jclendineng mentioned, but received an error sysctl: cannot stat /proc/sys/vm/max_map_count: No such file or directory Is no one else using an ELK stack? Link to comment
surfshack66 Posted September 20, 2018 Author Share Posted September 20, 2018 @sparklyballs Are you interested in this docker? I believe this is something similar to your pf-logstash docker, which I used prior to 2.4 Link to comment
Jclendineng Posted September 22, 2018 Share Posted September 22, 2018 I am running it, it works great. I am having a hard time figuring the best way to import data but the elk stack itself works well. Link to comment
ken-ji Posted September 23, 2018 Share Posted September 23, 2018 I am using my own ELK stack. Its a bit old, as I've had it setup and running since the 6.0beta days (https://hub.docker.com/r/roninkenji/docker-elk/) I have the template attached but you won't get much support from me... I might see if I have some time to refresh my stack and publish it. ELK.xml Link to comment
Jclendineng Posted September 24, 2018 Share Posted September 24, 2018 https://hub.docker.com/r/sebp/elk/ This is the up-to-date ELK stack, you just need to do a couple things. Install this in docker : ELK Go through the wiki linked on the docker page, and make sure the variables are correct. Add a variable : MAX_OPEN_FILES set to 65536 To get this to stick you need to set the ELK image as privileged (need to toggle advanced) Download community apps script manager Add the script below to run at start of array: sysctl -w vm.max_map_count=262144 After this elk stack is fully running, you will still need to set it up with index and all that to parse data. Link to comment
surfshack66 Posted September 24, 2018 Author Share Posted September 24, 2018 Thanks @Jclendineng I added the script but received an error sysctl: cannot stat /proc/sys/vm/max_map_count: No such file or directory I attached a screenshot of the script Link to comment
Jclendineng Posted September 24, 2018 Share Posted September 24, 2018 Hmm, I just added a new script, named it vm.max_map_count, and used that command, saved. Link to comment
surfshack66 Posted September 25, 2018 Author Share Posted September 25, 2018 The error happens when the script runs...which is after rebooting the server and, subsequently, after the array starts... Link to comment
Jclendineng Posted September 25, 2018 Share Posted September 25, 2018 This is what mine looks like, Just tested and it runs fine Link to comment
surfshack66 Posted September 26, 2018 Author Share Posted September 26, 2018 Not a lot of room for user error here...I'm not sure how to debug this issue. I used MC to see if the file exists and it does... Link to comment
surfshack66 Posted October 16, 2018 Author Share Posted October 16, 2018 @Jclendineng Alright. So I was able to change the max map count by running the command as root. Next issue is this: [1]: max file descriptors [40960] for elasticsearch process is too low, increase to at least [65536] Did you come across this issue? If so, how did you address it? Link to comment
bobokun Posted March 10, 2019 Share Posted March 10, 2019 Did you end up getting this to work with unraid? Link to comment
FoxxMD Posted March 27, 2019 Share Posted March 27, 2019 Also having an issue changing vm.max_map_count. Running as root did not help. Link to comment
surfshack66 Posted April 2, 2019 Author Share Posted April 2, 2019 On 3/10/2019 at 7:57 PM, bobokun said: Did you end up getting this to work with unraid? I have elasticsearch and kibana running but not pointing any logs to it yet. Ultimately, I'd like to use Beats, so I have a Metricbeat container but also not fully configured yet. On 3/27/2019 at 1:05 PM, FoxxMD said: Also having an issue changing vm.max_map_count. Running as root did not help. I don't think I ever solved this either... This project has taken a back seat for some time but I'd like to get this up and running soon. Link to comment
FoxxMD Posted April 2, 2019 Share Posted April 2, 2019 @surfshack66 I actually got the max_map_count working by renaming the script to vm.max_map_count. I'm not sure why this works but it does ¯\_(ツ)_/¯ EDIT: I created an app for CA for elasticsearch, check it out. Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.