Elastic Stack Setup


surfshack66

Recommended Posts

Hi - I'm interested in using this docker image, but ran into a few issues trying to start it.

 

https://hub.docker.com/r/sebp/elk/

 

Has anyone else successfully implemented this docker? Has anyone tried installing three separate containers of Elasticsearch, Logstash, and Kibana?

 

Looking forward to the responses because having this log server would be awesome to complement Grafana!

 

 

EDIT: Changed the topic from "Request ELK Stack" to "Elastic Stack Setup", since Beats has been added to the stack.

Edited by surfshack66
Link to comment
  • surfshack66 changed the title to (REQUEST) ELK Stack

Looks like Elasticsearch is failing to start.

 

Quote

ErrorWarningSystemArrayLogin


* Starting periodic command scheduler cron
...done.
* Starting Elasticsearch Server
...done.
waiting for Elasticsearch to be up (1/30)
waiting for Elasticsearch to be up (2/30)
waiting for Elasticsearch to be up (3/30)
waiting for Elasticsearch to be up (4/30)
waiting for Elasticsearch to be up (5/30)
waiting for Elasticsearch to be up (6/30)
waiting for Elasticsearch to be up (7/30)
waiting for Elasticsearch to be up (8/30)
waiting for Elasticsearch to be up (9/30)
waiting for Elasticsearch to be up (10/30)
waiting for Elasticsearch to be up (11/30)
waiting for Elasticsearch to be up (12/30)
waiting for Elasticsearch to be up (13/30)
waiting for Elasticsearch to be up (14/30)
waiting for Elasticsearch to be up (15/30)
waiting for Elasticsearch to be up (16/30)
waiting for Elasticsearch to be up (17/30)
waiting for Elasticsearch to be up (18/30)
waiting for Elasticsearch to be up (19/30)
waiting for Elasticsearch to be up (20/30)
waiting for Elasticsearch to be up (21/30)
waiting for Elasticsearch to be up (22/30)
waiting for Elasticsearch to be up (23/30)
waiting for Elasticsearch to be up (24/30)
waiting for Elasticsearch to be up (25/30)
waiting for Elasticsearch to be up (26/30)
waiting for Elasticsearch to be up (27/30)
waiting for Elasticsearch to be up (28/30)
waiting for Elasticsearch to be up (29/30)
waiting for Elasticsearch to be up (30/30)
Couln't start Elasticsearch. Exiting.
Elasticsearch log follows below.
[2018-09-13T09:37:55,954][INFO ][o.e.n.Node ] [] initializing ...
[2018-09-13T09:37:56,025][INFO ][o.e.e.NodeEnvironment ] [o7wlA7C] using [1] data paths, mounts [[/var/lib/elasticsearch (shfs)]], net usable_space [72gb], net total_space [238.4gb], types [fuse.shfs]
[2018-09-13T09:37:56,025][INFO ][o.e.e.NodeEnvironment ] [o7wlA7C] heap size [989.8mb], compressed ordinary object pointers [true]
[2018-09-13T09:37:56,027][INFO ][o.e.n.Node ] [o7wlA7C] node name derived from node ID [o7wlA7CsSem_dAXjTL3sTA]; set [node.name] to override
[2018-09-13T09:37:56,027][INFO ][o.e.n.Node ] [o7wlA7C] version[6.4.0], pid[94], build[default/tar/595516e/2018-08-17T23:18:47.308994Z], OS[Linux/4.14.49-unRAID/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_181/25.181-b13]
[2018-09-13T09:37:56,027][INFO ][o.e.n.Node ] [o7wlA7C] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.2ABInCTu, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.enforce.bootstrap.checks=true, -Des.path.home=/opt/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[2018-09-13T09:37:57,742][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [aggs-matrix-stats]
[2018-09-13T09:37:57,742][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [analysis-common]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [ingest-common]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [lang-expression]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [lang-mustache]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [lang-painless]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [mapper-extras]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [parent-join]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [percolator]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [rank-eval]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [reindex]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [repository-url]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [transport-netty4]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [tribe]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-core]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-deprecation]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-graph]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-logstash]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-ml]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-monitoring]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-rollup]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-security]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-sql]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-upgrade]
[2018-09-13T09:37:57,743][INFO ][o.e.p.PluginsService ] [o7wlA7C] loaded module [x-pack-watcher]
[2018-09-13T09:37:57,744][INFO ][o.e.p.PluginsService ] [o7wlA7C] no plugins loaded
[2018-09-13T09:38:01,189][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/121] [Main.cc@109] controller (64 bit): Version 6.4.0 (Build cf8246175efff5) Copyright (c) 2018 Elasticsearch BV
[2018-09-13T09:38:01,474][DEBUG][o.e.a.ActionModule ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2018-09-13T09:38:01,655][INFO ][o.e.d.DiscoveryModule ] [o7wlA7C] using discovery type [zen]
[2018-09-13T09:38:02,229][INFO ][o.e.n.Node ] [o7wlA7C] initialized
[2018-09-13T09:38:02,229][INFO ][o.e.n.Node ] [o7wlA7C] starting ...
[2018-09-13T09:38:02,334][INFO ][o.e.t.TransportService ] [o7wlA7C] publish_address {172.17.0.7:9300}, bound_addresses {0.0.0.0:9300}
[2018-09-13T09:38:02,347][INFO ][o.e.b.BootstrapChecks ] [o7wlA7C] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2018-09-13T09:38:02,350][ERROR][o.e.b.Bootstrap ] [o7wlA7C] node validation exception
[2] bootstrap checks failed
[1]: max file descriptors [40960] for elasticsearch process is too low, increase to at least [65536]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2018-09-13T09:38:02,353][INFO ][o.e.n.Node ] [o7wlA7C] stopping ...
[2018-09-13T09:38:02,406][INFO ][o.e.n.Node ] [o7wlA7C] stopped
[2018-09-13T09:38:02,406][INFO ][o.e.n.Node ] [o7wlA7C] closing ...
[2018-09-13T09:38:02,413][INFO ][o.e.n.Node ] [o7wlA7C] closed
[2018-09-13T09:38:02,414][INFO ][o.e.x.m.j.p.NativeController] Native controller process has stopped - no new native processes can be started

Two reasons for failing that I see are..

 

1. Max file descriptors need to increase

2. Max virtual memory areas need to increase

 

Supposedly, this is a fix for the second issue, but not sure if I want to be messing with unraids max_map_count.

 

Quote

A limit on mmap counts equal to 262,144 or more

!! This is the most frequent reason for Elasticsearch failing to start since Elasticsearch version 5 was released.

On Linux, use sysctl vm.max_map_count on the host to view the current value, and see Elasticsearch's documentation on virtual memory for guidance on how to change this value. Note that the limits must be changed on the host; they cannot be changed from within a container.

 

Anyone from Linuxserver.io interested in making an unraid compatible container for this?  

Link to comment

https://hub.docker.com/r/sebp/elk/

 

This is the up-to-date ELK stack, you just need to do a couple things.

 

 

Install this in docker ELK

 

Go through the wiki linked on the docker page, and make sure the variables are correct.

 

Add a variable : MAX_OPEN_FILES set to 65536

 

To get this to stick you need to set the ELK image as privileged (need to toggle advanced)

 

Download community apps script manager

 

Add the script below to run at start of array:

 

sysctl -w vm.max_map_count=262144

 

After this elk stack is fully running, you will still need to set it up with index and all that to parse data.

  • Thanks 1
Link to comment
  • 3 weeks later...
  • 4 months later...
  • 3 weeks later...
On 3/10/2019 at 7:57 PM, bobokun said:

Did you end up getting this to work with unraid? 

I have elasticsearch and kibana running but not pointing any logs to it yet. Ultimately, I'd like to use Beats, so I have a Metricbeat container but also not fully configured yet.

 

On 3/27/2019 at 1:05 PM, FoxxMD said:

Also having an issue changing vm.max_map_count. Running as root did not help.

I don't think I ever solved this either...

 

This project has taken a back seat for some time but I'd like to get this up and running soon.

Link to comment
  • surfshack66 changed the title to Elastic Stack Setup

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.