Hi all,
I'm having an issue with Graylog.
It was working fine, then I upgraded Unraid from 6.8 to 6.10.
Since then Graylog stops after a short time.
It stopped processing messages and the journal fills up.
If I restart the container, it will process the journal for a time, then stop again.
When it stops, I get this error:
2022-06-17 11:21:57,189 ERROR: com.google.common.util.concurrent.ServiceManager - Service LocalKafkaMessageQueueReader [FAILED] has failed in the RUNNING state.
java.lang.IllegalStateException: Invalid message size: 0
at org.graylog.shaded.kafka09.log.FileMessageSet.searchFor(FileMessageSet.scala:141) ~[graylog.jar:?]
at org.graylog.shaded.kafka09.log.LogSegment.translateOffset(LogSegment.scala:105) ~[graylog.jar:?]
at org.graylog.shaded.kafka09.log.LogSegment.read(LogSegment.scala:148) ~[graylog.jar:?]
at org.graylog.shaded.kafka09.log.Log.read(Log.scala:506) ~[graylog.jar:?]
at org.graylog2.shared.journal.LocalKafkaJournal.read(LocalKafkaJournal.java:677) ~[graylog.jar:?]
at org.graylog2.shared.journal.LocalKafkaJournal.readNext(LocalKafkaJournal.java:617) ~[graylog.jar:?]
at org.graylog2.shared.journal.LocalKafkaJournal.read(LocalKafkaJournal.java:599) ~[graylog.jar:?]
at org.graylog2.shared.messageq.localkafka.LocalKafkaMessageQueueReader.run(LocalKafkaMessageQueueReader.java:110) ~[graylog.jar:?]
at com.google.common.util.concurrent.AbstractExecutionThreadService$1$2.run(AbstractExecutionThreadService.java:67) [graylog.jar:?]
at com.google.common.util.concurrent.Callables$4.run(Callables.java:121) [graylog.jar:?]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_332]
I cannot work out why.
Elasticsearch is fine, according to it's logs.
I have no pipelines and I tried removing all extractors, same error.