InfluxDb 1.8.4 beendet sich einfach

[icebreaker01]

Guten Morgen zusammen,

 

ich habe hier leider ein kleines Problem mit InfluxDB, dieses beendet sich seit gestern abend selbständig und ich bekomme es nicht mehr zum laufen.
Hier der eintrag aus dem Protokoll von InfluxDB
 

ts=2022-07-25T06:58:24.539337Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2009" duration=31.972ms
ts=2022-07-25T06:58:24.539382Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2011" duration=28.206ms
ts=2022-07-25T06:58:24.539608Z lvl=info msg="Reading file" log_id=0bu2EpjG000 engine=tsm1 service=cacheloader path="/var/lib/influxdb/wal/varken/varken 30d-1h/2018/_00001.wal" size=1494
ts=2022-07-25T06:58:24.539674Z lvl=info msg="Reading file" log_id=0bu2EpjG000 engine=tsm1 service=cacheloader path="/var/lib/influxdb/wal/varken/varken 30d-1h/2019/_00001.wal" size=1110
ts=2022-07-25T06:58:24.539696Z lvl=info msg="Reading file" log_id=0bu2EpjG000 engine=tsm1 service=cacheloader path="/var/lib/influxdb/wal/varken/varken 30d-1h/2021/_00001.wal" size=358
ts=2022-07-25T06:58:24.539617Z lvl=info msg="Reading file" log_id=0bu2EpjG000 engine=tsm1 service=cacheloader path="/var/lib/influxdb/wal/varken/varken 30d-1h/2017/_00001.wal" size=2843
ts=2022-07-25T06:58:24.539706Z lvl=info msg="Reading file" log_id=0bu2EpjG000 engine=tsm1 service=cacheloader path="/var/lib/influxdb/wal/varken/varken 30d-1h/2020/_00001.wal" size=540
ts=2022-07-25T06:58:24.565027Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2015" duration=33.925ms
ts=2022-07-25T06:58:24.565098Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2016" duration=33.919ms
ts=2022-07-25T06:58:24.565559Z lvl=info msg="Reading file" log_id=0bu2EpjG000 engine=tsm1 service=cacheloader path="/var/lib/influxdb/wal/varken/varken 30d-1h/2024/_00001.wal" size=718
ts=2022-07-25T06:58:24.565563Z lvl=info msg="Reading file" log_id=0bu2EpjG000 engine=tsm1 service=cacheloader path="/var/lib/influxdb/wal/varken/varken 30d-1h/2023/_00001.wal" size=180
ts=2022-07-25T06:58:24.568867Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2014" duration=37.770ms
ts=2022-07-25T06:58:24.569109Z lvl=info msg="Reading file" log_id=0bu2EpjG000 engine=tsm1 service=cacheloader path="/var/lib/influxdb/wal/varken/varken 30d-1h/2025/_00001.wal" size=360
ts=2022-07-25T06:58:24.576513Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2020" duration=37.156ms
ts=2022-07-25T06:58:24.576845Z lvl=info msg="Reading file" log_id=0bu2EpjG000 engine=tsm1 service=cacheloader path="/var/lib/influxdb/wal/varken/varken 30d-1h/2026/_00001.wal" size=358
ts=2022-07-25T06:58:24.577527Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2019" duration=38.169ms
ts=2022-07-25T06:58:24.577965Z lvl=info msg="Reading file" log_id=0bu2EpjG000 engine=tsm1 service=cacheloader path="/var/lib/influxdb/wal/varken/varken 30d-1h/2027/_00001.wal" size=363
ts=2022-07-25T06:58:24.579968Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2018" duration=40.653ms
ts=2022-07-25T06:58:24.580057Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2017" duration=40.773ms
ts=2022-07-25T06:58:24.580071Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2021" duration=40.642ms
ts=2022-07-25T06:58:24.580382Z lvl=info msg="Reading file" log_id=0bu2EpjG000 engine=tsm1 service=cacheloader path="/var/lib/influxdb/wal/varken/varken 30d-1h/2028/_00001.wal" size=721
ts=2022-07-25T06:58:24.580486Z lvl=info msg="Opened file" log_id=0bu2EpjG000 engine=tsm1 service=filestore path="/var/lib/influxdb/data/varken/varken 30d-1h/1678/000000001-000000001.tsm" id=0 duration=0.096ms
ts=2022-07-25T06:58:24.585548Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2024" duration=20.377ms
ts=2022-07-25T06:58:24.590086Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2025" duration=21.183ms
ts=2022-07-25T06:58:24.590120Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2023" duration=25.021ms
ts=2022-07-25T06:58:24.594480Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2026" duration=17.920ms
ts=2022-07-25T06:58:24.595834Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/1678" duration=15.748ms
ts=2022-07-25T06:58:24.595872Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2027" duration=18.304ms
ts=2022-07-25T06:58:24.595890Z lvl=info msg="Opened shard" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open index_version=inmem path="/var/lib/influxdb/data/varken/varken 30d-1h/2028" duration=15.893ms
ts=2022-07-25T06:58:24.613980Z lvl=info msg="Open store (end)" log_id=0bu2EpjG000 service=store trace_id=0bu2Eq80000 op_name=tsdb_open op_event=end op_elapsed=1989.425ms
ts=2022-07-25T06:58:24.614029Z lvl=info msg="Opened service" log_id=0bu2EpjG000 service=subscriber
ts=2022-07-25T06:58:24.614037Z lvl=info msg="Starting monitor service" log_id=0bu2EpjG000 service=monitor
ts=2022-07-25T06:58:24.614043Z lvl=info msg="Registered diagnostics client" log_id=0bu2EpjG000 service=monitor name=build
ts=2022-07-25T06:58:24.614048Z lvl=info msg="Registered diagnostics client" log_id=0bu2EpjG000 service=monitor name=runtime
ts=2022-07-25T06:58:24.614052Z lvl=info msg="Registered diagnostics client" log_id=0bu2EpjG000 service=monitor name=network
ts=2022-07-25T06:58:24.614061Z lvl=info msg="Registered diagnostics client" log_id=0bu2EpjG000 service=monitor name=system
ts=2022-07-25T06:58:24.614089Z lvl=info msg="Starting precreation service" log_id=0bu2EpjG000 service=shard-precreation check_interval=10m advance_period=30m
ts=2022-07-25T06:58:24.614101Z lvl=info msg="Starting snapshot service" log_id=0bu2EpjG000 service=snapshot
ts=2022-07-25T06:58:24.614106Z lvl=info msg="Starting continuous query service" log_id=0bu2EpjG000 service=continuous_querier
ts=2022-07-25T06:58:24.614107Z lvl=info msg="Storing statistics" log_id=0bu2EpjG000 service=monitor db_instance=_internal db_rp=monitor interval=10s
ts=2022-07-25T06:58:24.614125Z lvl=info msg="Starting HTTP service" log_id=0bu2EpjG000 service=httpd authentication=false
ts=2022-07-25T06:58:24.614139Z lvl=info msg="opened HTTP access log" log_id=0bu2EpjG000 service=httpd path=stderr
ts=2022-07-25T06:58:24.614307Z lvl=info msg="Listening on HTTP" log_id=0bu2EpjG000 service=httpd addr=[::]:8086 https=false
ts=2022-07-25T06:58:24.614329Z lvl=info msg="Starting retention policy enforcement service" log_id=0bu2EpjG000 service=retention check_interval=30m
ts=2022-07-25T06:58:24.614391Z lvl=info msg="Listening for signals" log_id=0bu2EpjG000
ts=2022-07-25T06:58:24.618864Z lvl=info msg="Sending usage statistics to usage.influxdata.com" log_id=0bu2EpjG000
ts=2022-07-25T06:58:25.608476Z lvl=info msg="TSM compaction (start)" log_id=0bu2EpjG000 engine=tsm1 tsm1_level=1 tsm1_strategy=level trace_id=0bu2F0n0000 op_name=tsm1_compact_group op_event=start
ts=2022-07-25T06:58:25.608506Z lvl=info msg="Beginning compaction" log_id=0bu2EpjG000 engine=tsm1 tsm1_level=1 tsm1_strategy=level trace_id=0bu2F0n0000 op_name=tsm1_compact_group tsm1_files_n=8
ts=2022-07-25T06:58:25.608514Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_level=1 tsm1_strategy=level trace_id=0bu2F0n0000 op_name=tsm1_compact_group tsm1_index=0 tsm1_file=/var/lib/influxdb/data/_internal/monitor/1997/000000091-000000001.tsm
ts=2022-07-25T06:58:25.608522Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_level=1 tsm1_strategy=level trace_id=0bu2F0n0000 op_name=tsm1_compact_group tsm1_index=1 tsm1_file=/var/lib/influxdb/data/_internal/monitor/1997/000000092-000000001.tsm
ts=2022-07-25T06:58:25.608529Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_level=1 tsm1_strategy=level trace_id=0bu2F0n0000 op_name=tsm1_compact_group tsm1_index=2 tsm1_file=/var/lib/influxdb/data/_internal/monitor/1997/000000093-000000001.tsm
ts=2022-07-25T06:58:25.608536Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_level=1 tsm1_strategy=level trace_id=0bu2F0n0000 op_name=tsm1_compact_group tsm1_index=3 tsm1_file=/var/lib/influxdb/data/_internal/monitor/1997/000000094-000000001.tsm
ts=2022-07-25T06:58:25.608543Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_level=1 tsm1_strategy=level trace_id=0bu2F0n0000 op_name=tsm1_compact_group tsm1_index=4 tsm1_file=/var/lib/influxdb/data/_internal/monitor/1997/000000095-000000001.tsm
ts=2022-07-25T06:58:25.608550Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_level=1 tsm1_strategy=level trace_id=0bu2F0n0000 op_name=tsm1_compact_group tsm1_index=5 tsm1_file=/var/lib/influxdb/data/_internal/monitor/1997/000000096-000000001.tsm
ts=2022-07-25T06:58:25.608566Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_level=1 tsm1_strategy=level trace_id=0bu2F0n0000 op_name=tsm1_compact_group tsm1_index=6 tsm1_file=/var/lib/influxdb/data/_internal/monitor/1997/000000097-000000001.tsm
ts=2022-07-25T06:58:25.608571Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_level=1 tsm1_strategy=level trace_id=0bu2F0n0000 op_name=tsm1_compact_group tsm1_index=7 tsm1_file=/var/lib/influxdb/data/_internal/monitor/1997/000000098-000000001.tsm
ts=2022-07-25T06:58:25.611876Z lvl=info msg="TSM compaction (start)" log_id=0bu2EpjG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0bu2F0nl000 op_name=tsm1_compact_group op_event=start
ts=2022-07-25T06:58:25.611894Z lvl=info msg="Beginning compaction" log_id=0bu2EpjG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0bu2F0nl000 op_name=tsm1_compact_group tsm1_files_n=8
ts=2022-07-25T06:58:25.611899Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0bu2F0nl000 op_name=tsm1_compact_group tsm1_index=0 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/1850/000000130-000000004.tsm
ts=2022-07-25T06:58:25.611905Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0bu2F0nl000 op_name=tsm1_compact_group tsm1_index=1 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/1850/000000162-000000003.tsm
ts=2022-07-25T06:58:25.611912Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0bu2F0nl000 op_name=tsm1_compact_group tsm1_index=2 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/1850/000000171-000000002.tsm
ts=2022-07-25T06:58:25.611920Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0bu2F0nl000 op_name=tsm1_compact_group tsm1_index=3 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/1850/000000172-000000001.tsm
ts=2022-07-25T06:58:25.611927Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0bu2F0nl000 op_name=tsm1_compact_group tsm1_index=4 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/1850/000000174-000000001.tsm
ts=2022-07-25T06:58:25.611932Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0bu2F0nl000 op_name=tsm1_compact_group tsm1_index=5 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/1850/000000176-000000001.tsm
ts=2022-07-25T06:58:25.611937Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0bu2F0nl000 op_name=tsm1_compact_group tsm1_index=6 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/1850/000000178-000000001.tsm
ts=2022-07-25T06:58:25.611942Z lvl=info msg="Compacting file" log_id=0bu2EpjG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0bu2F0nl000 op_name=tsm1_compact_group tsm1_index=7 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/1850/000000180-000000001.tsm
ts=2022-07-25T06:58:26.070188Z lvl=info msg="TSM compaction (end)" log_id=0bu2EpjG000 engine=tsm1 tsm1_level=1 tsm1_strategy=level trace_id=0bu2F0n0000 op_name=tsm1_compact_group op_event=end op_elapsed=461.724ms
panic: keys must be added in sorted order: tsm1_engine,database=varken,engine=tsm1,hostname=c92104d17c1d,id=1998,indexType=inmem,path=/var/lib/influxdb/data/varken/varken\ 30d-1h/1998,retentionPolicy=varken\ 30d-1h,walPath=/var/lib/influxdb/wal/varken/varken\ 30d-1h/1998#!~#tsmLevel1CompactionErr < tsm1_engine,database=varken,engine=tsm1,hostname=c92104d17c1d,id=1998,indexType=inmem,path=/var/lib/influxdb/data/varken/varken\ 30d-1h/1998,retentionPolicy=varken\ 30d-1h,wa|Path=/var/lib/influxdb/wal/varken/varken\ 30d-1h/1998#!~#tsmLevel1CompactionDuration

goroutine 5803 [running]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*directIndex).Add(0xc003203920, 0x14abafcfe420, 0xfe, 0xd36a9, 0x1, 0x1704c770c8e68400, 0x1704c7e533390c00, 0xa99e7, 0xc00000001c)
        /go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/writer.go:337 +0x594
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*tsmWriter).WriteBlock(0xc003d33540, 0x14abafcfe420, 0xfe, 0xd36a9, 0x1704c770c8e68400, 0x1704c7e533390c00, 0x14abaf70cb3c, 0x18, 0x6c4f8d, 0x0, ...)
        /go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/writer.go:686 +0x1ea
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Compactor).write(0xc000fee1b0, 0xc004516730, 0x49, 0x2591ce0, 0xc001cbf900, 0xc004df6701, 0x0, 0x0)
        /go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/compact.go:1162 +0x287
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Compactor).writeNewFiles(0xc000fee1b0, 0x62, 0x2, 0xc0004c5380, 0x8, 0x8, 0x2591ce0, 0xc001cbf900, 0x1, 0x2591ce0, ...)
        /go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/compact.go:1045 +0x18f
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Compactor).compact(0xc000fee1b0, 0xc0004c5300, 0xc0004c5380, 0x8, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/compact.go:953 +0x40d
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Compactor).CompactFull(0xc000fee1b0, 0xc0004c5380, 0x8, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/compact.go:971 +0x17e
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*compactionStrategy).compactGroup(0xc007b03ab0)
        /go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:2259 +0x1b4e
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*compactionStrategy).Apply(0xc007b03ab0)
        /go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:2236 +0x4d
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactHiPriorityLevel.func1(0xc0030da510, 0xc0000d2500, 0x1, 0xc007b03ab0)
        /go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:2153 +0x12e
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).compactHiPriorityLevel
        /go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:2148 +0x123

** Press ANY KEY to close this window ** 

Hoffe mal das mir da jemand weiter helfen kann.