I went to fire up Plex and my server was offline... I took a look at my NAS and it was true, Plex was stopped. In fact all of my containers had stopped!

 

All of my disks were fine but my log was full of errors. 

 

I'm attaching the diagnostics if anybody wants to take a look.

 

Latest version of Unraid (6.6.5)

 

No idea what the cause was, I've just rebooted now so hopefully all is good 🤞

raptor-diagnostics-20181115-2127.zip

Hmm...

 

2:16 Raptor kernel: loop: Write error at byte offset 2392465408, length 4096.
Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672784
Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 2, rd 0, flush 1, corrupt 0, gen 0
Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 4215803904, length 4096.
Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 8233992
Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 3, rd 0, flush 1, corrupt 0, gen 0
Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392465408, length 4096.
Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672784
Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 4, rd 0, flush 1, corrupt 0, gen 0
Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392473600, length 4096.
Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672800
Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 5, rd 0, flush 1, corrupt 0, gen 0
Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392608768, length 4096.
Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4673064
Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 6, rd 0, flush 1, corrupt 0, gen 0
Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392465408, length 4096.
Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672784
Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 7, rd 0, flush 1, corrupt 0, gen 0
Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 4367192064, length 4096.
Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 8529672
Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 8, rd 0, flush 1, corrupt 0, gen 0
Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392465408, length 4096.
Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672784
Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 9, rd 0, flush 1, corrupt 0, gen 0
Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392469504, length 4096.
Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672792
Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 10, rd 0, flush 1, corrupt 0, gen 0
Nov 15 21:32:16 Raptor kernel: loop: Write error at byte offset 2392489984, length 4096.
Nov 15 21:32:16 Raptor kernel: print_req_error: I/O error, dev loop2, sector 4672832
Nov 15 21:32:16 Raptor kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 11, rd 0

Crap?

 

-------

 

Nov 15 21:31:57 Raptor kernel: BTRFS error (device sdl1): unable to find ref byte nr 2331165265920 parent 0 root 5 owner 18608628 offset 4014080
Nov 15 21:31:57 Raptor kernel: BTRFS: error (device sdl1) in __btrfs_free_extent:6950: errno=-2 No such entry
Nov 15 21:31:57 Raptor kernel: BTRFS info (device sdl1): forced readonly
Nov 15 21:31:57 Raptor kernel: BTRFS: error (device sdl1) in btrfs_run_delayed_refs:3057: errno=-2 No such entry
Nov 15 21:31:57 Raptor kernel: BTRFS error (device sdl1): pending csums is 3751936

 

/dev/sdl

Edited November 15, 20187 yr by nexusmaniac

 

This says it all

Nov 15 21:31:57 Raptor kernel: BTRFS info (device sdl1): forced readonly

Wait for @johnnie.black, the resident btrfs guru.

In these cases always best to backup, format and restore cache data.

Is it possible to figure out the cause? 

 

Last thing I want to do is encounter the same issue again 'soon'

 

Looks like I'd best get backing up in the meantime then

27 minutes ago, nexusmaniac said:

Is it possible to figure out the cause? 

Nov  9 08:37:01 Raptor kernel: BTRFS info (device sdl1): bdev /dev/sdl1 errs: wr 22108, rd 3126, flush 303, corrupt 86, gen 88

Lots of read and write errors on one of the your cache devices, this happened in the past, before current logs, but likely it dropped offline, with SSDs this is usually the result of a cable problem, suggest you replace both cables and keep monitoring.