Drives mounted as read-only after a while

[mattenz]

Hi all,

I'm having an intermittent issue where my Docker containers will stop after a while due to trying to write to a read-only file. After restarting the array, fix common problems will flag that problem.

Rebooting the server resolves the issue temporarily, but it will reoccur within a few days. As far as I can see, neither the array or cache are full. I'd appreciate some guidance on where to start tracking down this problem.

 

 

Edited Tuesday at 08:32 PM by mattenz
Removing diagnostics

[JorgeB]

 

Mar 17 08:29:02 Tower kernel: BTRFS info (device sdh1): bdev /dev/sdi1 errs: wr 556872763, rd 1666244, flush 8528958, corrupt 26137804, gen 0

One of your pool devices dropped offline in the past and is out of sync, run a correcting scrub and post the output.
 

[mattenz]

Okay, would that be:

btrfs scrub start -Bd /mnt/cache

 

I get:

Scrub device /dev/sdh1 (id 1) done
Scrub started:    Mon Mar 20 09:50:37 2023
Status:           finished
Duration:         0:10:35
Total to scrub:   465.73GiB
Rate:             303.26MiB/s
Error summary:    csum=2
  Corrected:      0
  Uncorrectable:  2
  Unverified:     0

Scrub device /dev/sdi1 (id 3) done
Scrub started:    Mon Mar 20 09:50:37 2023
Status:           finished
Duration:         0:15:59
Total to scrub:   465.73GiB
Rate:             200.80MiB/s
Error summary:    super=3 verify=17451 csum=6709649
  Corrected:      6727098
  Uncorrectable:  2
  Unverified:     0
ERROR: there are uncorrectable errors

 

Edited Sunday at 09:08 PM by mattenz

[JorgeB]

Please post the diagnostics after the scrub, if you rebooted since run a new one and post after that.

[mattenz]

On 3/21/2023 at 12:26 AM, JorgeB said:

Please post the diagnostics after the scrub, if you rebooted since run a new one and post after that.

Thanks, attached are the new diagnostics.

 

Edited 7 hours ago by mattenz

[JorgeB]

(path: appdata/Nginx-Proxy-Manager-Official/data/logs/proxy-host-1_access.log)
(path: appdata/Nginx-Proxy-Manager-Official/data/logs/proxy-host-2_access.log)

 

The corrupt files are listed in the syslog, like the examples above, these need to be deleted/restored from backup, then run another scrub to confirm there are no more uncorrectable errors, also the Lexar SSD appears to be failing, probably a good idea to replace it.