"failed to open stream: Read-only file system" - Array Usable, but can't Update

[TheNore]

I have been having issues Making any changes to my system because of the "failed to open stream: Read-only file system" error.

I had an error on one of my drives and it seemed to corrupt the XFS file system.

I have performed many searches, and have not found a solid solution on how to move forward.

I am making the post as a final resort. I know this issue has happened to others in the past, but I feel as if i am in a unique position.

This happened around the time I did the 6.11 update from 6.10. There is probably no correlation though.

I had to manually convert to 6.11.5. I also did a "repair" in windows, on my USB stick, but no files were found corrupt, and no change in the issue.

 

 

The error in the log after every boot:

emhttpd: Warning: file_put_contents(/boot/config/plugins/parity.check.tuning/parity.check.tuning.manual): failed to open stream: Read-only file system in /usr/local/emhttp/plugins/parity.check.tuning/parity.check.tuning.helpers.php on line 150

 

Edited November 27, 2022 by TheNore
Grammar

[itimpi]

The error message you give us saying that there is a problem writing to the flash drive (which is what is mounted at /boot).   Have you tried putting the flash drive into a PC or Mac to check it?

[TheNore]

Yes, It was one of the first things I tried. I noted it above. Are there any additional steps that I can try for a repair? 

 

I also get error similar to this when trying to update anything on the system. Community apps, docker, unraid updates, etc... 

Edited November 28, 2022 by TheNore

[JorgeB]

Please post the diagnostics.

[TheNore]

4 hours ago, JorgeB said:

Please post the diagnostics.

Here you go.

megaraid-diagnostics-20221128-0817.zip

[JorgeB]

Backup and re-format the flash drive.

[TheNore]

33 minutes ago, JorgeB said:

Backup and re-format the flash drive.

OK. Just drag the file structure back on after formatting? 

[JorgeB]

No, recreate the flash drive manually or using the USB tool and restore only the config folder.

[TheNore]

2 hours ago, JorgeB said:

No, recreate the flash drive manually or using the USB tool and restore only the config folder.

Performed exact steps, used the utility, did a full format, and dragged config back into the new install and I am still having the same errors ;( 

 

I am also receiving the following error:

Nov 28 12:28:30 MegaRAID  emhttpd: Warning: file_put_contents(/boot/config/plugins/parity.check.tuning/parity.check.tuning.manual): failed to open stream: Read-only file system in /usr/local/emhttp/plugins/parity.check.tuning/parity.check.tuning.helpers.php on line 150
Nov 28 12:28:30 MegaRAID  emhttpd: Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 262144 bytes) in /usr/local/emhttp/plugins/parity.check.tuning/parity.check.tuning.helpers.php on line 216

 

Edited November 28, 2022 by TheNore

[JorgeB]

Strange, try booting in safe mode.

[TheNore]

12 minutes ago, JorgeB said:

Strange, try booting in safe mode.

I did reboot in safe mode, and I get random errors (below) when starting the array. but no Read-Only file system from the dynamix plugin. However, the array always thinks it needs to do a parity check because of an "unsafe shutdown" on the previous boot. It can not "write" to the files that say a parity check has been performed. I have performed about 3 on reboots in the last week, but on reboot it still thinks it had an error shutting down.

 

Error 1:

Nov 28 12:59:11 MegaRAID kernel: md8: running, size: 3907018532 blocks
Nov 28 12:59:11 MegaRAID kernel: md9: running, size: 7814026532 blocks
Nov 28 12:59:11 MegaRAID  emhttpd: shcmd (155): udevadm settle
Nov 28 12:59:11 MegaRAID  emhttpd: error: put_disk_settings, 3998: Read-only file system (30): fopen: /boot/config/disk.cfg
Nov 28 12:59:11 MegaRAID  emhttpd: Opening encrypted volumes...
Nov 28 12:59:11 MegaRAID  emhttpd: shcmd (156): touch /boot/config/forcesync
Nov 28 12:59:11 MegaRAID root: touch: cannot touch '/boot/config/forcesync': Read-only file system

 

Error 2:

Nov 28 12:59:16 MegaRAID  emhttpd: shcmd (186): mount -t btrfs -o noatime,space_cache=v2,discard=async -U 04df26e0-59f3-42e9-acbd-74f0f7912042 /mnt/cache
Nov 28 12:59:16 MegaRAID kernel: BTRFS info (device nvme0n1p1): turning on async discard
Nov 28 12:59:16 MegaRAID kernel: BTRFS info (device nvme0n1p1): using free space tree
Nov 28 12:59:16 MegaRAID kernel: BTRFS info (device nvme0n1p1): has skinny extents
Nov 28 12:59:16 MegaRAID kernel: BTRFS info (device nvme0n1p1): bdev /dev/sdm1 errs: wr 34, rd 0, flush 22, corrupt 0, gen 0
Nov 28 12:59:16 MegaRAID kernel: BTRFS info (device nvme0n1p1): enabling ssd optimizations
Nov 28 12:59:16 MegaRAID  emhttpd: error: put_disk_settings, 3998: Read-only file system (30): fopen: /boot/config/disk.cfg

 

Error 3:

Nov 28 12:59:21 MegaRAID root: Starting libvirtd...
Nov 28 12:59:21 MegaRAID kernel: tun: Universal TUN/TAP device driver, 1.6
Nov 28 12:59:21 MegaRAID kernel: mdcmd (36): check correct
Nov 28 12:59:21 MegaRAID kernel: md: recovery thread: check P ...
Nov 28 12:59:21 MegaRAID kernel: write_file: error 30 opening /boot/config/super.dat
Nov 28 12:59:21 MegaRAID kernel: md: could not write superblock file: /boot/config/super.dat

 

 

 

[JorgeB]

I would try with a different flash drive.

[TheNore]

1 hour ago, JorgeB said:

I would try with a different flash drive.

 

I honestly didn't think this would fix the issue  

But it did! Thank you for your help - What could have caused this to happen?! 

[JorgeB]

Looks like a flash drive problem, though strange that there weren't issues when reformatting