Hello community,

 

yesterday my NAS suddenly froze during use. No web UI, no share access and no SSH access. Because nothing else worked, I restarted the PC manually. After restarting I only got the message "cache - too many missing/wrong devices" in the UI when starting the array. After that I tried it with a reboot. After this reboot all disks were suddenly no longer available. After another reboot nothing worked, no UI and no SSH connection possible. Today I connected the PC to the screen to see what happens. After the boot sequence shows some disturbing errors the output ends with "Logger: send Message failed: Bad file descriptor". I can then log in via ssh, but the system does not respond to any input. When I start in safe mode I get a UI again. At first there is nothing to see. No missing disks or other notifications. When I try to start the array I get the error "cache - too many missing/wrong devices" again. Also in maintenance mode the array can not be started.
 

I must mention that I installed two cache ssd, which should provide redundancy. One of the two SSDs apparently even broke a few weeks ago, as it is no longer listed. Even changing the slots did not change anything. Therefore, the NAS has been running only with "cache 2" lately. Since I suspected that for some reason the cache must be suddenly populated again, so that the NAS can start I have an old SSD connected and wanted to restore "cache". But when I want to start the array again, it remains with the error message. 

Since I would probably create more problems if I tried anything now, I would like to ask for help. I have no idea what the logs are supposed to tell me. does anyone have any ideas?

 

nas-diagnostics-20230822-2225.zip syslog-6.txt IMG_9691.HEIC

Post the output of:

btrfs fi show

 

Hey, this is the output

 

Label: none  uuid: def06ab6-5d13-4d32-8a61-08a2ab429729
        Total devices 1 FS bytes used 488.04GiB
        devid    2 size 931.51GiB used 829.06GiB path /dev/nvme0n1p1

 

Pool is reporting being a single device, since it's devid #2 likely devid #1 was removed sometime in the past, unassign all pool devices, start array, stop array, assign only nvme0n1, start array, post new diags.

I followed the instruction and now I can start the array in safe mode. Unfortunately, this is limited to safe mode. When I boot normally the boot process hangs again with the message "Logger: send Message failed: Bad file descriptor". as I write this text a new message has actually appeared and I can use the terminal. A look at /boot looks scary. 

Run chkdsk on the flash drive, or backup the /config folder and recreate.

I ran chkdsk /f /r and /x on the flash drive. It found some sector errors in ./git and repaired them, nothing else was found. After booting in normal mode I have the corrupted directories again, but in safe mode I don't see any errors like the ones in normal mode. I was also able to copy all files from the drive to my Mac and Windows PC without any warnings.

I could now format the USB stick and copy the data back to it, but I am not sure if the USB stick is the cause of the error.

 

Edited August 24, 20232 yr by andr0id

Thy re-formatting the same flash drive and recreate it manually or using the USB tool, then restore only the /config folder from the backup, if after that you still have issues in normal mode, and not in safe mode, it's likely another issue.

Oh, it seems that the problem is solved. I can start Unraid and the array. No errors seem to appear in the system logs either. Thank you very much!!!

 

One last question: Is the way I configured the cache with two drives (Cache and Cache2) (see screenshot above) the right way to prevent data loss if one of the two cache SSDs dies? 

Yes, if they are using the default raid1 profile, in doubt you can post new diags to confirm.

Btw, I'm also getting the logger: send message failed: Bad file descriptor error on boot but the system boots just fine (at least that I know of). What does that error imply?

PS: I'm using Unraid inside Proxmox.

That should be harmless, but you can try booting in safe mode to rule out a plugin issue.