EH5

I forgot to mention I checked the Immich config and it looked fine as far as I could tell. Immich has been working fine for a while.

I removed power from the troublesome drive and rebooted. I got the memory notice again. I hate to be dense but I can't figure out how to control the memory used by Immich. I found the menu to control CPU but not memory. Which menu has that option? I have attached the diagnostics again just in case tower-diagnostics-20240701-1347.zip

My server has 32 Gigs of ram. I was preclearing one disc and running an extended smart test on another when I got the out of memory message. message. The fix common problems tool says to post the diagnostics here to figure out what to do if anything. tower-diagnostics-20240629-1802.zip

I upgrade my power supply to get sufficient sata power connections and switched to a case with easier access to swap out the drives. Tings appeared fine but I got a memory error. I will start a new thread for that as I think all the issues I was dealing with in this tread are fixed.

I moved everything off the emulated disks and unplugged them. When I reconfigured without them and tried to restart the array and rebuild parity it shows that another disk is disconnected, the Dockers didn't start, and the same issue where the CPU Usage diagnostics don't show and the parity rebuild is not advancing. Is it possible that my UNRAID USB drive is corrupted somehow? tower-diagnostics-20240621-1847.zip

Yes there are power splitters. It is a 600W power supply but not enough HDD plugs

I disconnected that drive (WD-WX11D19A4C2V) and the system started. It looks like drive 3 and 4 of the array may just be toast. Should I just remove them from the array? what is the proper procedure for this? tower-diagnostics-20240619-1037.zip

WD-WX11D19A4C2V was one of the non-array disks I was testing to see if it would pass preclear

I shut down from the array operation menu on the Main tab and swapped the cables on disk 3. When I powered up it indicated that it was not a clean shut down. Interestingly, when I checked the notices it reported successful parity check from a couple of days ago. Now disk 3 and 4 show failures and the array just says starting but has not started. the fix common problems just says disk 4 is disabled and refers me to the main tab. tower-diagnostics-20240618-1808.zip

I had been transferring files from portable drives to the array and moving files off of a drive in the that has issues before shutting the array down to fiddle with the failed drive when, after a break a parity check had started. The CPU was at nearly 100% utilization which slowed everything down so I paused the parity check to finish what I was doing and restarted the parity check. It says running but is making no progress and the CPU utilization graphics on the Dashboard don't seem to be working any more. I tried to pause the parity check and even cancel it but after confirming there is no impact on the condition of the parity check. I am including the current diagnostic file. I didn't want to try a reboot until I check here. tower-diagnostics-20240618-1058.zip

That got the dockers up and running. The missing disk is not 366. The 366 disk is the one I installed to run preclear (it failed) so I think it is a bad disk. I will run the extended smart test. The missing disk is actually not showing up anywhere. That is why I was considering shutting down and checking all the connections. I was thinking either something is loose or it really failed hard. Just to confirm I should run check file system a second time on disk 6? In the UI it looks ok right now. Thanks for your help.

Here is the new diagnostic file. I have not rebooted the system since I got disk 6 back up tower-diagnostics-20240604-0741.zip

Disk 6 seems to have been recovered but the Dockers have still not started. Should I just follow the instructions I see on other docker failed to start posts? Also, disk3 lists as not installed. I could shut down the server and try to re-seat the power and data cables. Is that a good idea? I was able to click on the icon next to disk 3 and see all the files so it seems like they are emulated even though there is no notice listing the drive as emulated. Would it be reasonable to use unballanced to move everything off drive3 before doing anything else?

I could not figure out the formatting so here is a text file which may be easier to read Filesystemreport.txt

Here is the top of the report: <?>Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... Metadata CRC error detected at 0x47191d, xfs_inobt block 0x18/0x1000 btree block 0/3 is suspect, error -74 root inode chunk not found Phase 3 - for each AG... - scan and clear agi unlinked lists... found inodes not in the inode allocation tree found inodes not in the inode allocation tree - process known inodes and perform inode discovery... - agno = 0 148c620e4680: Badness in key lookup (length) bp=(bno 0x1b800, len 4096 bytes) key=(bno 0x1b800, len 16384 bytes) 148c620e4680: Badness in key lookup (length) bp=(bno 0x1b820, len 4096 bytes) key=(bno 0x1b820, len 16384 bytes) 148c620e4680: Badness in key lookup (length) bp=(bno 0x491a80, len 4096 bytes) key=(bno 0x491a80, len 16384 bytes) 148c620e4680: Badness in key lookup (length) bp=(bno 0x491aa0, len 4096 bytes) key=(bno 0x491aa0, len 16384 bytes) 148c620e4680: Badness in key lookup (length) bp=(bno 0x8ae040, len 4096 bytes) key=(bno 0x8ae040, len 16384 bytes) 148c620e4680: Badness in key lookup (length) bp=(bno 0x8ae060, len 4096 bytes) key=(bno 0x8ae060, len 16384 bytes) 148c620e4680: Badness in key lookup (length) bp=(bno 0xf1d180, len 4096 bytes) key=(bno 0xf1d180, len 16384 bytes)<?>