PsyCl0ne

Alright I got it sorted. I dont have proof of exactly what the issue was but by doing the following I was able to get everything working. 1) As per Fix Common Problems, I went into each share, deleted the last number for "Minimum free space" put that number back and applied the change. 2) The drives that I am removing from my system since they are listed under "Unassigned Devices" I had to mount all of them and I deleted the only file that I could see which was "data" after once again checking that there were no files in them. To do this I used File Manager in the unraid GUI. 3) I unmounted the drives and stopped the array. 4) Navigating into Settings > Global Share Settings I was now able to see all the drives I am keeping listed in the drop-down menu for "Included disk(s)". I enabled them all and applied. Then I hit the Done button. 5) Started the array back up and I got one more notification from Fix Common Problems for the ISOS share. I repeated step 1 to correct that issue. 6) Confirmed that all the missing directories were now visible under /mnt/user/. 7) Stopped the array once again, went back into the Global Share Settings, deselected all the drives from the inclusion list so that it showed all. Applied the change and then hit the Done button. 8)) Started the array once more and going into every share settings I can finally see all of my array drives in the Inclusion and Exclusion drop-down menus. 9) Quick skim through my data directory, it appears to now have what I would normally have expected prior to this mess.

Ah kk perfect. Apologies I started to doubt my response. But I'm glad we're getting some traction on this.

Unraid: 6.12.10, setup to be able to complete atomic moves. A couple of nights ago I went through the shrink array guide to remove multiple drives. My array setup at the time was a total of 17 drives. #1-8 were 4TB, #9-11 were 18TB, #12-13 were 4TB, and #14-17 were 18TB. I also have 2 parity drives. The goal was to remove all but 1 of the 4TB drives. Data from each of the to be removed drives were migrated over to a 18TB drive using unbalance, and I confirmed that there was no data left on the drives by browsing to the drive to see if there was anything left over. After completing the steps in "New Config" tool, and moving over to the main tab of unraid, I removed the drives I didnt want and rearranged the drives I wanted to keep starting from Disk 1. Parity rebuild has completed. PROBLEM: Navigating to /mnt/user/data/(Any folder) I see a tiny portion of directories that I would be expecting to see. Navigating to specific disks (/mnt/disk#/data/...) all the directories and all the files that I would expect to see are there. Permissions seem to be correct all the way down from /mnt/disk#/data to whichever directory I navigate to (drwxrwxrwx+ 2 nobody users). Fix Common Problems has a bunch of errors, but I am not sure what to do with them, as the shares appear to be showing correctly instead of what I changed when starting the shrink array guide steps. Although one thing that I do see is that when selecting the inclusion drop down list, there are no drives listed to select. Screenshots of Fix Common Problems and each of the share settings are attached. Last screenshot is of the global share settings. What is it that I have missed, making this process not complete properly? tower-diagnostics-20240412-1441.zip

@JorgeB Thank you, honestly you're a godsend. Thank you for your help once again.

JorgeB, apologies your help has been appreciated but while I understand what you have advised I am not sure of the steps to take. I know that normally with a failed drive if I take it out and then put in a new drive, I can rebuild the array. How would it work in this situation? Do I force the system to start a rebuild or do I take the drive out and start the rebuild? Would the array rebuild onto the 4 TB drives that are still in shape? Attached are screenshots of how my "Main" tab is looking for the drives and array options. A side note when I clicked on the little icon next to the drive it pulled up Logs? and I have the following showing in it for the drive that is disabled Nov 1 05:30:04 Tower emhttpd: spinning down /dev/sdo Nov 1 06:00:01 Tower emhttpd: read SMART /dev/sdo Nov 1 07:30:04 Tower emhttpd: spinning down /dev/sdo Nov 1 08:00:01 Tower emhttpd: read SMART /dev/sdo Nov 1 08:30:23 Tower emhttpd: spinning down /dev/sdo Nov 1 09:00:01 Tower emhttpd: read SMART /dev/sdo Nov 1 10:30:03 Tower emhttpd: spinning down /dev/sdo Nov 1 11:00:16 Tower kernel: sd 8:0:1:0: [sdo] tag#2775 CDB: opcode=0x85 85 06 20 00 00 00 00 00 00 00 00 00 00 40 e5 00 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2798 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=19s Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2798 Sense Key : 0x2 [current] Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2798 ASC=0x4 ASCQ=0x0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2798 CDB: opcode=0x88 88 00 00 00 00 06 33 f5 63 d0 00 00 04 00 00 00 Nov 1 11:00:20 Tower kernel: I/O error, dev sdo, sector 26641523664 op 0x0:(READ) flags 0x0 phys_seg 128 prio class 0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2799 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=19s Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2799 Sense Key : 0x2 [current] Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2799 ASC=0x4 ASCQ=0x0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2799 CDB: opcode=0x88 88 00 00 00 00 06 33 f5 5b d0 00 00 04 00 00 00 Nov 1 11:00:20 Tower kernel: I/O error, dev sdo, sector 26641521616 op 0x0:(READ) flags 0x0 phys_seg 128 prio class 0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2800 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=19s Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2800 Sense Key : 0x2 [current] Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2800 ASC=0x4 ASCQ=0x0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2800 CDB: opcode=0x88 88 00 00 00 00 06 33 f5 5f d0 00 00 04 00 00 00 Nov 1 11:00:20 Tower kernel: I/O error, dev sdo, sector 26641522640 op 0x0:(READ) flags 0x0 phys_seg 128 prio class 0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2801 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=19s Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2801 Sense Key : 0x2 [current] Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2801 ASC=0x4 ASCQ=0x0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2801 CDB: opcode=0x88 88 00 00 00 00 06 33 f5 4f d0 00 00 04 00 00 00 Nov 1 11:00:20 Tower kernel: I/O error, dev sdo, sector 26641518544 op 0x0:(READ) flags 0x0 phys_seg 128 prio class 0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2802 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=19s Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2802 Sense Key : 0x2 [current] Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2802 ASC=0x4 ASCQ=0x0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2802 CDB: opcode=0x88 88 00 00 00 00 06 33 f5 53 d0 00 00 04 00 00 00 Nov 1 11:00:20 Tower kernel: I/O error, dev sdo, sector 26641519568 op 0x0:(READ) flags 0x0 phys_seg 128 prio class 0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2803 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=19s Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2803 Sense Key : 0x2 [current] Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2803 ASC=0x4 ASCQ=0x0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2803 CDB: opcode=0x88 88 00 00 00 00 06 33 f5 57 d0 00 00 04 00 00 00 Nov 1 11:00:20 Tower kernel: I/O error, dev sdo, sector 26641520592 op 0x0:(READ) flags 0x0 phys_seg 128 prio class 0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2804 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2804 Sense Key : 0x2 [current] Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2804 ASC=0x4 ASCQ=0x0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2804 CDB: opcode=0x88 88 00 00 00 00 06 33 f5 67 d0 00 00 04 00 00 00 Nov 1 11:00:20 Tower kernel: I/O error, dev sdo, sector 26641524688 op 0x0:(READ) flags 0x4000 phys_seg 128 prio class 0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2815 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2815 Sense Key : 0x2 [current] Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2815 ASC=0x4 ASCQ=0x0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2815 CDB: opcode=0x88 88 00 00 00 00 06 33 f5 6b d0 00 00 04 00 00 00 Nov 1 11:00:20 Tower kernel: I/O error, dev sdo, sector 26641525712 op 0x0:(READ) flags 0x0 phys_seg 128 prio class 0 Nov 1 11:00:20 Tower emhttpd: read SMART /dev/sdo Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2761 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2761 Sense Key : 0x2 [current] Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2761 ASC=0x4 ASCQ=0x0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2761 CDB: opcode=0x8a 8a 00 00 00 00 06 33 f5 63 d0 00 00 04 00 00 00 Nov 1 11:00:20 Tower kernel: I/O error, dev sdo, sector 26641523664 op 0x1:(WRITE) flags 0x4000 phys_seg 128 prio class 0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2765 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2765 Sense Key : 0x2 [current] Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2765 ASC=0x4 ASCQ=0x0 Nov 1 11:00:20 Tower kernel: sd 8:0:1:0: [sdo] tag#2765 CDB: opcode=0x8a 8a 00 00 00 00 06 33 f5 67 d0 00 00 04 00 00 00 Nov 1 11:00:20 Tower kernel: I/O error, dev sdo, sector 26641524688 op 0x1:(WRITE) flags 0x4000 phys_seg 128 prio class 0 Nov 1 11:30:23 Tower emhttpd: spinning down /dev/sdo Nov 2 10:24:06 Tower emhttpd: read SMART /dev/sdo Nov 2 10:27:07 Tower emhttpd: shcmd (6623718): echo 128 > /sys/block/sdo/queue/nr_requests

Hey thank you for looking into the logs. What is the next best step? Reboot the server? Still swap the drive out? Am I able to still pull the data from this drive?

Hey everyone, just got 3 18TB HDDs, Completed preclear, without issue. While migrating from my smaller 4TB drives onto the new drives using unbalance one of the drives disabled itself. I have completed a long SMART test that came back clean so I am not sure what is going on. Attached are logs from the diagnostics page as well as the SMART log from the drive itself. Second question to go along with this, if the drive has actually shit the bed, can I use unbalance again to move the data off the drive back onto the 4TB drives or what is the best way to go about this? I have a new 18TB drive on its way but it will be another ~4 days at least. tower-smart-20231103-1459.zip tower-diagnostics-20231103-1456.zip

Alright so quick update, I have swapped the CPUs in their respective sockets after cleaning them up. Cleared the IPMI logs and have just got Unraid back up and running. Lets see what comes up in the logs next, but from the basic googling I have done it seems more likely that the PSU is starting to fail rather than the CPUs. On that topic, does anyone know where I could get a replacement Segate ss-400h2u? From what I am seeing in CAD its like $400-$500 "new". Or does anyone know if a Segate ss-600h2u would be swapable? I know that desktop PSUs are not to be trusted due to the cabling variations, regardless of brand or model. Does the same stand for server PSUs?

Apologies for the delayed response. I noticed that while in the IPMI I was unable to see any sensor information at all. Everything was showing and unknown/unavailable. What I have done since is actually reflash the BIOS and Firmware for my mobo, and now I am seeing proper errors and data from the sensors. I have attached all that I can from the system information screen and a new diagnostics output from unraid. From what I can see, IPMI log seems to be filling up with a bunch of CPU lines, but I am not 100% just yet on what it means. I wanted to get the info posted here before trying to figure out what it's trying to explain. Thank you for the reply, I'm not sure if I would have much faith in my equipment as I am still learning quite a bit and looks like there is still a ton more to learn. So maybe take my post with a grain of salt, could currently just be happenstance. IPMI Event Log.xlsx IPMI Sensor Readings + Threshholds.xlsx tower-diagnostics-20230312-1635.zip

Apologies, I don't follow. Its a dual socket motherboard, and both sockets have a X5670 CPU in them. Are you telling me to swap the CPUs in each socket? This server has been running for at least a couple of years in this config, if that is of any help.

Perfect, I figured out how to get the IPMI going. The only thing as of this year was Event ID Time Stamp Sensor Name Sensor Type Description 251 12/04/2023 16:41:04 Unknown [undefined] undefined - Asserted 250 09/18/2021 15:05:04 Unknown OS Critical Stop undefined - Asserted - Asserted

Apologies, how would I locate these logs? Is it the ones located at http://tower/log/syslog? If so, I have it attached, however I don't think I am seeing anything that stands out. It seems to be populated from after the server reset. syslog

Hey everyone, My server just crashed while I was streaming on Jellyfin, however this is a really strange crash as my server didn't fully restart. I'm not sure how to explain this, but I didn't hear it go through post and ramping up the chassis fans and then back down. I just heard the fans spin up for a moment and then go back down. However, unraid did have to boot back up, and I was able to connect to it once it was showing the IP address. The server is currently completing a parity check but in fix common problems I have the following error "Machine Check Events detected on your server" "Your server has detected hardware errors. You should install mcelog via the NerdPack plugin, post your diagnostics and ask for assistance on the Unraid forums. The output of mcelog (if installed) has been logged More Information" Trying to get more information for this error but installing NerdPack, I am unable to find that in the app store, I believe it was depreciated? There is, however, NerdTools, but I am unable to find that mcelog tool within it. I have attached the diagnostics from when the UI was accessible, but I am not sure if that would have information as to what happened. Any idea what direction I should move with this? tower-diagnostics-20230306-2238.zip

Thank you both, I don't have much experience with server hardware and didn't know that there was an option within the HBA to enable drives to boot from but it is sorted now! If anyone else does come across this issue, I have the following hardware: SAS9211-8i + SAS9207-8e. Info for the boot settings was found from https://support.osnexus.com/hc/en-us/articles/212131146-Setting-LSI-SAS-HBA-boot-support