Desperate to un-f#$% my server

[xjp99]

While trying to get GPU passthrough to work I was swapping the GPU and my sata expansion card pci slots and when I rebooted one of my parity and disks had a red x and says "device is disabled." I tried xfs_repair on the disk and it completed but nothing changed. None of my shares are showing up and I'm freaking out. Not sure where to go from here.  

Edited September 17, 2023 by xjp99

[Gragorg]

Your going to need to post your diagnostic for the gurus to look at.

[Frank1940]

Diagnostics       🙂

Edited September 17, 2023 by Frank1940

[itimpi]

Do you pass any  hardware through to a VM?   If so chances are that when you changed the hardware installed the IDs associated with the passed-through hardware changed and you are now passing through something that should not be passed through.   Providing the system's diagnostics would allow us to confirm this.

[xjp99]

13 hours ago, Frank1940 said:

Diagnostics       🙂

smithserver-diagnostics-20230918-0719.zip

[xjp99]

4 hours ago, itimpi said:

Do you pass any  hardware through to a VM?   If so chances are that when you changed the hardware installed the IDs associated with the passed-through hardware changed and you are now passing through something that should not be passed through.   Providing the system's diagnostics would allow us to confirm this.

I tried but when I updated the Windows VM's configuration it gave me an error about IOMMU group or something. That's when I read that I may have to swap pcie devices around. I did that and all hell broke loose. I believe I may have shut down to fast because its now yelling at me that I had an unclean shutdown and the UI locks up constantly. 

[JorgeB]

Check/replace cables for cache1 and post new diags after array start.

[xjp99]

6 hours ago, JorgeB said:

Check/replace cables for cache1 and post new diags after array start.

smithserver-diagnostics-20230918-1504.zip

 

I appreciate the help. 

[xjp99]

I will also add that I removed the 2 drives that were erroring out. The UI was/is still locking up and not shutting down when commanded. 

[JorgeB]

ATA errors are gone, the disk controller is being passed through to the Windows 10 VM, remove these lines for the XML:

   

 <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/drivers for VMs/Asus.GTX1050Ti.4096.161020.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>

 

New diags after doing this and starting the array.

[xjp99]

8 hours ago, JorgeB said:

ATA errors are gone, the disk controller is being passed through to the Windows 10 VM, remove these lines for the XML:

   

 <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/drivers for VMs/Asus.GTX1050Ti.4096.161020.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>

 

New diags after doing this and starting the array.

Ok. Side issue. I am getting this error when trying to stop the array:

Sep 19 11:33:17 SmithServer kernel: XFS (md4p1): metadata I/O error in "xfs_buf_ioend+0x111/0x384 [xfs]" at daddr 0x80 len 32 error 5

[JorgeB]

Check filesystem on that disk, if the array doesn't stop, type reboot on the CLI, if it doesn't reboot after 5 minutes you'll need to force it.

[xjp99]

On 9/19/2023 at 12:22 PM, JorgeB said:

Check filesystem on that disk, if the array doesn't stop, type reboot on the CLI, if it doesn't reboot after 5 minutes you'll need to force it.

UPDATE: OK, I decided to disable VMs on this server and just use this one for a document/media server. I will build a second server for virtualization. I managed to do a parity rebuild on Parity 1 disk and it completed successfully. I am not so lucky on disk 5. It still says the disk is "disabled and the contents are emulated." I also have an error on one of the cache drives that I am not sure how to fix.

 

smithserver-diagnostics-20230924-0550.zip

[JorgeB]

for disk5 - https://docs.unraid.net/unraid-os/manual/storage-management#rebuilding-a-drive-onto-itself

 

cache - acknowledge current SMART errors and if you get more UDMA CRC errors replace the SATA cable, also same for disk5 SMART warning.

[xjp99]

Everything seems happy. I got all disks rebuild. Not sure if I ended up with any data loss. My configs for prowlarr, radarr, sonarr and vaultwarden were missing. Thanks for the help.