Hi,

 

I have just added 3x SATA 1TB WD SA500 SSD drives to the system and created a zfs, raidz, 1 group of 3 devices.

 

I had a single nvme drive which hosted my VM's and docker containers, which I moved to the array then to the zfs cache (all without error). I had no docker containers so created a new docker image on the zfs cache which worked fine and installed plex also fine.

 

When I try and start the old Windows 11 virtual machine I get zfs offset errors and the VM page is unrepsonsive and the server will not reboot or shutdown. so I have re-powered the system (holding down the power button - as shutdown and reboot in the console did not work, not sure what else I should have done). tired going to settings -> Vm Manager -> Enable No (but just hangs)

 

Apr 15 20:56:52 Obsidian kernel: zio pool=zfs_cache vdev=/dev/sdc1 error=5 type=1 offset=121754525696 size=65536 flags=180880
Apr 15 20:56:52 Obsidian kernel: zio pool=zfs_cache vdev=/dev/sdd1 error=5 type=1 offset=121754521600 size=65536 flags=180880
Apr 15 20:56:52 Obsidian kernel: zio pool=zfs_cache vdev=/dev/sdb1 error=5 type=1 offset=121754521600 size=65536 flags=180880
Apr 15 20:56:52 Obsidian kernel: WARNING: Pool 'zfs_cache' has encountered an uncorrectable I/O failure and has been suspended.
Apr 15 20:56:52 Obsidian kernel: 
Apr 15 20:56:55 Obsidian kernel: WARNING: Pool 'zfs_cache' has encountered an uncorrectable I/O failure and has been suspended.
Apr 15 20:56:55 Obsidian kernel: 

 

root@Obsidian:~# zpool status -xv
  pool: zfs_cache
 state: SUSPENDED
status: One or more devices are faulted in response to IO failures.
action: Make sure the affected devices are connected, then run 'zpool clear'.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-HC
  scan: scrub repaired 0B in 00:04:16 with 0 errors on Mon Apr 15 20:40:48 2024
config:

        NAME        STATE     READ WRITE CKSUM
        zfs_cache   ONLINE       0     0     0
          raidz1-0  ONLINE       1     4     0
            sdc1    ONLINE       2     2     0
            sdb1    ONLINE       2     2     0
            sdd1    ONLINE       2     4     0

errors: List of errors unavailable: pool I/O is currently suspended

 

after the reboot, i did a scrub on the zfs cache which came back with no errors. i then decided to copy an new windows iso to the zfs cache and create a new VM, this worked with no reported zfs errors any ideas why the old VM image causes issues (wrong file system)???

 

Best regards,

Andy

After the forced power-off:

 

pool: zfs_cache
 state: ONLINE
  scan: scrub repaired 0B in 00:04:17 with 0 errors on Mon Apr 15 21:17:16 2024
config:

	NAME           STATE     READ WRITE CKSUM
	zfs_cache      ONLINE       0     0     0
	  raidz1-0     ONLINE       0     0     0
	    /dev/sdc1  ONLINE       0     0     0
	    /dev/sdb1  ONLINE       0     0     0
	    /dev/sdd1  ONLINE       0     0     0

errors: No known data errors

 

Now running a parity check on the main array . Should be fine it was spun down.

11 hours ago, Fluxonium said:

When I try and start the old Windows 11 virtual machine I get zfs offset errors

That suggests the SATA controller is being passedd-throught to that VM, please post the diagnostics.

You are correct, I thought by simply removing the IOMMU binding it would work. But looking at the XML VM config I also needed to remove the pass through settings by editing the xml.

 

Thanks for your help