Fluxonium

Yes, the Startech PEXESAT322I worked fine. Thanks for the help.

Thanks I have ordered the Startech PEXESAT322I arrives tomorrow. I will report back after testing

Hi All, Can anybody recommend a PCIe 3.0 x1 SATA 1x adapter? I need to connect a DVD / Blue-Ray to a Virtual Machine and have run out of motherboard connections. I did ty the following from amazon: MZHOU PCIe SATA Card 2 Port But the system gets stuck at the BIOS / motherboard logo screen. Tested it in both PCIe 3.0 1x ports and the same result, I can get past the boot screen if I disconnect the DVD drive but with it plugged in the system will not post. I have not tested with a HDD but given I need the DVD drive I see no point. I have tested the DVD drive in a motherboard SATA connection and it works fine so I think the problem is the cheap amazon adapter. Best regards, Andy

Hi All, Thanks for your suggestions, I have ordered another Intel X540-T2 adapter (this time Dell, instead of HP) from eBay. This will allow me to rule out the TP-Link card and test the HP Intel X540-T2 adapter. Maybe I have a broken / half working NIC??? I have found a few post online regarding the Marvell AQtion 10Gbit which is in the TP-Link.

OK, so turned on Jumbo Packet to 9014 and set the unraid MUT to 9000 and get the following: Unraid Client root@Obsidian:~# iperf3 -c 192.168.10.1 Connecting to host 192.168.10.1, port 5201 [ 5] local 192.168.10.100 port 55066 connected to 192.168.10.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 879 MBytes 7.37 Gbits/sec 0 489 KBytes [ 5] 1.00-2.00 sec 875 MBytes 7.34 Gbits/sec 0 437 KBytes [ 5] 2.00-3.00 sec 878 MBytes 7.36 Gbits/sec 0 437 KBytes [ 5] 3.00-4.00 sec 886 MBytes 7.43 Gbits/sec 0 454 KBytes [ 5] 4.00-5.00 sec 879 MBytes 7.37 Gbits/sec 0 454 KBytes [ 5] 5.00-6.00 sec 882 MBytes 7.40 Gbits/sec 0 489 KBytes [ 5] 6.00-7.00 sec 883 MBytes 7.41 Gbits/sec 0 437 KBytes [ 5] 7.00-8.00 sec 860 MBytes 7.22 Gbits/sec 0 454 KBytes [ 5] 8.00-9.00 sec 877 MBytes 7.36 Gbits/sec 0 507 KBytes [ 5] 9.00-10.00 sec 882 MBytes 7.39 Gbits/sec 0 437 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 8.57 GBytes 7.37 Gbits/sec 0 sender [ 5] 0.00-10.00 sec 8.57 GBytes 7.36 Gbits/sec receiver Unraid Server: ----------------------------------------------------------- Server listening on 5201 (test #1) ----------------------------------------------------------- Accepted connection from 192.168.10.1, port 62206 [ 5] local 192.168.10.100 port 5201 connected to 192.168.10.1 port 62207 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 945 MBytes 7.93 Gbits/sec [ 5] 1.00-2.00 sec 948 MBytes 7.95 Gbits/sec [ 5] 2.00-3.00 sec 952 MBytes 7.98 Gbits/sec [ 5] 3.00-4.00 sec 954 MBytes 8.00 Gbits/sec [ 5] 4.00-5.00 sec 954 MBytes 8.00 Gbits/sec [ 5] 5.00-6.00 sec 960 MBytes 8.05 Gbits/sec [ 5] 6.00-7.00 sec 956 MBytes 8.02 Gbits/sec [ 5] 7.00-8.00 sec 950 MBytes 7.97 Gbits/sec [ 5] 8.00-9.00 sec 958 MBytes 8.03 Gbits/sec [ 5] 9.00-10.00 sec 951 MBytes 7.98 Gbits/sec [ 5] 10.00-10.00 sec 3.38 MBytes 7.92 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 9.31 GBytes 7.99 Gbits/sec receiver ----------------------------------------------------------- Server listening on 5201 (test #2) ----------------------------------------------------------- not sure why this worked as i thought jumbo packets are normally not required. Maybe the issue is the TP-Link 10GB card in the windows 11 pc as it is a Marvell AQtion 10Gbit??? or the Intel X540-AT2 NIC has an issue? wrong firmware... (not sure how i would check).

Cable is 3m CAT6A also tested 3M CAT6 and CAT5E cable same result.

Unraid as the client: root@Obsidian:~# iperf3 -c 192.168.10.1 Connecting to host 192.168.10.1, port 5201 [ 5] local 192.168.10.100 port 47914 connected to 192.168.10.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 238 MBytes 2.00 Gbits/sec 0 283 KBytes [ 5] 1.00-2.00 sec 234 MBytes 1.96 Gbits/sec 0 294 KBytes [ 5] 2.00-3.00 sec 240 MBytes 2.01 Gbits/sec 0 286 KBytes [ 5] 3.00-4.00 sec 247 MBytes 2.07 Gbits/sec 0 311 KBytes [ 5] 4.00-5.00 sec 251 MBytes 2.10 Gbits/sec 0 297 KBytes [ 5] 5.00-6.00 sec 268 MBytes 2.25 Gbits/sec 0 277 KBytes [ 5] 6.00-7.00 sec 261 MBytes 2.19 Gbits/sec 0 288 KBytes [ 5] 7.00-8.00 sec 259 MBytes 2.17 Gbits/sec 0 280 KBytes [ 5] 8.00-9.00 sec 245 MBytes 2.05 Gbits/sec 0 272 KBytes [ 5] 9.00-10.00 sec 257 MBytes 2.15 Gbits/sec 0 277 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.44 GBytes 2.10 Gbits/sec 0 sender [ 5] 0.00-10.00 sec 2.44 GBytes 2.09 Gbits/sec receiver Unraid as the server: ----------------------------------------------------------- Server listening on 5201 (test #1) ----------------------------------------------------------- Accepted connection from 192.168.10.1, port 60681 [ 5] local 192.168.10.100 port 5201 connected to 192.168.10.1 port 60682 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 1.01 GBytes 8.68 Gbits/sec [ 5] 1.00-2.00 sec 1.01 GBytes 8.67 Gbits/sec [ 5] 2.00-3.00 sec 1.01 GBytes 8.66 Gbits/sec [ 5] 3.00-4.00 sec 1.00 GBytes 8.61 Gbits/sec [ 5] 4.00-5.00 sec 1.00 GBytes 8.61 Gbits/sec [ 5] 5.00-6.00 sec 1.02 GBytes 8.76 Gbits/sec [ 5] 6.00-7.00 sec 1.01 GBytes 8.69 Gbits/sec [ 5] 7.00-8.00 sec 1.01 GBytes 8.69 Gbits/sec [ 5] 8.00-9.00 sec 1.01 GBytes 8.65 Gbits/sec [ 5] 9.00-10.00 sec 1.00 GBytes 8.63 Gbits/sec [ 5] 10.00-10.00 sec 4.78 MBytes 8.51 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 10.1 GBytes 8.67 Gbits/sec receiver ----------------------------------------------------------- Server listening on 5201 (test #2) -----------------------------------------------------------

Hi All, I have purchased an Intel X540-AT2 dual port 10Gbe card to replace the motherboard 2.5Gbe network card. Installed and working but i have some odd things happening: Copy a 10GB file from Windows to Unraid i get the full 10Gb speed, but copy any file from the array, or cache to Windows PC am capped at 230mb/s any ideas ? Windows 11: Intel i7 12700K 32GB 3600 DDR RAM 1TB SN850 TP-Link 10 Gigabit PCI Express Network Adapter Unraid: Intel i5-2500 48GB 3200 DDR4 RAM NVME cache pool 1 - 2x 1TB WDC_WDS100T2B0C in btrfs mirror SATA cache pool 2 - 3x 1TB WDC_WDS100T1R0A in zfs array Intel X540-AT2 dual port 10Gbe connecting in the main PCIex16 Gen4 slot. Port 1 - 1gb switch connection - DHCP from router Port 2 - static ip to WIndows 10GB card Thanks, Andy

You are correct, I thought by simply removing the IOMMU binding it would work. But looking at the XML VM config I also needed to remove the pass through settings by editing the xml. Thanks for your help

Now running a parity check on the main array . Should be fine it was spun down.

After the forced power-off: pool: zfs_cache state: ONLINE scan: scrub repaired 0B in 00:04:17 with 0 errors on Mon Apr 15 21:17:16 2024 config: NAME STATE READ WRITE CKSUM zfs_cache ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 /dev/sdc1 ONLINE 0 0 0 /dev/sdb1 ONLINE 0 0 0 /dev/sdd1 ONLINE 0 0 0 errors: No known data errors

Hi, I have just added 3x SATA 1TB WD SA500 SSD drives to the system and created a zfs, raidz, 1 group of 3 devices. I had a single nvme drive which hosted my VM's and docker containers, which I moved to the array then to the zfs cache (all without error). I had no docker containers so created a new docker image on the zfs cache which worked fine and installed plex also fine. When I try and start the old Windows 11 virtual machine I get zfs offset errors and the VM page is unrepsonsive and the server will not reboot or shutdown. so I have re-powered the system (holding down the power button - as shutdown and reboot in the console did not work, not sure what else I should have done). tired going to settings -> Vm Manager -> Enable No (but just hangs) Apr 15 20:56:52 Obsidian kernel: zio pool=zfs_cache vdev=/dev/sdc1 error=5 type=1 offset=121754525696 size=65536 flags=180880 Apr 15 20:56:52 Obsidian kernel: zio pool=zfs_cache vdev=/dev/sdd1 error=5 type=1 offset=121754521600 size=65536 flags=180880 Apr 15 20:56:52 Obsidian kernel: zio pool=zfs_cache vdev=/dev/sdb1 error=5 type=1 offset=121754521600 size=65536 flags=180880 Apr 15 20:56:52 Obsidian kernel: WARNING: Pool 'zfs_cache' has encountered an uncorrectable I/O failure and has been suspended. Apr 15 20:56:52 Obsidian kernel: Apr 15 20:56:55 Obsidian kernel: WARNING: Pool 'zfs_cache' has encountered an uncorrectable I/O failure and has been suspended. Apr 15 20:56:55 Obsidian kernel: root@Obsidian:~# zpool status -xv pool: zfs_cache state: SUSPENDED status: One or more devices are faulted in response to IO failures. action: Make sure the affected devices are connected, then run 'zpool clear'. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-HC scan: scrub repaired 0B in 00:04:16 with 0 errors on Mon Apr 15 20:40:48 2024 config: NAME STATE READ WRITE CKSUM zfs_cache ONLINE 0 0 0 raidz1-0 ONLINE 1 4 0 sdc1 ONLINE 2 2 0 sdb1 ONLINE 2 2 0 sdd1 ONLINE 2 4 0 errors: List of errors unavailable: pool I/O is currently suspended after the reboot, i did a scrub on the zfs cache which came back with no errors. i then decided to copy an new windows iso to the zfs cache and create a new VM, this worked with no reported zfs errors any ideas why the old VM image causes issues (wrong file system)??? Best regards, Andy

OK, after some more testing it looks like the issue is the QNAP QM2-2P-374 as the nvme drives work in the motherboard slots without error (so far :)). I am guessing this is a Linux / unraid driver issue as the card works fine in Windows.

Unfortunately, "nvme_core.default_ps_max_latency_us=0 pcie_aspm=off" did not work. Strange thing is I cannot break the drive in Windows both work perfectly (on the same hardware other than the boot disk), going to try the following but cannot see it working unless Windows is not doing something unraid is (better / more stable drivers). i have even tried sleeping / hibernate from Windows and both drives still work. Remove adapter and swap drive positions (clean drive and adapter contacts) Remove both drives from the adapter and insert into motherboard and re-test

"nvme_core.default_ps_max_latency_us=0 pcie_aspm=off" I will give it ago, seems unlikely to fix this issue as the other WDS100T2B0C has no issue. I have also tested the drives and controller in Windows using using HD sentinel - Extended SMART test then 2x write - read surface scans - all with no issues, also copied enough data to fill the drive to 90% in windows and no issues or warnings, using the same hardware just a different boot drive, so looks like an issue with Linux / unraid for some reason. i am also getting the following in the log from time to time: Mar 27 16:03:29 Obsidian kernel: pcieport 0000:00:1c.4: AER: Multiple Corrected error received: 0000:08:00.0 Mar 27 16:03:29 Obsidian kernel: nvme 0000:08:00.0: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID) Mar 27 16:03:29 Obsidian kernel: nvme 0000:08:00.0: device [15b7:5009] error status/mask=00000001/0000e000 Mar 27 16:03:29 Obsidian kernel: nvme 0000:08:00.0: [ 0] RxErr Mar 27 16:24:19 Obsidian kernel: pcieport 0000:00:1c.4: AER: Multiple Corrected error received: 0000:09:00.0 Mar 27 16:24:19 Obsidian kernel: nvme 0000:09:00.0: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID) Mar 27 16:24:19 Obsidian kernel: nvme 0000:09:00.0: device [15b7:5009] error status/mask=00000001/0000e000 Mar 27 16:24:19 Obsidian kernel: nvme 0000:09:00.0: [ 0] RxErr device 15b7:5009 is the QNAP nvme to PCIe adapter both nvme3n1 and nvme2n1 are connected to: IOMMU group 22: [15b7:5009] 08:00.0 Non-Volatile memory controller: Sandisk Corp SanDisk Ultra 3D / WD Blue SN550 NVMe SSD (rev 01) [N:2:1:1] disk WDC WDS100T2B0C-00PXH0__1 /dev/nvme2n1 1.00TB IOMMU group 23: [15b7:5009] 09:00.0 Non-Volatile memory controller: Sandisk Corp SanDisk Ultra 3D / WD Blue SN550 NVMe SSD (rev 01) [N:3:1:1] disk WDC WDS100T2B0C-00PXH0__1 /dev/nvme3n1 1.00TB BIOS has been updated to the latest version and all drive firmware has been checked in Windows for the latest version. Thanks for your help.