Keliaxx

Anyone running 7.3 with a B50/B60/B70 that can confirm if the SR-IOV VFs are functional so that VFs can be passed to individual VMs? If so, this is a joyous day indeed, as having Unraid support a single GPU that can properly handle multiple VMs and containers has been a feature I've waited for years to see materialize.

Not supporting the creation of a configuration in the GUI is one thing, actively breaking something is a completely different story. If the upgrade to 7.1.0 couldn't properly import the ZFS configuration and support it, it should have just left it alone, not created some garbage configuration that required rollback and manual intervention with the config files to get back to an operational state. I'm glad everyone is pointing at TrueNAS because that is a good example of the right way to handle this sort of situation. The UI may not be there to build a ZFS pool this way, but it doesn't create a mess that prevents a simple ZFS import from working. While we're at it, consider importing from QNAP. Everything ZFS on that platform is done with multiple partitions per disk. Bottom line is pretty simple... If LT doesn't want to build out the UI to fully support ZFS, no problem. Just don't make claims about being able to support or import things that the product is unable to handle and, more importantly, don't perform actions on things that the product doesn't understand. The very first order of business in storage products is DO NO HARM. It seems that is sorely forgotten.

I have an UnRAID box with a zpool that consists of 7 total disks. 4 are spinning rust, 3 are NVME. The three NVME drives each contain 3 partitions. Those partitions are built into 3-way mirrors across all three drive, one mirror each for special, log, and cache. I have L2ARC persistence enabled. The spinning rust disks are LUKS encrypted. In UnRAID versions 7.0 and older, this configuration isn't understood and starting the array halts after unlocking the LUKS encrypted drives. Certainly a nuisance as I can't autostart, but navigable with a zpool import and restart of docker and libvirt (or not if you issue the zpool import at the right time after starting the array). Then along comes UnRAID 7.1.0 and the "ability to import zpools from other platforms"... Problem is, UnRAID still has no clue how to handle assignment of anything other than an entire disk to one of its "slots". I upgrade, it imports the fact that the pool has mirror devices with vdevs each for special, cache, and log and then proceeds to faceplant because it can't understand that the vdevs are made up of partitions, not entire physical drives. As a result, no way to start the array so rollback it is... In order to get back to an operational system, I had to roll back to 7.0 and remove the configs for special, cache, and log under /boot/config/pools. Until UnRAID FULLY supports zpool configurations that are valid in ANY other platform, it shouldn't be importing them or claiming that it can. When will partitions be supported in addition to allocation of entire physical disks? This half-baked ZFS implementation (as far as the UnRAID UI properly supporting ZFS goes) has existed for far too long.

Any idea when the native zfs implementation will support the use of partitions instead of entire physical drives for use as members of a vdev? Just wondering how long it will be before my array will actually work properly with the array start and array stop commands and I won't have to manually do a zpool import on startup and export prior to system shutdown.

Unfortunately, no. I haven't found a solution to the problem yet. In my case, its annoying but not causing me any major issues at the moment, aside from getting the warning every time I fire up a terminal session.

Just so I'm clear, you replaced the CPU section to have it reporting Intel architecture and attributes on a physical AMD platform?

Meaning *ONLY* on partition 1? The three NVME devices that are being used each have 3 paritions, paritition 1 on each is combined into a 3 way mirror VDEV for SLOG, partition 2 on each is a part of a 3-way mirror VDEV for SPECIAL, partition 3 on each is joined as a stripe VDEV for persistent CACHE. This is definitely something that should be considered for support in future releases, as the fault tolerance recommendations for SLOG and SPECIAL along with extremely high IOPs and low latency of 4xpcie4 NVME makes this model of using partitions instead of entire physical devices a lot more practical. I'm curious, how exactly is the zpool import being called on the back end that causes this part of it to fail? I can understand the UI getting confused about how to present it, but array start doesn't successfully import the pool.

I've been scouring the interwebs trying to find an answer to this and I read that it was fixed in kernel 6.1 (see here): https://bugzilla.kernel.org/show_bug.cgi?id=155211 Alas 6.12RC3 with kernel 6.1.23 seems to still exhibit this problem. When trying to use nested virtualization in Windows 11 for things like WSL and WSA, you get the following in the host dmesg. Would anyone happen to know if / what kvm cpu parameter voodoo might allow one to trick Windows 11 into playing nice? I've tried stripping the hypervisor property and forcing the svm capability flags with no luck. Within the respective VM, Windows is quick to announce that nested virtualization is not supported on the platform when you launch WSL2. [ 3830.698973] SVM: kvm [19055]: vcpu0, guest rIP: 0xfffff86757387841 unimplemented wrmsr: 0xc0010115 data 0x0 [ 3831.069250] SVM: kvm [19055]: vcpu1, guest rIP: 0xfffff86757387841 unimplemented wrmsr: 0xc0010115 data 0x0 [ 3831.193400] SVM: kvm [19055]: vcpu2, guest rIP: 0xfffff86757387841 unimplemented wrmsr: 0xc0010115 data 0x0 [ 3831.317537] SVM: kvm [19055]: vcpu3, guest rIP: 0xfffff86757387841 unimplemented wrmsr: 0xc0010115 data 0x0 [ 3831.441694] SVM: kvm [19055]: vcpu4, guest rIP: 0xfffff86757387841 unimplemented wrmsr: 0xc0010115 data 0x0 [ 3831.565861] SVM: kvm [19055]: vcpu5, guest rIP: 0xfffff86757387841 unimplemented wrmsr: 0xc0010115 data 0x0 [ 3831.690037] SVM: kvm [19055]: vcpu6, guest rIP: 0xfffff86757387841 unimplemented wrmsr: 0xc0010115 data 0x0 [ 3831.814213] SVM: kvm [19055]: vcpu7, guest rIP: 0xfffff86757387841 unimplemented wrmsr: 0xc0010115 data 0x0 [ 3831.938362] SVM: kvm [19055]: vcpu8, guest rIP: 0xfffff86757387841 unimplemented wrmsr: 0xc0010115 data 0x0 [ 3832.062556] SVM: kvm [19055]: vcpu9, guest rIP: 0xfffff86757387841 unimplemented wrmsr: 0xc0010115 data 0x0

I followed the steps mentioned and have pretty much the same results, except I now have the 3 NVME devices listed in the pool along with the 4 hard drives. I can still manually import it after starting the array and it works, but starting the array with all 7 devices defined doesn't import it during the array startup. I've attached the updated diagnostic report. If necessary, I have the space to vacate the pool and rebuild it if there is a different process I need to follow. Please note that that SLOG, SPECIAL, and CACHE VDEVs are pointed at partitions on the same 3 NVME devices. ZFS doesn't care, but I wanted to point it out in case it might have been overlooked. Because these VDEVs were added from the CLI after the pool was created from the web UI, they aren't encrypted with luks. If I need to rebuild to pool, I can vacate the data and do that. I would just need to know what steps to follow to make the UI happy with the resulting pool.

Attached is the diagnostic report after starting the array. At the point where this was captured, the zfs pool mainpool shows all 4 of the 20TB drives as unformatted, however they are unlocked by luks. From this point, if I just go to the cli, I can do a zpool import mainpool and it brings up and mounts the zpool without issues. From there, I just need to disable and re-enable VMs and everything works fine, but the UI never shows the pool correctly. Here is the topology of the zfs mainpool after import from the command line: pool: mainpool state: ONLINE scan: scrub repaired 0B in 00:22:50 with 0 errors on Sun Apr 23 23:38:41 2023 config: NAME STATE READ WRITE CKSUM mainpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 sde1 ONLINE 0 0 0 sdf1 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 sdg1 ONLINE 0 0 0 sdh1 ONLINE 0 0 0 special mirror-3 ONLINE 0 0 0 nvme4n1p2 ONLINE 0 0 0 nvme5n1p2 ONLINE 0 0 0 nvme6n1p2 ONLINE 0 0 0 logs mirror-2 ONLINE 0 0 0 nvme4n1p1 ONLINE 0 0 0 nvme5n1p1 ONLINE 0 0 0 nvme6n1p1 ONLINE 0 0 0 cache nvme4n1p3 ONLINE 0 0 0 nvme5n1p3 ONLINE 0 0 0 nvme6n1p3 ONLINE 0 0 0 errors: No known data errors

FWIW and to help anyone else that gets into a weird state like this, I did find a workaround that allowed me to create the second pool I needed. I temporarily renamed the pool configuration file under /boot/config/pools to [poolname].cfg.bak and rebooted. This way, it just saw all the disks that were previously part of that pool as unused and allowed me to format only the disks associated with the new pool. Once they were formatted and all was in good order, I just stopped the array, put the pool cfg file back and rebooted. I still have the issue with the zpool requiring manual import every time I reboot, but the newly added pool works fine.

Just to make sure I capture diagnostics from the correct workflow and get you the most useful set of data... Starting from a reboot and the array not configured for autostart, what steps would you like me to complete before capturing the diagnostics? Array start, command line zpool import, or both?

I'm working with 6.12 beta RC3. I have a large, raidz2 encrypted ZFS pool and the UI doesn't know how to handle it after I added mirrored NVME ZIL and Special devices, along with a striped L2ARC. Yes, I know how they work and when to use them and when not to with ZFS, so I'm hoping this stays on topic about the formatting question. Anyway, the ZFS pool works exactly as I had planned, but unraid has no idea how to handle it properly when starting the array, so you have to import it manually each time you start the array and then it works fine, but the UI complains that the disks in the array are unformatted. Annoying, but not a show stopper until I tried to add another pool. The problem now is that I can't format the disks in the new pool because the format option wants to format ALL of the disks that it THINKS are unformatted, including the disks in my main zfs pool? Is there a way to manually format just the disks that make up a specific pool and leave the rest untouched? Alternatively, has anyone had success fixing the UI after adding ZIL, special, and L2ARC to ZFS manually?