@JorgeB

A question on boot pool mirroring that came up in the Reddit beta thread worth clarifying here — specifically because the original question and answer were ambiguous in a way that matters.

In the Reddit r/unRAID 7.3 beta thread approximately 3 days ago:

Known_Palpitation805 asked: "Will we get boot drive mirroring if we use a multi DOM setup or is it restricted to SSDs etc?"

UnraidOfficial replied: "Yes."

The ambiguity is in the word "DOM" — which could mean two completely different things with different implications for boot pool support:

USB DOM — an industrial flash device connecting via USB protocol through an internal motherboard USB header. Subject to what the onboarding wizard identifies as USB_TRANSPORT exclusion — dconklin's bug report from this thread documented "USB_TRANSPORT: This disk is connected over USB, which is not allowed for internal boot" as a specific disk check failure.

SATA DOM — an industrial flash device connecting via SATA interface.

Treated identically to any SATA SSD from Unraid's perspective with no USB exclusion applicable.

Two specific questions:

Does the USB_TRANSPORT exclusion apply to boot pool membership generally — meaning USB DOMs on internal headers cannot be used in a mirrored boot pool regardless of their industrial-grade NAND quality?

If USB DOMs are excluded, can standard USB flash drives be mirrored through any mechanism in 7.3 — or is boot pool mirroring restricted exclusively to SATA and NVMe devices?

The reason this matters practically — USB DOMs on internal headers represent the optimal boot device hardware for always-on Unraid duty based on NAND type and thermal characteristics.

If they're excluded from boot pool mirroring by the USB_TRANSPORT restriction, that's worth documenting clearly so users evaluating their boot device options understand the constraint.

If they're supported, that's equally worth documenting as it represents an accessible high-quality mirrored boot configuration without consuming SATA or M.2 slots.

Edited March 22Mar 22 by Lolight

No. It's ZFS.

A couple of clarifications on the boot partition — it is ZFS as @Niklas noted, confirmed officially in the 7.3 release notes and by the Unraid team directly.

On mirroring — ZFS software mirroring operates entirely at the OS level and doesn't require BIOS RAID support.

That's one of ZFS's primary architectural advantages.

Two drives in a ZFS mirror are managed by the ZFS layer without any BIOS involvement.

The specific question I've asked @JorgeB above is whether USB-connected devices — USB DOMs on internal headers specifically — are supported in the ZFS boot pool mirror, or whether the USB_TRANSPORT exclusion documented in the onboarding wizard applies to boot pool membership generally.

That distinction matters for users evaluating industrial USB DOM hardware as a boot option.

Edited March 22Mar 22 by Lolight

2 hours ago, Lolight said:

The specific question I've asked @JorgeB above is whether USB-connected devices — USB DOMs on internal headers specifically — are supported in the ZFS boot pool mirror, or whether the USB_TRANSPORT exclusion documented in the onboarding wizard applies to boot pool membership generally.

You can use USB devices, you can create, for example, a boot pool from a couple of flash drives.

Just note that in that case you cannot use one of them for licensing, meaning you would need to use a 3rd flash drive for licensing, typically the one you are already using, (or TPM if available).

22 minutes ago, JorgeB said:

You can use USB devices, you can create, for example, a boot pool from a couple of flash drives.

Thank you — just want to make sure I'm understanding this correctly before sharing it more widely.

The configuration you're describing would be:

Two USB flash drives or DOMs forming the mirrored boot pool — providing redundancy.

A separate third USB drive holding the licence key as before — or TPM if available.

Is that correct?

And the USB_TRANSPORT exclusion that appeared in the onboarding wizard's disk checks — does that apply specifically to the currently active boot device being reassigned, rather than to USB devices generally as boot pool members?

If the three-device USB mirrored configuration is confirmed it seems worth prominent documentation — it addresses the single-point-of-failure concern for users without available internal slots or TPM hardware.

26 minutes ago, Lolight said:

Is that correct?

Yes.

27 minutes ago, Lolight said:

And the USB_TRANSPORT exclusion that appeared in the onboarding wizard's disk checks

Where did you see this? I believe this was being considered initially, not allowing USB for boot pools, but the ban was lifted before the public release, For sure, you can use USB devices for a boot pool (or any other pool); I've tested that.

5 minutes ago, JorgeB said:

Where did you see this?

The USB_TRANSPORT exclusion appears in dconklin's bug report from this thread -- posted Thursday at 7:39 PM.

His onboarding wizard showed it as a disk check failure reason.

Good to know it was a development-stage consideration that didn't make it to release.

This confirmation changes the practical picture significantly.

A mirrored boot pool from two quality USB flash drives or industrial DOMs -- with the existing license USB drive remaining in place -- addresses the single-point-of-failure concern for the broadest possible user base without requiring internal slots, TPM hardware, or NAND quality decisions beyond the existing USB guidance.

Given that this wasn't mentioned in the promotional materials or tutorial videos it's likely totally unknown to most users currently evaluating their boot options.

Worth prominent documentation -- it's the most accessible redundant boot configuration available and it works with hardware most Unraid users already have.

Edited March 22Mar 22 by Lolight

One thing I forgot to mention, boot pool devices need to be larger than 8GB, since currently Unraid uses less than 50% for the boot pool area, and the minimum for that is 4GB, so 8GB flash drives won't work, 16GB and larger will.

Thanks to @JorgeB confirmations above this is worth highlighting for anyone currently evaluating their boot options.

A mirrored boot pool from two USB flash drives or industrial DOMs is confirmed supported in 7.3 -- USB devices work in boot pools, tested by JorgeB.

The configuration is:

Two USB flash drives or DOMs forming the mirrored ZFS boot pool -- providing redundancy against single drive failure.

The existing USB license drive remaining in place as the license holder -- or TPM if available.

One important constraint JorgeB flagged -- boot pool devices need to be 16GB minimum.

The 16GB minimum restriction applies specifically to boot pool member devices — because those devices get the 50% partition rule applied to them during boot pool setup.

The license drive bypasses that entire partition process entirely.

This addresses the single-point-of-failure concern without requiring internal drive slots, TPM hardware, SATA or M.2 ports.

For users with internal USB headers:

two industrial USB DOMs of 16GB or larger or USB drives mounted internally via inexpensive 9-pin to USB-A adapters in the boot pool mirror plus the existing license drive on an external port or internal adapter is a particularly clean configuration -- redundant boot, no internal slots consumed, full hardware agnosticism preserved.

The 16GB minimum means NAND type verification matters for the boot pool drives -- the same NAND quality considerations that apply to internal NVMe selection apply here.

Verified 3D TLC or better from a reputable manufacturer at 16GB or above is the target.

The USB Flash section guide covers NAND type identification and verification.

This configuration wasn't mentioned in the promotional materials or tutorial videos.

The USB Flash section guide will be updated to include it as a confirmed supported configuration.

Edited March 22Mar 22 by Lolight

@JorgeB

One more clarification if you don't mind — two questions:

Does the onboarding wizard present USB flash drives as selectable boot pool devices in its interface, or does creating a USB mirrored boot pool require manual configuration outside the wizard?

And on the license drive — in the mirrored boot pool configuration where the license USB drive serves purely as a license holder with no boot function, is there a minimum size requirement for that drive?

The key file itself is kilobytes in size so theoretically any recognized USB device should work — but whether the licensing system has its own minimum size requirement independent of the boot drive requirements isn't documented anywhere I can find.

I installed it on my test server, migrated the license to TPM and migrated to internal boot. Feedback:

• The "Internal Boot FAQ (7.3+)" does not explicity explain, that this is only possible with a pool device and not with an array device. This was only mentioned in the videos

• The wizard does not explain, that it automatically copies the files of the usb flash drive, it should explain this and list the specific folders, which will be synced. I was happy, that it automatically synced /logs and /extra, but getting a visual confirmation about this copy process would be really nice.

• The wizard should be moved to Tools I think

• The wizard should not ask for already existing settings like language, timezone, design, default applications, etc., instead it should help only through the migration steps

• The wizard should rename the volume name of the flash drive after migration to "UNRAID_OLD" or similar, so it can be mounted through unassigned devices without additional steps (the UD pluging returns only an error, that it is not possible to mount (the reason was the volume label)

• Is there specific reason why only ZFS is supported? I'm absolutely not a fan of loosing 1 GiB of RAM only because of the boot partition:
  

• I think the reboot time is slower now, but I forgot to do multiple reboot benchmarks before migrating to internal boot

• The file manager links shouldn't be next to each other. Instead they should be next to the partition:
  

• Any reason to keep the boot devices section?
  

• The backup FAQ needs infos regarding internal boot:
  https://docs.unraid.net/unraid-connect/automated-flash-backup/#restoring-a-flash-backup
  Currently I would say it is needed to stop the array and overwrite all files in /boot?!

9 minutes ago, mgutt said:

The wizard should rename the volume name of the flash drive after migration to "UNRAID_OLD" or similar, so it can be mounted through unassigned devices without additional steps (the UD pluging returns only an error, that it is not possible to mount the flash drive as long its volume name is "UNRAID")

👍

44 minutes ago, mgutt said:

• Is there specific reason why only ZFS is supported?

I am also curious about this. Until and unless BTRFS is supported internal boot is of no use to me.

54 minutes ago, primeval_god said:

I am also curious about this. Until and unless BTRFS is supported internal boot is of no use to me.

Maybe say for what reason?

I am sooo impatient to be able to boot from a spare NVMe in a PCie slot so that I can ditch the USB boot drive and pass through my one and only XHCi controller to my VMs when needed. Have had a number of issues with interrupts with different devices when passed through individually, noticeaby a Webcam which has both Video and Sound devices and I still have intermittent problems with Teams!

All things being equal, when would the R.C. version likely be ready for consumption?

Edited March 22Mar 22 by Bob_C
Tidy up

2 hours ago, Kilrah said:

Maybe say for what reason?

I dont use zfs, anywhere on anything. My unpopular opinion is that Btrfs is the superior file system for use in the home lab. If the choice is internal boot with zfs or continuing to boot from a flash drive I will stick with a flash drive.

Edited March 22Mar 22 by primeval_god

2 hours ago, primeval_god said:

I dont use zfs, anywhere on anything. My unpopular opinion is that Btrfs is the superior file system for use in the home lab. If the choice is internal boot with zfs or continuing to boot from a flash drive I will stick with a flash drive.

Before the beta released the video showed it using btrfs, but it was changed to zfs cause btrfs caused problems (not sure which). Better have something that works than something that doesn't I guess.

Has the 7.3.0-beta.1 broken Frigate for anyone else?

Given 7.3.0 is now using Docker 29, could an option for the containerd snapshotters storage driver be made available via the GUI (and also the default for new installations, following the default for Docker 29)?

It seems to be a much better option than even overlay2 now.

16 hours ago, mgutt said:

• Is there specific reason why only ZFS is supported? I'm absolutely not a fan of loosing 1 GiB of RAM only because of the boot partition:
  

Unused RAM is useless RAM, but other that this contentious note; note that the ZFS ARC cache is programmed to release the space when under memory pressure.

20 hours ago, Lolight said:

Does the onboarding wizard present USB flash drives as selectable boot pool devices in its interface

Yes, as long as they are >8GB.

20 hours ago, Lolight said:

is there a minimum size requirement for that drive?

Nope, once you boot from a pool, the flash drive is not even mounted; the licensed file is also copied to the boot pool.

luks:zfs pool status is shown as UNKNOWN even though the pool is evidently online

16 hours ago, primeval_god said:

I am also curious about this. Until and unless BTRFS is supported internal boot is of no use to me.

The initial plan was to use Btrfs, which I'm also a big fan of, it has some advantages over ZFS, also some weaker points, notably the handling of dropped devices; they are not automatically brought up to date as with ZFS, and if the pool is kept being used while degraded, it will start creating single chunks for data and metadata, requiring a balance to RAID1 once the old device is brought online.

But those were being dealt with; the showstopper was an issue that was found, which is not directly a Btrfs problem but a Grub problem with its btrfs driver.

That driver is needed to first detect and load the pool devices, and it's used until the kernel driver is loaded later. Now the problem was that if you had a mirrored Btrfs pool with a stale device, and both were connected, it would fail to boot from the most recent one; it could only boot from the stale one. Example:

• Boot pool was a mirror consisting of sda and sdb

• While the server is online, sdb drops offline, let's say due to a bad cable

• The user reboots and checks/replaces the bad cable, or the device just comes back online by itself so that both devices are back online

• If the user tried to boot from sda, Grub would fail to load that filesystem, and it would just splat on boot

• Curiously, it could always boot from sdb, the stale device; just the more recent one would always fail.

After some research, it was found this was an old, known issue with btrfs and Grub, which likely won't be fixed any time soon, so after spending some days trying to work around it, LT decided to pivot to zfs, I'm sure it wasn't an easy decision, since it meant abandoning previous work done, and it ended up delaying the beta by 2 or 3 weeks, but in the end, I think it was the right decision to make the pool boot as reliable as possible.

5 minutes ago, andrebrait said:

luks:zfs pool status is shown as UNKNOWN even though the pool is evidently online

Known issues with encrypted pools, shoudl be fixed for beta.2.

2 hours ago, andrebrait said:

Given 7.3.0 is now using Docker 29, could an option for the containerd snapshotters storage driver be made available via the GUI (and also the default for new installations, following the default for Docker 29)?

It seems to be a much better option than even overlay2 now.

@JorgeB not sure whether this works for this case, but I have also opened a GitHub issue for it here: https://github.com/unraid/webgui/issues/2583

I assume it would just entail adding the option and what it implies in the generated docker config, but I haven't had the time to try it on my own yet.