Add ability to install unRAID OS to redundant drives

[jonp]

Probably goes hand in hand with the cache / user data co-mingling capability feature and linux disk quota capabilities, but definitely something we want to have planned for sometime in the future, even if it's a new product offering.

[NAStyBox]

Would it be possible to have a second USB drive, with some sort of code on it that knows what the primary is?

So you would have your primary USB/boot drive, with your licensed UUID. The backup would be a UUID you register somewhere in the primary. If someone tries to use the backup to boot a system have it check for the existence of the primary. If the primary isn't present, have it throw a message... "You are trying to boot from a backup boot drive. If this is not your intention please press 'S' and your system will shut down. If you would like to replace your primary boot drive with this drive, press 'C' to continue and we will update our license database." Then have the backup reach out to your server and make the change. You may want to include some sort of authentication requirement via emailed link to ensure a stolen USB drive can't render a licensed system inoperable. 

...or something to that effect. I'm not a coder but it sounds feasible, no?

[HellDiverUK]

Just boot off the cache drive and be done with it...keep the USB for licencing, nothing more.

[pwm]

You want to boot from the USB - that doesn't give any wear and allows all system state to be backed up etc by duplicating the flash content. But the system could make a mirror copy to a cache drive and only verify that the USB stays connected with the license key. And then have the system accumulate a grace timer so if the USB breaks after x days uptime, then you may get a failure mail and a number of days to react and arrange a replacement.

 

Even nicer would be if the replacement drive can be completely empty - just connect an empty drive and click on a menu choice to have the unRAID machine report to lime with the new serial number. Then copy back the full information from the SSD and write down the new key file. So the only user interaction would be to unplug the broken drive, insert an empty drive and click "register USB replacement drive".

[MadMage999]

However it ends up being implemented, I second the request to have redundancy on the system files.

[Abzstrak]

This needs to be done, plenty of easy ways to make it happen and maintain license structure.

[HNGamingUK]

Yes, I would love to see this sooner than later. I don't mind loosing some cache space (all be it not much) so that we can have unraid running with redundancy. As at current anything goes wrong with the Flash and you reboot. That's it bye bye.

[Magicmissle]

I would like to see a PxE boot option for unraid

[-Daedalus]

That's actually a cool idea. +1 if possible.

[sota]

I think a lot of these are interesting ideas, but I can reasons why limetech might not want to do it.

Perhaps an automated backup of the USB stick to some designated place, with the ability of a virgin stick to quickly recover itself and reset the license key, might solve most concerns?  I know I personally try and run a flash backup once a month or so, or whenever I make any Big Changes(tm), and restore it to a spare stick that's kept with my cold storage backup, but maybe have that function also store a copy onto /appdata, in addition to forcing the download of the ZIP file?

 

I'm wondering where the concern is coming from also.  Are people burning out sticks at a rapid rate all of the sudden?

Edited February 21, 2020 by sota

[Squid]

I'm still using my original sticks from 2012-2013.  I think a lot of this is "false" failures.  Using poor quality sticks or counterfeit Kingston sticks from China,  unclean shutdowns corrupting the stick etc.

 

Considering just how little the stick is actually written to, it should easily outlast by far any given hard drive in your system.

 

[-Daedalus]

In theory, absolutely. But I've found that USB sticks, when left in all the time, run extremely hot. I use USB sticks in work for a document backup, writing maybe 20-50MB a day (so significantly more than unRAID, yes) and I've gone through three in the past year. Every time I take one out of the dock, it's too hot to hold for the first couple of seconds. I know it's more USB 3.0, but it's very difficult to find high quality 2.0 drives these days.

 

A relatively small concern, I suppose, but I would be a nice option to have none-the-less. 

[JonathanM]

41 minutes ago, -Daedalus said:

it's very difficult to find high quality 2.0 drives these days.

https://www.kingston.com/us/usb-flash-drives/datatraveler-se9-usb-flash-drive

Cheaply available on amazon.

[Pri]

I think it's probably time Unraid supported more boot methods. Now that it has a lot of advanced functionality like Docker, VM's and is becoming more critical to peoples setups more redundancy should be available.

 

I'm not suggesting that the USB method should be done away with, but it should be just one way to boot Unraid. As a new user to Unraid and someone who has built many redundant systems and writes server software (including resilient databases and cluster software) a few design choices do seem wrong to me but can be corrected. Please give me the benefit of the doubt with that statement as I don't mean to come in as the new guy and say you're doing it all wrong!

 

I do like that Unraid runs from memory, keeps it fast and keeps developers constrained, they know it has to fit in memory and cannot be a resource hog. But the 32GB USB limit isn't great, it's becoming more difficult to get high quality name brand USB drives in this capacity, not impossible but difficult. It's also becoming harder to get USB 2.0 only drives, high quality ones anyway.

 

These things can be mitigated by supporting USB 3.0 drives and higher capacity drives but then we move onto the USB flash quality. As the capacities have increased the NAND quality has decreased. We're in QLC territory now and even then with bargain barrel flash and at the extremely high transistor density level which lends itself to unreliability. Combined with that many of the good sticks run very hot and aren't at all intended to be left plugged into a USB port powered on for years at a time.

 

I echo some earlier posters statements about heat and how some of the sticks are so hot even when just left idle without any read/writes occurring that you cannot pull them from the USB port without being burned or feeling that it's uncomfortably hot to the touch.

 

Probably the final nail in the USB situation is you're performing writes to it when configuration changes are made but can't easily verify they worked and if you then later reboot or cold start the system that is when the USB drive is most likely to fail leaving you in a precarious situation. There's no RAID1 capability, we don't even have file checksums.

 

What I'd really like is the capability to install Unraid on normal drives with RAID1 just for the added redundancy and flexibility. It will also allow those of us who are building more critically depended upon servers to spend the amount commiserate with our systems. I'm currently building a wildly expensive Unraid server and the lynch-pin is a $15 USB stick. If I had the option of using a SATA DOM with high quality SLC flash or a SATA or NVMe SSD with SLC flash then I would, and I would likely use RAID1.

 

Also I recently discovered after using Unraid that you cannot enable or startup VM's unless the raid is started. And some on discord alluded to this being tied to the license which as you all are aware is tied to the USB sticks GUID. It seems very odd to me that anything should stop the VM's from being started independent of the Unraid disk array. They seem so separate to me, what if I have all my VM's on a pool device or a ZFS array.

 

If I'm doing some kind of disk operation on my main array that means it has to be offline for several days then I cannot start up any VM's? - Seems like bad design to me and it's probably why some people on the forums and on Discord are running Unraid under a more full featured hypervisor.

 

I know this issue isn't strictly USB drive related but it's all part of the same tangled mess. The licensing is tied to the USB drive, that's currently the only boot method, they're prone to failure, you can't run VM's without the array enabled which requires the USB drive for licensing etc

 

I fully intend to build a whole bunch of Unraid servers for myself and friends and this is probably the biggest sore point, when you're building in so much redundancy, be it dual PSU's, dual Nics, dual parity protected disk arrays, dual SSD's in RAID1 for your cache and/or VM's but then it all comes down to such a unreliable booting method, that just seems to me something that needs to be corrected for the project to keep progressing and improving, just how TrueNAS is adding better VM support, offering Linux with better drivers, just how pfSense now supports ZFS booting with RAID1 disks etc - All these projects that are in the same sphere are examining things that made sense years ago and saying ya know, today that doesn't make sense, that needs to evolve so we can better serve the needs of our users.

 

I hope my post comes across in the positive intent that I meant it, I'm definitely not saying the sky is falling I just want to express how important I feel more boot options are. This topic started in 2014, it's now 2022. I'd love to see this happen before the end of 2024 really.

[Hexenhammer]

+1