A few questions before ordering pro license

[Corvinus]

I'm considering the pro license for a supermicro 24 bay.

 

1) Is there any SMART monitoring of the USB boot stick? Are there important data being written to it that may fail silently and lead to corruption? I like that the system is copied and run from RAM after boot; but I expect some data are still being updated on it, based on user actions.

 

2) If a second USB stick fails before the next eligible date to receive a replacement key, what are my options to get back up and running? Why even have such delay instead of a constant ability - e.g. via web account login - to release and re-register the same license?

 

3) In general regards to driver support, is Unraid well supported on most Supermicro mainboards?

 

4) I've used Drivepool before, but I don't like the 50% space requirement of duplication. Single or double parity level is attractive in terms of space. Snapraid is file-level and won't rebuild ACL/permissions/user rights so it's not viable for what I want. I expect Unraid to do keep file system level information intact since I assume it works on block level? 

 

5) Is the Unraid metadata resilient enough that I could place the drives quickly on another server, another HBA/SATA controller, different order, and it would come up as a recognized array on a freshly built USB stick with default config?

 

6) Are there any connectivity dependencies for booting Unraid? Does the server need to be connected to Internet to e.g. validate license?

 

Thanks.

[Hoopster]

2 hours ago, Corvinus said:

1) Is there any SMART monitoring of the USB boot stick? Are there important data being written to it that may fail silently and lead to corruption? I like that the system is copied and run from RAM after boot; but I expect some data are still being updated on it, based on user actions.

There is a plugin app (Backup/Restore Appdata) that will periodically backup the flash drive to a location outside your array; an unassigned device is a good idea as the array is inaccessible if unRAID cannot be booted from the flash drive.  The flash drive is only accessed when booting the server and updating configurations.  In general, there is not a lot of activity on the flash drive except in rare cases of having the syslog server write to the flash drive when trying to troubleshoot issues.

 

2 hours ago, Corvinus said:

2) If a second USB stick fails before the next eligible date to receive a replacement key, what are my options to get back up and running? Why even have such delay instead of a constant ability - e.g. via web account login - to release and re-register the same license?

You can contact Limetech if you need more than one replacement within a year.  They are generally very understanding in cases of legitimate problems.

 

2 hours ago, Corvinus said:

3) In general regards to driver support, is Unraid well supported on most Supermicro mainboards?

Yep, lot and lots of unRAID users with SuperMicro boards.  They are very well supported (even older ones).

 

2 hours ago, Corvinus said:

4) I've used Drivepool before, but I don't like the 50% space requirement of duplication. Single or double parity level is attractive in terms of space. Snapraid is file-level and won't rebuild ACL/permissions/user rights so it's not viable for what I want. I expect Unraid to do keep file system level information intact since I assume it works on block level? 

In unRAID, each data disk has its own filesystem and can be read independent of the array outside unRAID on a system that supports the unRAID filesystems  - XFS, BTRFS and even the older ReiserFS which is no longer recommended.  ZFS is supported via plugin and may soon become a native supported filesystem in unRAID. 

 

The parity drives have no filesystem and are just buckets of bits holding the parity calculation for every corresponding block on all the data drives.

 

Each supported filesystem has some repair tools which generally work very well if the filesystem on a particular drive is somehow corrupted.

 

2 hours ago, Corvinus said:

5) Is the Unraid metadata resilient enough that I could place the drives quickly on another server, another HBA/SATA controller, different order, and it would come up as a recognized array on a freshly built USB stick with default config?

unRAID tracks disks by serial number.  It generally does not care to what or how they are connected (a few exceptions).  If you have a freshly built USB stick with defaults and you have the same data disks connected even to a different machine or different HBA, just create a new config and assign the data disks to data slots and you will have a new array with all your data intact.  There are some special considerations with disk order if you have dual parity, but, not with single parity.  If you forgot which is which, there is also a way to discover which are your previous parity disk(s) so you don't accidentally assign them as data disks and lose your parity data by starting the array.

 

2 hours ago, Corvinus said:

 

6) Are there any connectivity dependencies for booting Unraid? Does the server need to be connected to Internet to e.g. validate license?

Only with a trial license.  Once licensed, you never have to be connected to the Internet to boot unRAID.  The license is associated with the GUID of the flash drive.  No online validation is needed.

Edited January 31, 2021 by Hoopster

[Corvinus]

Thank you. Sounds very good.

 

Just to be clear about 4). When restoring files and folders from parity data in Unraid, permissions and ownership are intact? If the parity was calculated and restored on block level, it should be. Snapraid does it within file level, so it's always lost. I need it to be intact.

[JonathanM]

23 minutes ago, Corvinus said:

Just to be clear about 4). When restoring files and folders from parity data in Unraid, permissions and ownership are intact?

You can't restore files and folders, only complete disks. As long as parity is accurate when the disk went offline, the emulated drive is a bit level copy. Unraid does its best to keep parity valid, but things like power cuts during a write can corrupt parity, so if an unclean shutdown is detected, a parity check is triggered, and it's a good idea to do periodic (monthly, typically) parity checks just to make sure all the drives are returning good data.

 

Parity has no concept of files, folders, or file systems.

[Corvinus]

3 minutes ago, jonathanm said:

You can't restore files and folders, only complete disks. As long as parity is accurate when the disk went offline, the emulated drive is a bit level copy. Unraid does its best to keep parity valid, but things like power cuts during a write can corrupt parity, so if an unclean shutdown is detected, a parity check is triggered, and it's a good idea to do periodic (monthly, typically) parity checks just to make sure all the drives are returning good data.

 

Parity has no concept of files, folders, or file systems.

 

Thanks, sounds good.

 

I forgot to ask, is there any regular surface scans/long diagnostic SMART tests scheduling in Unraid? I liked that Stablebit Scanner did this, and I do it today via HD Sentinel on some servers to be pro-active and detect pending and/or reallocated sector failures early.

[JonathanM]

1 minute ago, Corvinus said:

I forgot to ask, is there any regular surface scans/long diagnostic SMART tests scheduling in Unraid?

A parity check reads all sectors on all drives in the parity array.

 

You can initiate and check results of SMART tests in the GUI, and if you wanted, you could script them.

[Hoopster]

13 minutes ago, jonathanm said:

Parity has no concept of files, folders, or file systems.

@Corvinus  This an important point.  Many wrongly assume parity is a backup of the data on all the drives.  How can one (or even two) 12TB parity drives have a full backup of several 12TB data drives?

 

All the parity drive contains is a bunch of bits that contain the parity calculation for that location for ALL the drives.  If a drive needs to be rebuilt or replaced, parity can rebuild the drive by using the parity calculation and reading ALL the other drives (they must all be readable) to figure out what is missing.

 

Now, if you need to rebuild a drive, yes it will come back exactly as it was.  This will include all folders, files permissions, etc.  It will also include any errors or corruption introduced to parity before rebuilding; thus the need for a monthly parity check to ensure parity integrity.

 

And, yes SMART checks can be run on demand or scheduled via a script.

Edited January 31, 2021 by Hoopster

[Corvinus]

2 minutes ago, Hoopster said:

@Corvinus  This an important point.  Many wrongly assume parity is a backup the data on all the drives.  How can one (or even two) 12TB parity drives have a full backup of several 12TB data drives?

 

All the parity drive contains is a bunch of bits that contain the parity calculation for that location for ALL the drives.  If a drive needs to be rebuilt or replaced, parity can rebuilt the drive by using the parity calculation and reading ALL the other drives (they must all be readable) to figure out what is missing.

 

Now, if you need to rebuild a drive, yes it will come back exactly as it was.  This will include all folders, files permissions, etc.  It will also include any errors or corruption introduced to parity before rebuilding; thus the need for a monthly parity check to ensure parity integrity.

 

And, yes SMART checks can be run on demand or scheduled via a script.

I'm well aware how parity works. My point was that snapraid does it from the data within files, on file level. I was hoping Unraid did it on block level direct under the file system. That way all permissions and ownership will be included as well. From the way it has been explained, it sounds like it dots all my i's and crosses all my t's.

 

3 minutes ago, jonathanm said:

A parity check reads all sectors on all drives in the parity array.

 

You can initiate and check results of SMART tests in the GUI, and if you wanted, you could script them.

Sweet, the ability to set up cron scripts or equivalent will suite me fine.

[Hoopster]

3 minutes ago, Corvinus said:

I'm well aware how parity works.

I was not trying to imply otherwise.  Sometimes I reply thinking of others who might be reading the thread in the future because of a search.

 

FYI - here's the GUI SMART section:

 

[Corvinus]

Thank you, if I can schedule the extended self-test via scripting or otherwise, I'll be good. I like to run them every 30 days.

 

A followup if I may. I read in the Unraid 6 Manual: 
"You can subscribe to different types of notifications for each method and even add custom alerts for SMART values attribute monitoring."

 

Will this act similar to the N second checking mdmonitor can do in md setups? So that when a sector read or write operation is attempted, and there's a pending and/or reallocation event happening, I will be notified almost instantly on e-mail without having to run a test manually or wait for my script to run. 

[Hoopster]

11 minutes ago, Corvinus said:

Will this act similar to the N second checking mdmonitor can do in md setups? So that when a sector read or write operation is attempted, and there's a pending and/or reallocation event happening, I will be notified almost instantly on e-mail without having to run a test manually or wait for my script to run.

Yes, because of some cabling issues, I was getting a lot of UDMA CRC errors on a particular disk.  The error notifications were frequent and almost instantaneous.  I ended up attaching that disk to a different HBA SATA cable/port and rebuilding.  It was the same disk and I just rebuilt it onto itself and all was good.  Because of all the errors, unRAID eventually marked the disk unreadable.

 

Notifications can be email but also pop up in the unRAID GUI.

 

I had another case where I turned off SMART monitoring of a particular attribute because I was getting notified of an "error" that minutes later corrected itself. It got so frequent it was annoying and was due to a firmware bug in that particular SSD model.

[Corvinus]

1 minute ago, Hoopster said:

Yes, because of some cabling issues, I was getting a lot of UDMA CRC errors on a particular disk.  The error notifications were frequent and almost instantaneous.  I ended up attaching that disk to a different HBA SATA cable/port and rebuilding.  It was the same disk and I just rebuilt it onto itself and all was good.  Because of all the errors, unRAID eventually marked the disk unreadable.

 

Notifications can be email but also pop up in the unRAID GUI.

 

I had another case where I turned off SMART monitoring of a particular attribute because I was getting notified of an "error" that minutes later corrected itself. It got so frequent it was annoying and was due to a firmware bug in that particular SSD model.

 

Being able to opt out certain attributes is surely a nice option to have. 

 

Think I'll pull the trigger on the pro the coming week and give it a solid try. Thanks a lot for polite and constructive answers.

[Hoopster]

1 minute ago, Corvinus said:

 

Being able to opt out certain attributes is surely a nice option to have. 

Here's what it looks like (I turned off 197 because it was a false pending sector count reading):

[Corvinus]

That's a pretty serious firmware bug... To report false pending.

[itimpi]

1 minute ago, Corvinus said:

That's a pretty serious firmware bug... To report false pending.

Crucial SSD’s are known for doing this - and apparently they have no intention of fixing this firmware issue.

[Corvinus]

I'll keep that in mind... My last usage of Crucial was an M.2 that constantly reached 80+ celcius. Asking their support was not constructive at all. Been sticking with Samsung EVO's and WD and on rare occations Kingston SSD's since. 

[Hoopster]

1 minute ago, Corvinus said:

I'll keep that in mind...

 

It was a Crucial MX500 SSD.  Other models appear to be fine, I think. 

 

It was largely a Linux problem and was fine in Windows (until it wasn't).  When it started popping up in Windows, suddenly it was "expected bahavior" according to Crucial and they saw no need to fix it.