Flash drives fail occasionally and it occurs to me that I should probably setup some sort of backup plan to allow for this. I bought my flash pre-configured from LimeTech and I'm sure there is some sort of procedure I could follow to get another one. But it would be faster and cheaper to just have a backup that I could install on a flash drive myself. Is there an addon I can run on my server that would create a bootable backup to a remote location? Or maybe an app I can run on my Mac? I know with Mac OS's I can use Carbon Copy Cloner. Not sure with unRaid?

the unRAID license is tied to a particular flash drive (by the GUID of the drive) so you cannot simply restore a backup of that drive to another physical USB drive. 

 

If you want to have a backup drive ready to go then you might want to take advantage of the current promotion that allows you to buy a second unRAID key for $30.

Recovering from a failed flash is very easy. A backup could be useful if you are running a bunch of plugins, but can be very dangerous to use, even if you properly update the key file for the new flash. The flash is updated on every start and stop of the array and especially importAnt is updated when your disk config changes. If your backup was taken at a time when one of your data disks was the parity drive you will be screwed!

 

It is so unlikely to have one fail. and rebuilding it is pretty easy, I suggest not to back it up unless you really know what you're doing.

the unRAID license is tied to a particular flash drive (by the GUID of the drive) so you cannot simply restore a backup of that drive to another physical USB drive. 

 

If you want to have a backup drive ready to go then you might want to take advantage of the current promotion that allows you to buy a second unRAID key for $30.

 

So if my flash drive fails my only option is to buy a new license? Or am I buying a new flash drive?

 

---

 

Recovering from a failed flash is very easy. A backup could be useful if you are running a bunch of plugins, but can be very dangerous to use, even if you properly update the key file for the new flash. The flash is updated on every start and stop of the array and especially importAnt is updated when your disk config changes. If your backup was taken at a time when one of your data disks was the parity drive you will be screwed!

 

It is so unlikely to have one fail. and rebuilding it is pretty easy, I suggest not to back it up unless you really know what you're doing.

 

Could you point me to something describing how to rebuild the flash? I tried Google but just got a few threads discussing starting over from scratch with a new flash drive. Didn't see anything about how to restore what was on the old one.

So if my flash drive fails my only option is to buy a new license? Or am I buying a new flash drive?

estore what was on the old one.

This might be true in theory, but not in practise.  If you contact LimeTech when a USB drive fails they are very good about re-issuing you license to be tied to the GUID of the replacement USB drive.  However this can take a day or so, so many like to have a spare USB drive ready to go and thus are prepared to pay for the spare key.

 

Some people like to use a USB SD card reader instead of a USB drive on the assumption the card is more likely to fail than the reader..  That way the license is tied to the SD Card reader rather than the SD card, so you can swap out the card if required.  The limitation of this approach is that you have to use a brand of SD Card reader that can provide a unique id per device - many do not.  The commonest ones used in this approach seem to be the Kingston G2 and G3 ranges.

So if my flash drive fails my only option is to buy a new license? Or am I buying a new flash drive?

estore what was on the old one.

This might be true in theory, but not in practise.  If you contact LimeTech when a USB drive fails they are very good about re-issuing you license to be tied to the GUID of the replacement USB drive.  However this can take a day or so, so many like to have a spare USB drive ready to go and thus are prepared to pay for the spare key.

 

Some people like to use a USB SD card reader instead of a USB drive on the assumption the card is more likely to fail than the reader..  That way the license is tied to the SD Card reader rather than the SD card, so you can swap out the card if required.  The limitation of this approach is that you have to use a brand of SD Card reader that can provide a unique id per device - many do not.  The commonest ones used in this approach seem to be the Kingston G2 and G3 ranges.

 

Thanks. I can understand how the spare key could be useful, but the $30.00 is just to purchase a 2nd license right? I would also need to buy my own flash drive and find a pc somewhere that I could use to install the OS onto it, since I can't do this on a Mac.

Thanks. I can understand how the spare key could be useful, but the $30.00 is just to purchase a 2nd license right? I would also need to buy my own flash drive and find a pc somewhere that I could use to install the OS onto it, since I can't do this on a Mac.

You can create a flash drive from your Mac.  I helped develop the tool that is included in unRAID 5 that does it.

You can create a flash drive from your Mac.  I helped develop the tool that is included in unRAID 5 that does it.

That's good to know. I guess I didn't really need the pre-configured one.

Not that the price wasn't reasonable.

 

Still curious about the procedure for restoring a failed flash drive. Is all the data stored to parity when I make changes to the flash? Maybe if I install a new flash all I need to do is run a parity check to restore all the addons? These are just wild guesses.

Still curious about the procedure for restoring a failed flash drive. Is all the data stored to parity when I make changes to the flash? Maybe if I install a new flash all I need to do is run a parity check to restore all the addons? These are just wild guesses.

These are indeed wild guesses and completely wrong. Parity only stores the parity data which is calculated based on the contents of the array data drives and constantly updated when any array data drive is written. The parity disk plus all of the other disks in the array are necessary to rebuild an array disk.

 

It would be a good idea to study the wiki to make sure you understand exactly how unRAID protects you. Misconceptions about how unRAID works will likely bite you in the event you need to replace a disk. If you are ever unsure, ask first.

 

To your question of how to restore a failed flash drive. It is possible to get things going again from scratch but it would require a little work. One thing you can do is just make a complete copy of the contents of the flash drive any time you or unRAID writes anything to the flash drive. Any change you make in the webGUI, such as configuring a user share, will cause unRAID to write to flash. Any time you add, remove, or replace a disk, unRAID will write to flash. Once you get things set up you will probably not need to make a backup very often.

 

As mentioned, it also writes to the flash each time you start or stop the array. In the event of a power off while the array is not stopped, unRAID will automatically start a parity sync the next time it is started to make sure the power off didn't allow parity to get out of sync. A side effect of this is, if you copy the flash drive when the array is started, and then restore this copy to a new flash drive and boot from it, unRAID will think the array was running when it was last powered off and will start a parity sync. This is not usually a problem since it never hurts to do a parity sync assuming the array is healthy.

 

The most important thing is never restore a copy of the flash that was made before adding, removing, or replacing a disk because things will really get confused. Recently a user restored a backup copy of his flash, but he had replaced his parity disk and reused the old parity disk as a data disk since he had created that backup copy of the flash. The result was unRAID did a parity sync to his old parity disk, which was unfortunately now one of his data disks.

 

TL;DR

 

Always ask if you are unsure.

 

*edit to fix quote*

It is so unlikely to have one fail. and rebuilding it is pretty easy, I suggest not to back it up unless you really know what you're doing.

 

I'm completely confused now. If flash drives never fail and are easily rebuilt perhaps it would be better to wait for the situation to occur. And then someone could explain the easy rebuild procedure. I just didn't want to be in a position where I would have to re-install and re-configure all the addons from scratch. Had hoped a little pre planning could avoid that.

It is a good idea of the files on the Flash.  This can be a simple file level copy of the contents.    you might want to repeat the backup when you reconfigure something on the unRAiD system. 

 

Note that this needs to be somewhere other than the unRAiD server as the only time you are going to need to use the backup is when you cannot access the unRAiD system for some reason.

So if my flash drive fails my only option is to buy a new license? Or am I buying a new flash drive?

estore what was on the old one.

This might be true in theory, but not in practise.  If you contact LimeTech when a USB drive fails they are very good about re-issuing you license to be tied to the GUID of the replacement USB drive.  However this can take a day or so, so many like to have a spare USB drive ready to go and thus are prepared to pay for the spare key.

 

Some people like to use a USB SD card reader instead of a USB drive on the assumption the card is more likely to fail than the reader..  That way the license is tied to the SD Card reader rather than the SD card, so you can swap out the card if required.  The limitation of this approach is that you have to use a brand of SD Card reader that can provide a unique id per device - many do not.  The commonest ones used in this approach seem to be the Kingston G2 and G3 ranges.

 

Thanks. I can understand how the spare key could be useful, but the $30.00 is just to purchase a 2nd license right? I would also need to buy my own flash drive and find a pc somewhere that I could use to install the OS onto it, since I can't do this on a Mac.

 

The spare key can be useful for other reasons. You can boot a workstation to do a preclear, for example. And if  you ever want to setup a backup array, this key can be used. Tom is quick to turn around key replacements MOST of the time, but occasionally he is on vacation or otherwise not responsive, you would have a risk of not being able to get your array updated for some period of time. (Update: Replacement keys are now available online, so the need for a second key is reduced).

 

In order to build a new USB for unRAID ...

 

1 - Get a new USB thumbdrive. Size is not that important. 2G is more than enough. But even that small is hard to find. Just get one from a known company that gets good reviews on Amazon or Newegg. I just set up a 16G flash and it works fine with my system.

 

2 - Prepare your USB. Every version of unRAID contains instructions to do that. Read the announcement thread of the version you plan to install. Iconicbadger has a video on setting up version 6 and demonstrates creating the flash on a Mac - there is no problem preparing it on a Mac. (See FIRST VIDEO IN THIS POST).

 

3 - Boot with the new flash. unRAID will boot but none of your disks are configured. Its ok. This is the free (3 drive) version anyway. Capture the GUID for the flash. That's it. Don't configure anything. (In addition to getting the GUID, this step also verifies unRAID can boot from this flash drive).

 

4 - Send Tom an email with your new GUID explaining the prior drive failed. He will send you a new key file. Copy it to the config directory on your thumbdrive and reboot your array. (Tom maintains a list of dead GUIDs. He may add your old one on that list and the next version of unRAID would not accept your old key. So using Tom's generous replacement policy is not a good way to get a new key for free.).

 

5 - Boot with the new keyfile in place. (copy the .key file from Tom to the config directory on the flash drive)

 

6 - It is helpful to have a recent screenshot of the unRAID GUI, showing which slot each physical disk is assigned to. If you are going to backup something, backup that screenshot. Use it to assign each disk to its slot. If you don't know, assign EVERY disk to a disk slot LEAVING PARITY EMPTY. Start the array. One of the disks will show unformatted. That is the parity disk. You also need to identify the cache drive, and any non-array disks you have installed in the server.

 

7. Once you have figured out which is the parity disk (usually the only unformatted disk), which are the data disks in the array, which is the cache disk, and what are the other random disks in the machine (usually none). Do a new config (see step 8 for more info) and then assign the disks to the appropriate slots. If you don't know what data disk goes in what slot it doesn't matter from a parity perspective. When you reestablish your user shares you'll be able to figure out what disks below in what shares.

 

8. There is a checkbox somewhere (I have not done this with 5.0) to tell unRAID that you are reassembling the array that will cause it to not rebuild parity. Maybe someone can post a screenshot of this as I've never done this with version 5.0.5 and I have not seen that option. But you would want to select that option. It may be on the new config screen (step 4).

 

9. Double and triple check the settings. You don't want to screw this up!

 

10. Start the array.

 

11. Do a non-correcting parity check. If you messed up the configuration (you didn't because you double and triple checked ) you will start getting parity errors at an alarming rate. You are running non-correcting so nothing is getting updated. Simply stop the array, do another new config, and go back to step 6. If you get a few parity errors, that is ok. It could be that your array had crashed with the failed USB stick, so it is not completely unexpected that you'd get some parity sync errors. I'd like the parity check run for 2-3 minutes and if the number of parity issues < 50, I'd stop the parity check and run a correcting check to fix the parity issues. Let it run to completion. If you got zero parity errors, you should still run a full parity check. If parity errors are more than 50, ask a question on the forums.

 

12. Do your unRAID configurations and reinstall you addons, plugins, packages, etc. Screenshots of these config panels would be useful!

 

Thanks. I think I finally understand. It's clear now that "changes to configuration" refers to configuration of the server not the flash drive. I'm sure I had the whole thing backwards. Sometimes I get a misconception in my head and it requires a brain surgeon to correct it.

 

---

 

12. Do your unRAID configurations and reinstall you addons, plugins, packages, etc. Screenshots of these config panels would be useful!

 

Thanks for the step by step. I hope I never need it but you never know. If I keep a file copy of the flash as itimpi stated, can I just copy them back to the new one to restore the addons?

Thanks. I think I finally understand. It's clear now that "changes to configuration" refers to configuration of the server not the flash drive. I'm sure I had the whole thing backwards. Sometimes I get a misconception in my head and it requires a brain surgeon to correct it.

 

---

 

12. Do your unRAID configurations and reinstall you addons, plugins, packages, etc. Screenshots of these config panels would be useful!

 

Thanks for the step by step. I hope I never need it but you never know. If I keep a file copy of the flash as itimpi stated, can I just copy them back to the new one to restore the addons?

Yes but you will need to replace the existing ".key" on the copy with the new one that Tom sends you when you ask for a replacement because the ".key" file is specific to a USB stick and cannot be used on a different one.

The spare key can be useful for other reasons. You can boot a workstation to do a preclear, for example. And if  you ever want to setup a backup array, this key can be used.

 

Does my array have to be offline to run preclear? I thought I could run it on unassigned drives while the array is online. Have to admit I haven't really checked into it very deeply. Right now I'm waiting for my first parity check to complete. Should be done sometime tomorrow. Then I'll start installing addons.

Yes but you will need to replace the existing ".key" on the copy with the new one that Tom sends you when you ask for a replacement because the ".key" file is specific to a USB stick and cannot be used on a different one.

 

Step 5

 

The spare key can be useful for other reasons. You can boot a workstation to do a preclear, for example. And if  you ever want to setup a backup array, this key can be used.

 

Does my array have to be offline to run preclear? I thought I could run it on unassigned drives while the array is online. Have to admit I haven't really checked into it very deeply. Right now I'm waiting for my first parity check to complete. Should be done sometime tomorrow. Then I'll start installing addons.

 

Yes - you can run preclear on your array machine. But sometimes when you have several to do at once you don't have enough ports on your unRAID box and would prefer to preclear on a different machine. Now that I think of it you can do this with the free version, so not a good reason to need a license. The key reasons are if Tom is delayed responded with a new key file; and ability to set up a new (e.g., backup) array. There have been discussions of what happens if Tom disappears. He said that he has taken steps that unRAID would become public domain and the keys would not be an issue. But some of us like to have a spare thumbdrive and key in the back of the tool drawer just in case.

Does my array have to be offline to run preclear? I thought I could run it on unassigned drives while the array is online. Have to admit I haven't really checked into it very deeply. Right now I'm waiting for my first parity check to complete. Should be done sometime tomorrow. Then I'll start installing addons.

No - in fact it is a good idea if it IS online as then a check is carried out to make sure you are not trying to pre-clear a disk that has already been assigned to the array.  Of course this does require you to have enough SATA ports to support both the array and the disk(s) you want to pre-clear.

The spare key can be useful for other reasons. You can boot a workstation to do a preclear, for example. And if  you ever want to setup a backup array, this key can be used.

 

Does my array have to be offline to run preclear? I thought I could run it on unassigned drives while the array is online. Have to admit I haven't really checked into it very deeply. Right now I'm waiting for my first parity check to complete. Should be done sometime tomorrow. Then I'll start installing addons.

I always do preclears with the array online. Just to make sure you know what preclear means. Preclear is an addon script that tests and clears a drive (sets it to all zeros). A drive added to the array must be all zeros so it is in sync with existing parity. If a drive is added without being precleared, not only is it not tested, but unRAID will have to take the array offline while it clears it. Once a drive has been cleared (or precleared) unRAID will format it. Formatting is fairly quick, but clearing must write zeros to the whole disk and will take a while.

 

Preclearing allows the array to stay online when you add a disk. Not preclearing will make unRAID take the array offline while it clears the added disk.

Does my array have to be offline to run preclear? I thought I could run it on unassigned drives while the array is online. Have to admit I haven't really checked into it very deeply. Right now I'm waiting for my first parity check to complete. Should be done sometime tomorrow. Then I'll start installing addons.

No - in fact it is a good idea if it IS online as then a check is carried out to make sure you are not trying to pre-clear a disk that has already been assigned to the array.  Of course this does require you to have enough SATA ports to support both the array and the disk(s) you want to pre-clear.

At the risk of taking this discussion totally off topic- Is it possible to connect external sata enclosures for just this purpose? SNAP maybe?

The spare key can be useful for other reasons. You can boot a workstation to do a preclear, for example. And if  you ever want to setup a backup array, this key can be used.

 

Does my array have to be offline to run preclear? I thought I could run it on unassigned drives while the array is online. Have to admit I haven't really checked into it very deeply. Right now I'm waiting for my first parity check to complete. Should be done sometime tomorrow. Then I'll start installing addons.

I always do preclears with the array online. Just to make sure you know what preclear means. Preclear is an addon script that tests and clears a drive (sets it to all zeros). A drive added to the array must be all zeros so it is in sync with existing parity. If a drive is added without being precleared, not only is it not tested, but unRAID will have to take the array offline while it clears it. Once a drive has been cleared (or precleared) unRAID will format it. Formatting is fairly quick, but clearing must write zeros to the whole disk and will take a while.

 

Preclearing allows the array to stay online when you add a disk. Not preclearing will make unRAID take the array offline while it clears the added disk.

Thanks. I already had read most of that. It does performance testing also correct?

Does my array have to be offline to run preclear? I thought I could run it on unassigned drives while the array is online. Have to admit I haven't really checked into it very deeply. Right now I'm waiting for my first parity check to complete. Should be done sometime tomorrow. Then I'll start installing addons.

No - in fact it is a good idea if it IS online as then a check is carried out to make sure you are not trying to pre-clear a disk that has already been assigned to the array.  Of course this does require you to have enough SATA ports to support both the array and the disk(s) you want to pre-clear.

At the risk of taking this discussion totally off topic- Is it possible to connect external sata enclosures for just this purpose? SNAP maybe?

You don't need SNAP for preclearing. In fact, it would get in the way.

 

I always do my preclears in an external SATA enclosure.

 

Preclear does not explicitly do performance testing but you might get some indication of performance problems based on how long it takes. My last preclear of a 3TB drive took about 35 hours if I recall, which is reasonable.

 

You don't need SNAP for preclearing. In fact, it would get in the way.

 

I always do my preclears in an external SATA enclosure.

 

Preclear does not explicitly do performance testing but you might get some indication of performance problems based on how long it takes. My last preclear of a 3TB drive took about 35 hours if I recall, which is reasonable.

 

So I can just hook up something like this to a USB port on my server and be able to run the preclear script on it?

You don't need SNAP for preclearing. In fact, it would get in the way.

 

I always do my preclears in an external SATA enclosure.

 

Preclear does not explicitly do performance testing but you might get some indication of performance problems based on how long it takes. My last preclear of a 3TB drive took about 35 hours if I recall, which is reasonable.

 

So I can just hook up something like this to a USB port on my server and be able to run the preclear script on it?

I don't know if that would work or not. Not clear if unRAID actually supports USB 3. I always use an eSATA enclosure. Of course you would need a spare SATA port for that.

That same dock is available with an esata input and I've got 2 spare sata connection on my SAS2LP-MV8 controller. Might need some sort of adapter? Not sure about how sata and esata configurations compare. But I would think it wouldn't be too hard to work that out if USB isn't supported.

SATA to eSATA cable or bracket for example