Hi,

 

I have been running unRAID for a while and now want to migrate to 6.2 with dual parity on a new machine. The "new machine" is basically the same, but with dual parity, larger disk drives, and hopefully xfs filesystems.

 

The question is can a temp or trial license run in the new machine while I preclear and copy data (~10TB)?  I am not worried about the 30 day as I expect to finish the copying within a week. I wondering about limits on number of drives and/or size.

 

After that I can move the USB from the old machine into the new machine.

 

If that is doable...

 

What needs to be done to the USB when it is moved into the new machine? It has 5.x unRAID now. I know the license is tied to the USB so that has to move, but I want the new machine all setup and running 6.2 and not go through any upgrade processes.

 

Thanks

 

PS: the license is Pro, but only expect to start with 5 disks in the new machine (2 parity+3 data, all 6TB).

Details on the trial licence are here.

 

Yeah, that does not cover the fact that trial has a limit on the size and number of devices. I know there are limits as I tried 6.x before and was limited of 250Gb or something. I am hoping by using 6TB drives I can get all 10TB into the 3 disk limit.

The device limits are the same as for the Pro licence. The only differences are that the Trial is time limited and needs an Internet connection to verify its validity on start-up.

 

Yeah, that does not cover the fact that trial has a limit on the size and number of devices. I know there are limits as I tried 6.x before and was limited of 250Gb or something. I am hoping by using 6TB drives I can get all 10TB into the 3 disk limit.

there are no limits on the Trial license other than the time limitation.  I suspect you are thinking of some of the things that were tried in the early 6.0 beta releases.

Or, since you are still running unRAID 5.x you may be thinking of the unRAID 5 trial limits - which have changed in unRAID 6.

 

Keeping in mind that the .key file is tied to the GUID of the old USB drive I think I would:

• Format a new USB with the unRAID 6 trial.
  
• Stand up the new server with the new stick and trial license.
  
• Copy the data from your unRAID 5 server to your new unRAID 6 server (assuming you are using all new disks).
  
• Backup both the old and new USB drives, especially the .key file on the old drive.
  
• Reformat the old USB stick and copy the contents of the new one to it.
  
• Run makebootable on the old stick .
  
• Copy the .key file to the old USB drive.
  
• Boot the server with the old USB drive (which now contains an unRAID 6 installation).
  

Or, since you are still running unRAID 5.x you may be thinking of the unRAID 5 trial limits - which have changed in unRAID 6.

 

Keeping in mind that the .key file is tied to the GUID of the old USB drive I think I would:

• Format a new USB with the unRAID 6 trial.
  
• Stand up the new server with the new stick and trial license.
  
• Copy the data from your unRAID 5 server to your new unRAID 6 server (assuming you are using all new disks).
  
• Backup both the old and new USB drives, especially the .key file on the old drive.
  
• Reformat the old USB stick and copy the contents of the new one to it.
  
• Run makebootable on the old stick .
  
• Copy the .key file to the old USB drive.
  
• Boot the server with the old USB drive (which now contains an unRAID 6 installation).
  

 

I am doing a modified version of this as I found the new download does not include the preclear_disk.sh (or screen). Once that is finished, I'll try the new download again.

I am doing a modified version of this as I found the new download does not include the preclear_disk.sh (or screen). Once that is finished, I'll try the new download again.

preclear and screen are not directly provided by limetech, you have to set them up after you get the basics running. The nerdpack plugin has screen, preclear is a whole nuther matter, there are several versions of console scripts and a gui plugin available.

I am doing a modified version of this as I found the new download does not include the preclear_disk.sh (or screen). Once that is finished, I'll try the new download again.

preclear and screen are not directly provided by limetech, you have to set them up after you get the basics running. The nerdpack plugin has screen, preclear is a whole nuther matter, there are several versions of console scripts and a gui plugin available.

 

Ouch, sounds like a mess, a prerequisite is not included... Is there licensing issues around the script or screen? Or is preclear no longer needed? It is a major pain, perhaps the biggest for consumer acceptance.

 

I am 12 hours into the preclear, pre-read almost done.

Or is preclear no longer needed? It is a major pain, perhaps the biggest for consumer acceptance.

 

Pre-Clear has never been "needed" or required.  In the early versions of UnRaid, if you added a new disk to the parity-protected array, the array would not be available until that new disk had been cleared (all zeroes written to it) so parity would be maintained.

 

To eliminate that long period of the array not being available, Joel wrote a "preclear" script; which would clear the disk BEFORE it was added to the array ... and with a special "pre-clear signature" that Tom provided and supported, so UnRaid would recognize that the disk had already been cleared.  This allowed a new disk to be added instantly -- no array downtime.

 

The pre-clear script also included a pre-read and a post-read phase to provide a level of testing for the new disk ... and this testing became a nearly defacto method folks would use to confirm a new disk was good before adding it to an array -- even if the clearing function wasn't necessary (e.g. a parity disk or cache disk).

 

Starting with v6, UnRaid no longer takes the array offline to clear a new disk -- it simply clears it and THEN adds it to the operational array.  So the CLEARING function is no longer needed.  But you SHOULD test any new disk to confirm it's okay -- either using pre-clear or a good disk diagnostic utility  (I use WD's Data Lifeguard).   

 

I am doing a modified version of this as I found the new download does not include the preclear_disk.sh (or screen). Once that is finished, I'll try the new download again.

preclear and screen are not directly provided by limetech, you have to set them up after you get the basics running. The nerdpack plugin has screen, preclear is a whole nuther matter, there are several versions of console scripts and a gui plugin available.

 

Ouch, sounds like a mess, a prerequisite is not included... Is there licensing issues around the script or screen? Or is preclear no longer needed? It is a major pain, perhaps the biggest for consumer acceptance.

 

I am 12 hours into the preclear, pre-read almost done.

Preclear is not technically a prerequisite.

 

The only time unRAID requires a clear disk is when adding it to a new slot in an array that already has valid parity. A clear disk is all zeros, so adding a clear disk doesn't make the existing parity invalid.

 

If you add a disk that is not clear to a new slot in an array that already has valid parity, unRAID will clear it for you. Previous versions of unRAID took the array offline for clearing, but recent versions do the clearing while keeping the array accessible.

 

Preclear was created to prevent the array downtime caused by clearing, but since this downtime no longer happens, it is not strictly necessary.

 

Nevertheless, many people use preclear to test a disk, whether they actually need a clear disk or not. Probably if it were not for disk testing, preclear wouldn't be done anymore.

 

Testing a new disk is important though. Most electronics will either fail early, or will not fail for a long time, possibly becoming obsolete before actual failure.

 

It is very important that every bit of every disk is good, because every bit of every disk is required to rebuild a disk if one of them has problems. Parity by itself doesn't allow you to rebuild, parity just provides data that allows a disk's data to be calculated from all of the other drives data.

 

There are other ways you could test a disk. The disk manufacturers provide free diagnostics for this.

As noted above, you can do exactly what you want with no issue ... just register a trial license and then build your new system.

 

One question, however.  You said ... "The "new machine" is basically the same, but with dual parity, larger disk drives, and hopefully xfs filesystems." ==>  Does this mean you plan to use the same hardware?  I. e. both systems can't run at the same time?  (e.g. to copy the data from the old to the new)

 

If that's the case, I'd do this differently ... but first confirm whether or not that's your plan.

 

 

As noted above, you can do exactly what you want with no issue ... just register a trial license and then build your new system.

 

One question, however.  You said ... "The "new machine" is basically the same, but with dual parity, larger disk drives, and hopefully xfs filesystems." ==>  Does this mean you plan to use the same hardware?  I. e. both systems can't run at the same time?  (e.g. to copy the data from the old to the new)

 

If that's the case, I'd do this differently ... but first confirm whether or not that's your plan.

 

There are two machines, same motherboard, same disk controller, same CPU, same memory. The difference is the new machine has fewer, larger disks. Both are running. One is running unRAID 5.x and the other is running unRAID 6.x doing preclears.

 

But there is no reason to use two, if one will do.

 

The preclear has reached the zeroing phase. Maybe 10 more hours of that.

As noted above, you can do exactly what you want with no issue ... just register a trial license and then build your new system.

 

One question, however.  You said ... "The "new machine" is basically the same, but with dual parity, larger disk drives, and hopefully xfs filesystems." ==>  Does this mean you plan to use the same hardware?  I. e. both systems can't run at the same time?  (e.g. to copy the data from the old to the new)

 

If that's the case, I'd do this differently ... but first confirm whether or not that's your plan.

 

There are two machines, same motherboard, same disk controller, same CPU, same memory. The difference is the new machine has fewer, larger disks. Both are running. One is running unRAID 5.x and the other is running unRAID 6.x doing preclears.

 

But there is no reason to use two, if one will do.

 

The preclear has reached the zeroing phase. Maybe 10 more hours of that.

Two systems will do better than one I think. You are going to have to copy disks instead of rebuilding them since you want to change filesystem. Might as well copy them over the network, and you will have a complete backup system when you're done.

Indeed the nice thing about having two complete systems is you'll now have (at least initially) a complete backup system.  As I presume you know, UnRAId does NOT substitute for having a good set of backups -- so if you don't already, you will after you build the new system 

Indeed the nice thing about having two complete systems is you'll now have (at least initially) a complete backup system.  As I presume you know, UnRAId does NOT substitute for having a good set of backups -- so if you don't already, you will after you build the new system 

 

Well, I do not plan on having two complete systems. As the topic mentions, the idea is to avoid purchasing (and running) a second license.

 

The post-read has begun, but it does not look like I will be able to begin the copying today.

Understand ... and you certainly don't need it if you already have a good backup strategy.  I was just noting that with a complete 2nd server that already has all your data, that's certainly a good point to implement a very easy backup strategy if you needed one.    A complete fault-tolerant backup system for the price of a 2nd UnRaid license certainly seems like a bargain.

 

So, I began the process of copying files from old unRAID to newer unRAID. I used Windows and Explorer 'drag and drop". After completing a few shares, I wanted to double check things.

 

I found and tried WinMerge. Works really well, until I tried it on a rather deep and large music folder. The compare took nearly 2 days, this folder has hundreds of thousands of files, so taking time is expected. The drag and drop process had issued errors/warnings than files would be too long. WinMerge found those and more. Digging into those it seems some are erroneous. WinMerge found both copies, but failed to match them up. Maybe there were just too many files in the share?

 

So, I am not looking for suggestions for both the copy process and a verification process.

I haven't tried it yet, but I know Teracopy is well regarded here.

I would look into a tool to compute MD5 (or similar) checksums. Doing those on the machine hosting the files is about as quick as you can get, and then you can match them up (Excel works well) to make sure the checksums match.

 

When copying a bunch of files on the same machine, doing from the unRAID box vs using your workstation as a go between is highly recommended. It avoids the latency of the files going back and forth over your network. I like teracopy for small stuff, but when doing things in the 100G+ category, I'll usually opt for the less convenience but faster Linux approach.

 

There is a fancy rsync command if you look around to copy and compare in one step! Never used it but supposed to be quite efficient.

With millions of files, I don't think Excel is going to be up to the task.

 

I have used teracopy in the past, but it seemed to have the same problem with filenames being too long.

 

rsync will probably get the best of me, but I do some research on that.

I believe the command to copy and then verify is ...

 

rsync -avPX {source} {target}