Hi there, my eventual unraid setup (it's pre clearing as i type) will consist of 4 x 2tb drives with a 2tb parity and a 160gb 2.5 drive as cache/apps unit.

 

I have two preexisting nas units on my network that are pretty much up to capacity now, with music , tv shows and movies.

 

As it stands at the moment I intend to keep the existing movies where they are and only add newly acquired ones to the unraid box, I want to move my music to the unraid box as that gets added to more than any other media type.

 

And tv shows i'd like to consolidate all my shows onto the unraid box, which brings me to my question.

 

The tv shows are split between the two nas units (each show and season etc..... being completely on one nas) , however a lot of them are old tv shows (eg original star trek , fawlty towers and so on) and some are active shows, I plan on having a tvshows active share and another for inactive.

 

would it be better to split the shows between active and inactive on the nas units they are currently on, or can i just dump the whole lot on to the unraid and do it there ?

I'm confused what you mean, unraid runs on a slackware base and has full access to any normal file system commands (mv (move), cp (copy), rm (remove), etc..), so, you can customize the filesystem either way.

 

Personally, I'd do it before you move it to unraid purely for the reason that you can do it while it's preclearing, instead of waiting until it's finished, although, it really doesn't matter.

ok, i guess i was confusing disks and shares on the unraid, what i didn't want was to copy all the stuff to the unraid only to have to wait again for it all to be copied between drives on the array, got to start thinking in array terms rather than separate drives.

 

it's pre 8am here and i'm probably confusing myself let alone anyone else, lol.

Just think of your UnRAID server as ONE big disk.    Think about the directory structure you want to have on this -- e.g. "Music", "Old TV Shows",  "New TV Shows", etc.  and create a Share for each of those top-level folders.

 

Then just copy your data to the shares exactly like you would to any other folder ... i.e. just copy your StarTrek folder to \\Tower\Old TV Shows\,  your NCIS shows to \\Tower\New TV Shows\,  and all of your music to \\Tower\Music

 

You may, of course, want further subfolders within those -- you can organize them as much as you'd like.  UnRAID will automatically manage what actually gets written to the various component disks.    [You can control this to some extent with the "split level" settings -- but don't be overly concerned about that]

 

1) I would first read up on usershares and their "split levels". It will explain better how its split between drives (VERY IMPORTANT): http://lime-technology.com/wiki/index.php/Un-Official_UnRAID_Manual#Split_level

2) I would recommend you DONT add parity or cache drives until after youre done moving. If you add parity it will take for ever, if you add cache you will have to manually pay attention to stop moving to run the mover (moves file from cache to drives, again very slow if needs parity check).

3) The default setting on how it fills drives initially is "high water" this means it will fill up your disk1 first until it reaches half, then moves on the the next (read more here). If you don't care about this its ok, but personally I didn't like the fact that I ended up with 80% of my files on one drive before it started filling the others, and since I went by type of file first, I ended up with movies on one drive and TV on another. Not a big deal, as I get more files it will spread out more but I generally like this a bit more even to start.

 

1) I would first read up on usershares and their "split levels". It will explain better how its split between drives (VERY IMPORTANT): http://lime-technology.com/wiki/index.php/Un-Official_UnRAID_Manual#Split_level

 

 

Very interested in this split level thing,  will read that and make sure i've digested it before doing my data dump.

 

spent all afternoon going through my tv show folders making sure all of them were split in to season folders with just the .nfo's and .jpg's in each show's root folder.

 

 

2) I would recommend you DONT add parity or cache drives until after youre done moving.

This is good advice in a very limited set of circumstances. If you are comfortable with the possibility of data loss, or are willing to take the extra time to verify the data moved properly with checksum comparisons after you are done moving it and parity is first built, then checked good, then it's fine. I would rather the transfer take a little longer by having the array in the fully configured state, that way you can be more confident that everything is going to work as you expect.

 

It all comes down to risk vs. reward, how much time do you want to shave at the expense of a little more security. Some people have a higher risk tolerance than others, especially when it comes to their data. All hard drives will fail, it's just a matter of when.

Absolutely agree with jonathanm => let the transfers go a bit slower and have the peace-of-mind that all of your data is fault-tolerant the instant it's on the UnRAID server      [That's also the reason I don't use a cache drive]

 

So you would preclear both a data and a parity drive. When cleared, you would transfer the data from an "old" drive to the cleared drive.

Then calculate parity, check if parity is faultless and repeat the process?!

So you would preclear both a data and a parity drive. When cleared, you would transfer the data from an "old" drive to the cleared drive.

Then calculate parity, check if parity is faultless and repeat the process?!

Preclearing parity is not strictly necessary but is highly recommended so it gets tested before use.

 

If you have a parity drive installed then any writes to the array automatically calculate and write parity. That's why writes to a parity-protected array are slower than writes to a disk without parity involved and so that is the reasoning behind the other recommendation to wait to install parity.

 

Personally I would just go ahead and install parity. Preclears are already going to take a long time. Copying the data without parity is going to be faster and your (unprotected) data will be available sooner, but then after you install parity it will have to calculate and write parity and that will take a while and your data won't be protected until it is done.

 

Checking parity after the transfer would just give you a little more confidence that everything is working OK. You should also periodically check parity (once a month is common) to help catch any issues that may be developing. If everything is working correctly you will not have any parity errors.

Perfect! Since I am in absolutely no hurry, and would rather have my data protected, I will install parity right away.

Thanks for the clarification

2) I would recommend you DONT add parity or cache drives until after youre done moving.

This is good advice in a very limited set of circumstances. If you are comfortable with the possibility of data loss, or are willing to take the extra time to verify the data moved properly with checksum comparisons after you are done moving it and parity is first built, then checked good, then it's fine. I would rather the transfer take a little longer by having the array in the fully configured state, that way you can be more confident that everything is going to work as you expect.

 

It all comes down to risk vs. reward, how much time do you want to shave at the expense of a little more security. Some people have a higher risk tolerance than others, especially when it comes to their data. All hard drives will fail, it's just a matter of when.

Indeed, I offered that option because it was media and because he was already copying it over from the original NAS. So the protection would be offered by the original. If his previous nas doesn't have protection, then there is still a risk that both source and unraid would fail, but that is unlikely.

Just for reference, I used Teracopy to both transfer and then validate the file. Once I had files moved, verified by teracopy and parity built I went ahead and wiped my original NAS.

Thank you for pointing out the downside of my suggestion though, I was rushing and forgot to include it

Just for reference, I used Teracopy to both transfer and then validate the file. Once I had files moved, verified by teracopy and parity built I went ahead and wiped my original NAS.

Perfect.  If you had outlined that for the OP in your first post, I wouldn't have jumped in. 

 

After parity is built, you still need a non-correcting check run to be REALLY safe though. When building parity, unraid reads from all data drives, and writes to the parity drive, and trusts that everything wrote correctly. On a check, all drives are read and the math is verified. If the check is non-correcting, any errors are noted and left alone. On a correcting check, the parity drive is changed to match the current data drives. If one of your data drives is failing, or has corruption, it's better to note the errors, run smart reports on all drives to figure out if it's better to trust the data drives and update the parity, or pull a failing data drive and rebuild from parity.

Just for reference, I used Teracopy to both transfer and then validate the file. Once I had files moved, verified by teracopy and parity built I went ahead and wiped my original NAS.

Perfect.  If you had outlined that for the OP in your first post, I wouldn't have jumped in. 

 

After parity is built, you still need a non-correcting check run to be REALLY safe though. When building parity, unraid reads from all data drives, and writes to the parity drive, and trusts that everything wrote correctly. On a check, all drives are read and the math is verified. If the check is non-correcting, any errors are noted and left alone. On a correcting check, the parity drive is changed to match the current data drives. If one of your data drives is failing, or has corruption, it's better to note the errors, run smart reports on all drives to figure out if it's better to trust the data drives and update the parity, or pull a failing data drive and rebuild from parity.

Do you know if teracopy can be set up to do a check after the fact like this? I haven't looked through it, honestly for me it was just media so i was kinda like meh. But it would be interesting to know if teracopy can be used just to validate without having to set up a transfer first.

Do you know if teracopy can be set up to do a check after the fact like this? I haven't looked through it, honestly for me it was just media so i was kinda like meh. But it would be interesting to know if teracopy can be used just to validate without having to set up a transfer first.

I think that's what the "test" button is for.

Teracopy isn't designed for file comparisons -- although it can just test two identical folders against each other if they are identical => just do a "copy", but when it prompts whether or not you want to replace the first file, click on the "All" button to the right of "Skip".  It will then skip the actual copies;  but will still do a verify of the CRCs.

 

A much better utility for this is FolderMatch -- not free, but very nice.

http://www.foldermatch.com/fminfo.htm

 

There are also several free comparison utilities - just Google for "file compare utilities"