About to take the plunge...new Unraid build

[Gabbiee]

Hi all,

 

After reading as many posts and Unraid configurations as possible (thanks Raj), I think I am ready to order the hardware and treat myself to an early holiday gift  .  While I wait for the hardware to arrive, I have a few questions about setting up Unraid and what the best next-steps are.

 

A little background of what I currently have.  I am running one 2-bay Dlink 323 NAS and a 6-bay QNap 619 NAS.  I also have 4 external WD 2 TB USB 3.0 drives.  Entire total is roughly 17.5 TBs of data.  My goal is to copy over all this data to the new Unraid server and keep these drives as a physical back-up.  I won't delete this as I prefer to have the safety of physical back-ups with the parity/data protection of the new Unraid server.  

 

With the new Unraid server, I will have 3 HTPC clients and AirVideo for iPad.  I may look into setting up an FTP client as well.

 

Unraid Hardware Setup:

 

Norco RPC-4220

Supermicro MBD-X8SIL-F-O

Intel Core i3-540

Kingston 4GB (2 x 2GB) 240-Pin DDR3 SDRAM DDR3 1333 ECC Unbuffered

20x Hitachi Deskstar 7K2000 2 TB (only sale at Microcenter for $119)

Corsair CMPSU-750TX

2x Supermicro AOC-SASLP-MV8

4x Norco C-SFF8087-D SFF-8087 to SFF-8087 Internal Multilane SAS Cable

1x Norco C-SFF8087-4S Discrete to SFF-8087 (Reverse breakout) Cable

 

Questions

1) What is the best way to copy over 17 TBs of data?  Before or after setting up a parity drive?  

2) If I don't setup a parity drive at first, can I set one up after the data is copied over?  

3) Will I still have access to my media while a parity drive is being setup?

4) Can I write to the Unraid server while a parity drive is being built?

5) Should I setup a cache drive before copying over the data?

6) Do I have to pre-clear the Hitachi drives?

7) How long will it take to build a new Unraid server with 18 data drives, 1 parity and 1 cache?

Are these calculations correct?  17.5 TB = 18350080 MBs.  At an average copy  rate of 40 MBs, it should about 5 days to transfer everything over.  

 

Please let me know if you have any feedback or advice on how to get my current data over to the new Unraid build faster.

 

Thanks!

Gabbiee

[dgaschk]

1) What is the best way to copy over 17 TBs of data?  Before or after setting up a parity drive?  

2) If I don't setup a parity drive at first, can I set one up after the data is copied over?  

3) Will I still have access to my media while a parity drive is being setup?

4) Can I write to the Unraid server while a parity drive is being built?

5) Should I setup a cache drive before copying over the data?

6) Do I have to pre-clear the Hitachi drives?

7) How long will it take to build a new Unraid server with 18 data drives, 1 parity and 1 cache?

Are these calculations correct?  17.5 TB = 18350080 MBs.  At an average copy  rate of 40 MBs, it should about 5 days to transfer everything over.  

 

1) After

2) Yes

3) Yes

5) No

6) You should

7) depends on your speed with a screwdiver  

Yes 5-10

[theone]

1) What is the best way to copy over 17 TBs of data?  Before or after setting up a parity drive?  

2) If I don't setup a parity drive at first, can I set one up after the data is copied over?  

3) Will I still have access to my media while a parity drive is being setup?

4) Can I write to the Unraid server while a parity drive is being built?

5) Should I setup a cache drive before copying over the data?

6) Do I have to pre-clear the Hitachi drives?

7) How long will it take to build a new Unraid server with 18 data drives, 1 parity and 1 cache?

Are these calculations correct?  17.5 TB = 18350080 MBs.  At an average copy  rate of 40 MBs, it should about 5 days to transfer everything over.  

 

1) After

2) Yes

3) Yes

5) No

6) You should

7) depends on your speed with a screwdiver  

Yes 5-10

 

I would change the answer to (1) to Before ONLY if you have your old data as a backup (you don't reuse the old drives).

Copying without parity will be 1.5-2 times faster - depending on your network - you should use GIGABIT and then you may be able to reach 60-70MB/sec.

 

[wsume99]

I agree with theone.  I would copy all the data over from your existing server without a parity drive installed.  But you need to be aware that WRT #3 you need to make sure that you preclear the parity drive before adding it to the array.  If you don't then your server will be offline while unRAID clears it.  If you preclear it first you just stop the array, assign the drive as parity, and then start the array.  If you follow this method your server will only be down a few minutes and all your data will be accessible while parity is being calculated.

[ohlwiler]

unRAID only performs the clear operation on the data drives if you are adding them to an array that currently has a parity drive assigned. The parity drive is never cleared. It is good practice to do a preclear on all drives before adding them to the array, even parity. During parity calculation your server will still be useable, although will not be quite as responsive due to the parity calculation.

[Gabbiee]

Thanks, everyone. Just hit ordered my shopping cart! Can't wait.

 

So to be safe, I believe this is my "to do" list.

 

1) install hardware

2) setup Unraid via USB

3) pre-clear 20 hard drives

4) setup 18 data drives

5) copy (not move) existing data to 18 drives

6) after data is copied, setup a parity drive

7) after parity drive is created, create a cache drive

enjoy my new Unraid server

 

Please let me know if I am missing anything important.

 

Thanks!

[theone]

Thanks, everyone. Just hit ordered my shopping cart! Can't wait.

 

So to be safe, I believe this is my "to do" list.

 

1) install hardware

2) setup Unraid via USB

3) pre-clear 20 hard drives

4) setup 18 data drives

5) copy (not move) existing data to 18 drives

6) after data is copied, setup a parity drive

7) after parity drive is created, create a cache drive

enjoy my new Unraid server

 

Please let me know if I am missing anything important.

 

Thanks!

 

As suggested before - I would add pre-clearing parity drive just to make sure it is in good working condition.

you can do it while copying the data just to save the time.

 

[wsume99]

^^I think she covered that in step 3 - preclear 20 hard drives.  If I've read this correctly she's going to have 18 data drives, one parity, and one cache.

 

Also, FYI Gabbiee - preclearing a 2TB 7200 rpm drive takes approx 24-26 hrs.  If you install screen (package available in unMenu) you can preclear up to six drives at the same time.  I'd expect that running 6 at a time would increase the preclear time a little but not sure how much.  So it's going to take about 3-4 days to get your 18 data drives precleared.  You can preclear the other two (parity & cache) while you are copying over data if you want.

[theone]

^^I think she covered that in step 3 - preclear 20 hard drives.  If I've read this correctly she's going to have 18 data drives, one parity, and one cache.

 

Also, FYI Gabbiee - preclearing a 2TB 7200 rpm drive takes approx 24-26 hrs.  If you install screen (package available in unMenu) you can preclear up to six drives at the same time.  I'd expect that running 6 at a time would increase the preclear time a little but not sure how much.  So it's going to take about 3-4 days to get your 18 data drives precleared.  You can preclear the other two (parity & cache) while you are copying over data if you want.

 

Correct.

 

I think you can start copying to the first 6 precleared drives while preclearing the next 6 drives and so on...

That could save some time as well - some parallel "processing"...