Jump to content
resakse

Pooling

34 posts in this topic Last Reply

Recommended Posts

Sure.  Just assign all the drives as data drives; and create a single share (e.g. "MyBigDrive") without setting any split level, includes, or excludes.    Then \\Tower\MyBigDrive  will refer to this combined set of drives.

 

However, that fails to take advantage of the fault-tolerance of UnRAID.  Doing the same thing with a parity drive assigned will let the array be fault-tolerant.

 

Share this post


Link to post

man, that was quick. I asked the same question on flexraid forum yesterday, got no reply.. pm'ed their owner about the same question on his new t-raid, he didnt tell me the answer but told me not to pm him about technical issue. I just want to know about this specific feature before I'm buying a product.

 

thank you unraid for your great community and support!

Share this post


Link to post

Sure.  Just assign all the drives as data drives; and create a single share (e.g. "MyBigDrive") without setting any split level, includes, or excludes.    Then \\Tower\MyBigDrive  will refer to this combined set of drives.

 

However, that fails to take advantage of the fault-tolerance of UnRAID.  Doing the same thing with a parity drive assigned will let the array be fault-tolerant.

 

Yes, but atm I got 13TB on NTFS.. but in order to convert it to reiserFS, I need to buy at least 2 drives..but I can only buy 1 this month (I'm going to buy a new 4TB drive, but all my other drives are 2TB and 1TB..if I understand correctly, I need to use the largest size drive for parity.)

 

which means that I'm need to convert 4TB drives to reiserfs 1st, then copy the other data from another drive to it and convert that drive to reiserFS..one by one. and after I got another 4TB drives next month, I can use it as parity. Is that right?

 

 

Share this post


Link to post

Yes, the parity drive must be as large as any other drive in the array.    So if you're going to start using 4TB data drives, parity will also need to be 4TB.  You can, of course, add a parity drive whenever you're ready to do so -- the array simply won't be fault-tolerant until you do.

 

Share this post


Link to post

Also no need yet to spend money on this product, there is a free version you can use to try it out.

 

And arguably right now you can get away with buying a single drive and then start shuffling data.  I'd recommend buying a single 4TB right now to start.  There are some nuances to making this as easy as possible but the basics are:

 

1) Build your single drive unraid box

2) Create your desired user share structure

3) Copy one NTFS-disk worth of data to it the array

4) Move that now-duplicated NTFS-disk into the unraid box and assign it to the array

5) Go play a game, with your kids, or go to sleep while unraid clears and formats the drive

6) Copy one NTFS-disk worth of data to the array until it is about 90% full

7) repeat 5 and 6 until all NTFS disks are moved into unraid

8.) hopefully you have enough room on all those disks to move the data from the 4TB drive back onto the old-NTFS disks, if you don't then like garycase described, you'll just won't be fault-tolerant until you can get another 4TB

9) assign the 4TB as parity

10) Profit

 

Alternative to avoid even buying a drive at this time:

- Move all data off one 2TB into all the other drives, then see all the steps above with the assumption that the 2TB will be your parity drive at the very end.  If you can do this AND buy a 4TB then you can even use the steps above but skip adding the 4TB into the array until you are all done copying and step 8 become, "assign parity" without worrying about moving data off the 4TB.

 

This may seem long and involved, but the longest steps, the data copying and drive clearing, all has to happen anyway.  The physical drive shuffling is literally a drop in the bucket.

Share this post


Link to post

All this drive shuffling ASSumes you have backups of your important data elsewhere. If not, ignore everything about re purposing your current drives in the unraid array, and leave them alone as your backup. If you accidentally erase a disk that has your only copy of data, you are going to be very unhappy.

 

As jumperalex said, you could use the free version of unraid with 4TB disks and get 8TB of unprotected storage right now, add a 4TB parity drive in a couple months to gain protection against one of your 4TB disks failing. I would not move your current drives to a new array until you have your important files stored in 2 locations. Drives fail at unpredictable times, and human error moving stuff from box to box compounds the risk.

Share this post


Link to post

^^^ This ^^^ is 100% correct advice.

 

I only posted what I did based on OP's comment,

which means that I'm need to convert 4TB drives to reiserfs 1st, then copy the other data from another drive to it and convert that drive to reiserFS..one by one. and after I got another 4TB drives next month, I can use it as parity. Is that right?

 

So it sounds like his intent is to still only have one copy (except for the small window after copy-paste and before conversion to RFS).  I was just letting him know that he didn't have to wait to make the transition or wait to get parity so long as he planned it correctly.  At least at that point he will be in the same position of having no backup, but he'll at least be fault tolerant.

Share this post


Link to post

When you copy the data over the network, you may want to use a product like teracopy which can be configured to copy the data, and reading it back to insure the data was written correctly.

Share this post


Link to post

hi guys, thanks for the advice. Actually I already bought unraid plus license 3 years ago, but at that time I remember I need to have a lamp server and working linux environment, so I changed to flexraid. (yes, I should have thought virtualising at that time..my bad). So I convert all my reiserfs to NTFS using the jumperalex's method. Then my flexraid box got problem with motherboard, so I took all the hard drive and stuff it all up into my gaming win7 machine.. still using flexraid until now, I learned about esxi and thinking to virtualize unraid box.

 

But, my unraid license in unraid plus..which means that I need to upgrade it to Pro version (I got 8 HDD now). So.. if unraid can do pool without doing raid for temporary (which as NAS said, I can!) until I got my 2nd 4TB, I'll upgrade the license. If not, then I just need to find other alternative.

 

 

I actually dont have any important data store on my box, except that its just pain in the ass to re-download 13TB worth of movies and tv series :P

Share this post


Link to post

Well parity or not, new HD or not, you're going to need to come up with at least one completely empty HD that you can put into the unRaid array so that you can start copying data (Teracopy FTW), move the next empty drive into the array, rinse repeat. 

 

If you can do that, then as garycase said in the very first response, you'd just create a single share and put everything under that.  You might want to reconsider that exact method (not a poke at you gary)  and instead create shares based on the data (Photos, Movies, TV Shows, Desktop Backups, VM ISOs, etc) but that is up to you.  In either case, you will have accomplished your goal of having an expandable "pool" of disks appear to be a single share or series of shares spread among those disks.

Share this post


Link to post

yes sir..I completely understand that. Last time I did this 3 years ago (debian from raid5 to unraid), I used rsync.

 

But now I'm thinking just to let unraid format my 4TB drives, mount all other NTFS drives under unraid and simply local rsync it. I can do that, right?

Share this post


Link to post

Yes you can.

I would create an md5sum file for each drive first.

Then after the rsync, use md5sum to verify the data was copied without incident.

 

After that I might do a full preclear cycle to be sure the drive is a worthy candidate for putting data back onto it.

 

While the drive may have been in service for a long time, unless you are verifying each sector and reviewing the smart data, the drive may not be 100%. The other choice is to trigger a smart LONG test, let it run to completion and verify the logs.

If you have any pending sectors STOP and do a preclear.

 

Any drive that has pending sectors has the potential for read problems later on.

Share this post


Link to post
mount all other NTFS drives under unraid
Just be ABSOLUTELY sure you do NOT assign an NTFS drive with data to a drive slot in unraid. It will immediately and without warning overwrite the partition table and cause you to have to do a data recovery to get your NTFS files back.

Share this post


Link to post

mount all other NTFS drives under unraid
Just be ABSOLUTELY sure you do NOT assign an NTFS drive with data to a drive slot in unraid. It will immediately and without warning overwrite the partition table and cause you to have to do a data recovery to get your NTFS files back.

Not quite true. You have to set a checkbox indicating that you want to format the drive.

Share this post


Link to post

mount all other NTFS drives under unraid
Just be ABSOLUTELY sure you do NOT assign an NTFS drive with data to a drive slot in unraid. It will immediately and without warning overwrite the partition table and cause you to have to do a data recovery to get your NTFS files back.

Not quite true. You have to set a checkbox indicating that you want to format the drive.

It won't give you the opportunity to format a drive until it clears it.

Share this post


Link to post

Also keep in mind Preclear takes many many hours and you want to perform before you start up your array simply because your array will be in so many words useless until its cleared. So preclear it first to save you a lot of time.

Share this post


Link to post

Nothing really to add to the OP issue

 

But preclear reminds me of SpinRite, anyone use that product, good or bad?

 

I don't use it.

Share this post


Link to post

Nothing really to add to the OP issue

 

But preclear reminds me of SpinRite, anyone use that product, good or bad?

 

I don't use it.

I don't really see the similarity between SpinRite and preclear. Maybe it will do some read/write tests like preclear does but it will not prepare a disk for adding to an unRAID array and it isn't free.

Share this post


Link to post

I was also wondering what relationship you thought Spinrite had with this.  Spinrite is a very good tool to thoroughly scrub a drive and confirm that it's good (or not) ... in fact, the detailed testing it does at the higher levels (Level 3 or above) is a good bit more than the pre-clear script does;  but it does NOT "clear" the drive in any way.

 

You could use it to test a drive ... but you'd still need to either pre-clear it or let UnRAID do the clearing before it could be added to a protected array (no clearing is needed if the array doesn't have a parity drive).

 

Share this post


Link to post

I used to be a big spinrite fan. It has its place, but is very slow.

 

Here is a situation where it proved invaluable ...

 

http://lime-technology.com/forum/index.php?topic=1483.msg9973#msg9973

 

Yes, when it's needed it's VERY useful ... but it's certainly SLOW -- especially on Level 3 or above (a Level 5 pass on a modern 3-4TB drive is a many-day affair !!).    I don't use it much anymore, but every once-in-a-while it comes in VERY handy.

 

I noticed in your thread from a few years ago you noted:

(I've used it since v5.0).

 

... I've used in since v1 -- back in the 80's  :)

 

Share this post


Link to post

I used to be a big spinrite fan. It has its place, but is very slow.

 

Here is a situation where it proved invaluable ...

 

http://lime-technology.com/forum/index.php?topic=1483.msg9973#msg9973

 

Yes, when it's needed it's VERY useful ... but it's certainly SLOW -- especially on Level 3 or above (a Level 5 pass on a modern 3-4TB drive is a many-day affair !!).    I don't use it much anymore, but every once-in-a-while it comes in VERY handy.

 

I noticed in your thread from a few years ago you noted:

(I've used it since v5.0).

 

... I've used in since v1 -- back in the 80's  :)

 

I started having problems with Spinrite hanging on the larger drives.  Very annoying after waiting days for it to do its thing. Is Spinrite still being maintained?

 

It was much better in the old days when drives were much smaller.

 

You and I are similar vintage. I didn't use Spinrite in those old days, but remember adjusting the interleave to 1:1 on my old MFM drives.  You ever do that? Those were the good old days!

Share this post


Link to post

Yes, I've done that a a lot more.    I've been "messing" with computers since I was a young (15) college freshman; and have had them at home since the Altair.  My first hard drive was a 26MB 14" platter Seagate -- cost me $4500 at a time when I definitely could NOT really afford it ... but I did it anyway  :)  => today I could easily afford that, but drives cost a tiny fraction of that !!  8)

 

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now