I am not sure how to go with this huge project. Let me tell you on what i am trying to achieve here

 

I am trying to build a file server atleast that might serve for years run to extend the storage as and when i wanted. The hardware except the storage , i wanna build right now.

 

my thoughts.

SuperMicro 24Drive bays server.

Asus P5E-V Full ATX MB

Q6600 Processor

4GB Ram

Vista 32 or 64 Bit OS

 

And would like to have unRaid

The main purpose to use as HTPC for my home theater. Right now. I have about 120 HD Movies and 500 DVDs sitting on External 2TB Drives on each, which i usually hook up with USB Connection to my computer. Latest build was HTPC with 6TB using onboard Sata Controllers. Using drives as it is with no Raid or unRaid. As i have a request from my friend to let him buy my 6TB HTPC and i am trying to build another with massive storage in mind as my 6TB ran so far out of storage.

 

I am not sure whether to build to separate server or keep everything in one case. Budget is always constraint. So less money to spend , more happier to buy other stuff.

 

Also i am looking for 24 SATA Controller for unRaid, all i could find is 8 PCIx-64 Interface's which P5E-V HDMI MB doesn't have. Any PCI interface's contains more than 8 sata internal connections...

 

Or any other solutions you guru's can think of

 

Any advices are appreciated.

Keep in mind, the current unRAID drive limit is 15 data drives, 1 parity disk + 1 cache disk.

We have not heard yet of expansion past these limits.

Ok, here's two solutions.

 

The simple one is to simply populate unRAID with as many drives as you have to have right now.  You ran out of space with 6TB, then start it with 7 or 8TB.  When you need that 9th terabyte, maybe they'll be out with the 1.5TB drives.  Replace your parity drive with the 1.5TB, and use the old parity as a data drive.  When you run out of space, get another 1.5TB drive.  When 2TB drives are released, then replace your now-1.5TB drive with the 2TB drive, and use the old 1.5TB parity drive for data.  Rinse and repeat.  By the time you hit the current 15 data-drive limit, there'll be drives much larger than 1TB, so you will be closer to getting that 24TB with less than 24 drives (or rather, 25 since one is used for parity).  Perhaps unRAID will handle more than 15 data drives by then.  If not, then start replacing some of your ancient 1TB drives with 3TB drives (or whatever's out by then).  This will work fine as long as you don't need all 24TB available TODAY.

 

Second solution, assuming you need all 24TB TODAY.  You'll need to get several external enclosures that will run in RAID-0.  You will have them populated with two 1TB drives, giving you a total of 2TB per enclosure.  The cool thing is they report to the OS as one large drive, so as far as unRAID is concerned, those two 1TB drives are really just one 2TB drive (note that the enclosure must natively support RAID-0, not just JBOD).  You can buy 13 of these, 12 will be for data and the last for parity.  You're just under the 15 drive limit.

 

I don't recommend the second solution.  Each enclosure acts as one drive, but is twice as likely to fail (just one of the two drives has to fail).  It's also expensive, you'll have to deal with plugging in 13 different enclosures plus finding a place to install them all, etc.  It should work, but I personally wouldn't do it.

 

Now as far as running Vista, that will need to be done on a separate PC.  unRAID is its own OS.  Yes, it is possible to run windows as a virtual machine, but it's not ideal for an htpc (it won't have direct access to your video card or sound card, will run slower, etc.).

Second solution, assuming you need all 24TB TODAY.  You'll need to get several external enclosures that will run in RAID-0.

This IS feasible and I have tested it. The downside here is the drives probably will not spin down. They didn't in my enclosure.

Besides, I might be more inclined to go with as large a case as you can (if you know you want to support 24TB in the future).

and take a wait and see approach. As Josetann said, drives are always getting larger.

There still is the possibility of expanding unRAID to larger number of drives. (ask Tom).

There's also another shortcut of using a real RAID controller on some of the drives to create the RAID-0 Internally.

Then doing a kernel mod or driver install (ask tom) and using these drives as part of the array.

 

Frankly with the power requirements, if I needed that much storage, I might choose to built two servers when I was nearly out of room on one.

This way all your eggs are not in one basket.

It may be possible to build a 24TB machine today, but with only 15 of the data drives being parity protected.  I'm sure one of our local Linux experts can devise a method to export the data on the drives that are not a part of the unRAID array.  As an example, you could build a 15TB unRAID array, an 8TB Areca RAID 5, and a 2TB RAID 1 for performance.

 

Tom may raise the limit in the future, but there are some already who may say that 15 drives is too many for one parity drive, living a little too dangerously.

 

A better solution for now would be to build 2 unRAID arrays, 12TB each, with 2 parity drives protecting you.

 

Edit:  If you are looking to fill a 24 bay case exactly, then my math is a little screwy.  Should be 15TB unRAID array and 7TB Areca RAID 5, or 2 11TB unRAID arrays.

 

Another thought, it'd be nice if user shares could work across multiple unRAID servers.  I'm sure this would be no easy feat for Tom, but maybe he's already looking into it?  It might even be possible to run two unRAID servers in one case.  I've seen cases that support two motherboards, but they're not that easy to find nor very cheap.  Just not practical for most people (but if you're a datacenter, you might take a second look at it).

Another thought, it'd be nice if user shares could work across multiple unRAID servers.  I'm sure this would be no easy feat for Tom, but maybe he's already looking into it?

He actually did mention something like that once... no idea if he ever did anything towards it as a feature. 

 

As everybody has said, disks keep getting bigger, and cost per gig getting less and less. Get as big a case as you can afford, with room for the disks you anticipate, with multiple power supplies, populate the slots you need for today's media collection, and grow as needed. unRAID makes that part pretty easy.  unRaid will only handle 16 drives... unless that changes in the future, you will probably end up with two smaller arrays Or much larger drives than the 1T drives available today.

> it'd be nice if user shares could work across multiple unRAID servers.

 

You mean clustering. That really would be cool.

 

The issue is that the shfs environment cannot be exported.  At least in it's current incarnation.

I tried exporting it via NFS and it wouldn't work.

 

Although I was able to get all the disks in /mnt to export by creating a tmpfs on mnt before emhttp was run.

I cannot seem to get /mnt/user to export

 

root@Media:/mnt/disk1# exportfs -varf 
exporting *.cotrone.com:/mnt/disk1
exporting *.cotrone.com:/mnt/disk2
exporting *.cotrone.com:/mnt/user
exportfs: Warning: /mnt/user does not support NFS export.
exporting *.cotrone.com:/mnt

 

In any case, clustering would be a viable solution if the two shfs daemons could communicate.

Or another clustering filesystem was used.

 

unRaid will only handle 16 drives...

There's a hard limit defined in the md.c driver.

/* The maximum number of disks per array that we support.

*/

#define MD_SB_DISKS                    16

 

Perhaps we could coax Tom to up this to 24.

I think the normal linux driver has a max of 32 so I would take it that there are at least 32 device nodes set aside.

I'm not a kernel hacker so I really don't know the details.

I did some research recently and learned that 4T drives should be available in 2009.  Current unRAID will get you to 15T.  When 4T drives come out you'll be able to get to 60T.  If you could live within 15T for a year or two, unRAID should be able to grow with you up to very high capacities.

 

If you can't wait, build two servers.  Maybe a bit of a hassle to use, but better (IMO) than buying and managing 15 RAID-0 enclosures.  Ouch!

Or offer your first born to Tom to up the max disks to 24 ;-)

I'm not sure why anyone wants to build a high spindle-count server.  I agree with bjp999, why not just build multiple smaller servers?  The lost space due to the multiple parity drives pays back in a more reliable system as the odds of having a second failure while recovering from the first is obviously higher the more drives you have.

 

Plus, multiple smaller parts tend to be cheaper than super-sized components.  PSUs and cases, for example.

 

Yes, you will need multiple copies of Unraid and yes, you will need multiple mobo/memory/cpu, etc., but I believe when you add up all the costs, it will be pretty much a break even.

 

 

Bill

why not just build multiple smaller servers?

 

Why not just up the drive count to 24?. It's not that impractical if you have the need or money.

It's up to the person building the server to assess the risk and cost.

With port multipliers it's quite possible to have a huge storage server with external drive arrays.

 

What's probably needed is the ability to divide up the drives internally into multiple array protected parity disks.

In the meantime, I say up the drive count see the feasibility.

Sorry for the late reply and thanks for the great suggestions here.

 

As i have in my mind, this is going to be longterm to fill all my 24TB at once. I am trying to add the drives as and when needed. The Enclosure sits for longterm, so that i don't need to buy small enclosures everytime and build so many computers around. As i use this one mainly serving my home theater and i would like to have everything in one enclosure rather than spreading around. Initially i would like to start with 6TB and move forward.

 

I have P5E-V HDMI Board coming on my way from Specialtytech. And would like to buy Supermicro Chasis 24 Bays http://www.supermicro.com/products/chassis/4U/846/SC846TQ-R900.cfm and adding 8Port SATA http://www.supermicro.com/products/accessories/addon/AoC-SAT2-MV8.cfm from Supermicro aswell to fit into my PCI slots. I have question on this one whether it is perfect fit for pci slots with the given 64-bit PCI-X133MHz.

 

At present the onboard 6SATA ports would be sufficient, as i grow with the content i would like to add drives and add-on card .

 

And i am trying to use Vista Ultimate, does this support these many drives for future purpose. Please share your valuable inputs. Appreciated

Sorry for the late reply and thanks for the great suggestions here.

 

As i have in my mind, this is going to be longterm to fill all my 24TB at once. I am trying to add the drives as and when needed. The Enclosure sits for longterm, so that i don't need to buy small enclosures everytime and build so many computers around. As i use this one mainly serving my home theater and i would like to have everything in one enclosure rather than spreading around. Initially i would like to start with 6TB and move forward.

 

6T is way less than 24T and easily fits in unRAID's comfort zone.  I believe that as hard drives grow in size you'll be able to expand your array and unRAID will be able to meet your storage needs in a single "box".

 

I have P5E-V HDMI Board coming on my way from Specialtytech. And would like to buy Supermicro Chasis 24 Bays http://www.supermicro.com/products/chassis/4U/846/SC846TQ-R900.cfm and adding 8Port SATA http://www.supermicro.com/products/accessories/addon/AoC-SAT2-MV8.cfm from Supermicro aswell to fit into my PCI slots. I have question on this one whether it is perfect fit for pci slots with the given 64-bit PCI-X133MHz.

 

PCI-X is backward compatible with PCI.  It just runs at 1/2 the interface speed.  This is not a problem unless you start running lots of drives at the same time (like in rebuilding parity).  Using PCI-E cards might be a better choice if you really are planning to max out your server.

 

At present the onboard 6SATA ports would be sufficient, as i grow with the content i would like to add drives and add-on card .

 

6 SATA ports is okay, but if I were buying a MB just for unRAID, I'd get one with 8 (or 9).

 

And i am trying to use Vista Ultimate, does this support these many drives for future purpose. Please share your valuable inputs. Appreciated

 

Confusing me here.  unRAID runs under its own OS (a Linux variety).  Your Vista machine would just connect to the drives for read and write.  Issues of how many drives are supported is an unRAID question and has nothing to do with your desktop OS.  unRAID currently supports 17 drives max (1 parity, 15 data, and one cache).

PCI-X is backward compatible with PCI.  It just runs at 1/2 the interface speed.  This is not a problem unless you start running lots of drives at the same time (like in rebuilding parity).  Using PCI-E cards might be a better choice if you really are planning to max out your server.

 

Could you please suggest any other PCI-E with 8 SATA Ports or more

 

 

6 SATA ports is okay, but if I were buying a MB just for unRAID, I'd get one with 8 (or 9).

Confusing me here.  unRAID runs under its own OS (a Linux variety).  Your Vista machine would just connect to the drives for read and write.  Issues of how many drives are supported is an unRAID question and has nothing to do with your desktop OS.  unRAID currently supports 17 drives max (1 parity, 15 data, and one cache).

So far i thought i can fit everything into one Computer, i guess i have to read all about unRaid then as sits on own OS, i need to consider building another box for my HD/BLURAY Playback etc...Thanks for the reminder

 

 

 

 

 

Same question asked here.

 

http://lime-technology.com/forum/index.php?topic=2038.0

Sorry for the late reply and thanks for the great suggestions here.

 

As i have in my mind, this is going to be longterm to fill all my 24TB at once. I am trying to add the drives as and when needed. The Enclosure sits for longterm, so that i don't need to buy small enclosures everytime and build so many computers around. As i use this one mainly serving my home theater and i would like to have everything in one enclosure rather than spreading around. Initially i would like to start with 6TB and move forward.

 

I have P5E-V HDMI Board coming on my way from Specialtytech. And would like to buy Supermicro Chasis 24 Bays http://www.supermicro.com/products/chassis/4U/846/SC846TQ-R900.cfm and adding 8Port SATA http://www.supermicro.com/products/accessories/addon/AoC-SAT2-MV8.cfm from Supermicro aswell to fit into my PCI slots. I have question on this one whether it is perfect fit for pci slots with the given 64-bit PCI-X133MHz.

 

At present the onboard 6SATA ports would be sufficient, as i grow with the content i would like to add drives and add-on card .

 

And i am trying to use Vista Ultimate, does this support these many drives for future purpose. Please share your valuable inputs. Appreciated

 

I agree with BJP. You're not going to need 24TB right now like you said. Hard drive sizes will increase in the future. By the time you max out 15TB with current 1TB drives there will probably be 2TB drives on the market (maybe smaller or even larger... who knows). I would build my system around the current 15TB size knowing that when hard drives increase in size in the future my total space on my unRAID server will grow as well.

 

I also agree that you should get a Motherboard with 8 SATA ports. In my setup I have a board with 8 SATA ports and 2 PCI cards with 4 each. This allows me to run 16 hard drives.

I might go with a large case as you have selected.

Get a good motherboard with lots of SATA ports. (Gigabyte or ABIT AB9 PRO 9 sata + 1 esata).

Then upgrade to a controller when the need arises.

By the time you need all 24 TB, unRAID may have expanded enough to support them.

Or, drives may have gotten larger so you don't need them.

Or you could go with a high end raid card, Stripe "some of the drives" and add them in to the array to fake unRAID into believing you have a 2TB drive. LOL.

 

But ask your self, do you really need 24TB and/or 24 spindles?

I know at one of our data centers we are using 16 spindles for backups.

 

As far as using a high end card, Perhaps you could convince Tom to compile in the drivers.

If not, one of the advanced people on the board may assist with a customization so you could use a raid card.

 

One point to consider if you use an advanced card in raid configuration.

unRAID will be unable to spin the drive down. (it might work in JBOD, but I bet it won't in RAID mode).

 

Still.. do you really need 24TB of space now?

 

 

Also, Ask yourself, is parity rebuild speed of paramount importance?

 

I believe you might achieve drive expansion at a cost effective level by using port multipliers.

I believe there are 3 ports on the ABIT AB9 PRO that are port multiplier capable. (actually 4, but one is external).

Each port multipler card from addonics turns 1 port into 5.

so 6 ICH ports + 1 SIL (x5) + 2 Jmicrons ( +5 and +5 ) = 21. If you add another controller or port multiplier you will get up there to 24. (i bet you could rig the eSata port to go back inside or at the very least purchase a $40 controller + port multipler to give you more ports.

 

Keep in mind, you would be venturing in new territory. Not may people are using them here, however, it has made it into the kernel proper and the manufacturers have removed their driver links to the beta code moons ago.

 

Also there would be a slight performance penalty. However, if you arrange the drives in alternating use by PCIe lanes, you could possibly circumvent the performance issue.

 

I.E.

 

ICH is on the southbridge.

SIL is on a PCIe x1 lane but you have drives 1-5 on a port multiplier.

JMICRON is on another PCIE x1 Lane, and you have another port multiplier.

 

What I might do is set the drives like

Drive 1 - ICH

Drive 2 - SIL (slot 1)

Drive 3 - JMICRON (Slot 1)

Drive 4 - ICH (Slot 2)

Drive 5 - SIL (slot 2)

Drive 6 - JMICRON (slot 2).

etc, etc.

 

I believe if you added the drives sequentally, then the port multiplier may be a bottle neck.

Then again, I bet it's less then a bottleneck against a PCI card with 8 ports.

I know it's bordering on off-topic, but my recurring thought while reading this thread was 'look for more efficient ways to store your data'. BluRay's 50GB capacity is great, but 12gb x264 1080p compressed files look stunning, I'd go as far as to say near-transparent (on my lower end 720p projector) and... yeah. Just as MPEG2 on DVD is wildly inefficient (a standard for set-top boxes to be able to decode 10 years ago) BluRay encodes (or MPEG2 HD broadcasts - even worse!) can also be squeezed with little to no ill-effect (might have to give up Menus). You'll be playing them with a HTPC which has a great deal more processing power than a stand-alone HD player.

 

Just a thought.