Hello

I am building a new media server,and have been looking at how to run it. I would like to know a few things about unraid before i decide what to use.

 

If i have all my media already stored on 10x2TB wd discs,will i be able to add them to a new build as is (NTFS format) ,and retain all the data if using unraid,or do i have to back up all the data to a different drive,and then add the files when unraid has prepaired the discs? Several of the drives are 4k advanced format EARS and EARX drives

 

Can i set unraid to boot up and shutdown at specific times?

 

Can unraid show my drives individually? I do not like my array to be 1 giant drive

 

I have gone overkill on the hardware,and thus have not bought items on your "compatability" list, do you still have a lot of hardware limitations to the unraid software? does unraid utilize 8GB of ram? since ram is so cheap,i bought 2x4GB kingston for this build.

 

I will also be using a Core I3 sandy bridge 2100T on an asrock mainboard,and 5 asrock satax2port cards just to add more sata ports (the cards are basic sata port extenders,no raid support)

 

unRAID looks very promising,as i have absolutly no linux experience,so it needs to be very easy to setup,it needs to be able to connect with my other 2 win7 pc´s as well as my popcorn hour a-210

 

Thank you

If i have all my media already stored on 10x2TB wd discs,will i be able to add them to a new build as is (NTFS format) ,and retain all the data if using unraid,or do i have to back up all the data to a different drive,and then add the files when unraid has prepaired the discs? Several of the drives are 4k advanced format EARS and EARX drives

unRAID will want to clear the drive and format it.  All data will be wiped if you assign the drive as a data drive within unRAID.  The 4K thing is fine, just set it properly from within unRAID's settings.

 

Can i set unraid to boot up and shutdown at specific times?

The boot up... I don't think so.  The shutdown can be done using the "at" command in the go script.

 

Can unraid show my drives individually? I do not like my array to be 1 giant drive

Yes, and technically speaking unRAID is a JBOD array with parity protection.  You can view each drive separately or use the User Shares feature to make it appear there is one big disk.

 

I have gone overkill on the hardware,and thus have not bought items on your "compatability" list, do you still have a lot of hardware limitations to the unraid software? does unraid utilize 8GB of ram? since ram is so cheap,i bought 2x4GB kingston for this build.

In general most hardware will work.  5.0bX has more hardware support and some newer boards may require 5.0bX

 

I will also be using a Core I3 sandy bridge 2100T on an asrock mainboard,and 5 asrock satax2port cards just to add more sata ports (the cards are basic sata port extenders,no raid support)

Would love to get the exact model of the board and links to the SATA 2 port cards

Can i set unraid to boot up and shutdown at specific times?

 

If you can use the system boards BIOS to power on at a certain time you could get round the boot up hurdle that way.

 

does unraid utilize 8GB of ram? since ram is so cheap,i bought 2x4GB kingston for this build.

 

It will use the additional memory as disk data cache where it can. This will be especially useful if you run the cache_dirs script (which you likely will). So whilst unraid won't utilise it directly it will certainly improve your experience and, as you say, RAM is so cheap it would be rude not to.

Thank you so much for the answers. I have a few more,if that is ok :-)..

 

Can unraid support more than 20 disks,if so how is an array of for example 30 discs handled with drive letter assignment?

Does unraid support samba so my popcornhour can read from it,and my windows 7 pc´s also?

 

This is the mainboard i will be using http://www.asrock.com/mb/overview.asp?model=h61del , reason is the 5 pciex slots,as i find the the sata cards with 4 or 8 ports to be extremely expensive compared to these,and i can always add a 4 or 8 port card later on should i require it.

 

The link for the sata cards are here http://www.asrock.com/mb/spec/card.asp?Model=SATA3%20Card

 

To power this with an initial 12 drives i will use a Corsair GS600 single rail PSU,i hope that will suffice?

 

Also, if i use the free unraid for 3 discs and everything works using this hardware,will i be able to just buy the reg key,and add more drives,or do i have to setup the whole thing again from scratch if i buy the pro version?

 

Thank you

20 drives maximum in the protected array. You counld add more, but they're not part of the unRAID drives. SAMBA is the default.

 

I'm not sure about your SATA card and if it's supported.

 

That PS should be fine.

 

You just copy the key file to the flash and the extra drive slots appear for use without messing up what you already have.

 

Peter

 

 

Can unraid support more than 20 disks,if so how is an array of for example 30 discs handled with drive letter assignment?

Not at this time

 

Does unraid support samba so my popcornhour can read from it,and my windows 7 pc´s also?

Yes, along with NFS and AFP (in the betas)

 

This is the mainboard i will be using http://www.asrock.com/mb/overview.asp?model=h61del , reason is the 5 pciex slots,as i find the the sata cards with 4 or 8 ports to be extremely expensive compared to these,and i can always add a 4 or 8 port card later on should i require it.

Not a bad looking board but you are going to have to run the latest beta to support the Realtek 8111E NIC that is on the board.  The only thing I have to say about using so many of the 2 port cards is that some boards can be fickle about that and it does leave room for more parts to fail and cables to deal with.  A single Supermicro SASLP card is a better choice in my opinion.

 

The link for the sata cards are here http://www.asrock.com/mb/spec/card.asp?Model=SATA3%20Card

Not sure about the card, you are going to want to do a search on the forums for the chipset mentioned on the page to see if anyone is using a similar card.

 

To power this with an initial 12 drives i will use a Corsair GS600 single rail PSU,i hope that will suffice?

Yes, that PSU should be fine for 12 drives.

 

Also, if i use the free unraid for 3 discs and everything works using this hardware,will i be able to just buy the reg key,and add more drives,or do i have to setup the whole thing again from scratch if i buy the pro version?

Yes, just register the GUID of the flash drive, get the .key file, place it on the flash drive, and reboot.

Hmm, well the supermicro uses a pciex4 slot,and the mb i will be using have none of those, only 5 pciex1 and 1 pciex16.

 

Also,since the asrock sata port cards,are just that port cards,and not have any raid function or whatever,they theoretically should work just as the onboard sata connectors.

 

Also, couldnt i just try out the array using the free 3 disc version,and if everything works with the hardware i bought, upgrade unraid and add more drives after?

 

Should i have bought the Corsair GS700 instead to begin with? I will be adding drives on a regular basis,as i have slots for 16 drives,and capability of 20 drives. Will the GS600 be able to power the array if fully loaded?

 

*UPDATE* the asrock sata cards are reported working,so all should be good there http://lime-technology.com/forum/index.php?topic=7442.msg72138#msg72138

 

Thank you

the x16 slot will work with the SASLP.

 

The 700 would give you a little more headroom when you add more drives.

ok,thank you. If using the supermicro,is it normal the card won´t fill out the x16 pcie slot on the mb completely like gfx cards does? Also,if using the supermicro,i would still have to run a few of the asrock cards to reach 20 sata ports when fully loaded,would that be a problem? using the 5 cards and the onboard 4 slots i will have 14 sata ports to start with,which will probably be enough for the foreseeable future,using only 1 supermicro will only give the the 12 slots i start with,but then i have no room to expand should the need arise..

 

Anyhow,since the asrock cards are reported working,and since they are dirt cheap,i will be using them for now,i can always exchange stuff later on.

 

If not,i would have to change my order for hardware completely,and buy a mb with at least 2 pcie x16 slots to accomodate using the supermicro cards,and the order has been placed with my retailer,so i will make it work as is :-)

 

Thank you so much for your help,and i am certain more questions will arise once i begin build next week (parts arrive wednesday if all goes well)

 

Have a good weekend

ok,thank you. If using the supermicro,is it normal the card won´t fill out the x16 pcie slot on the mb completely like gfx cards does?

Correct, the card will not take up the entire slot and that is expected.

 

Also,if using the supermicro,i would still have to run a few of the asrock cards to reach 20 sata ports when fully loaded,would that be a problem?

It should not be a problem, though weirder things have been known to happen.

 

Anyhow,since the asrock cards are reported working,and since they are dirt cheap,i will be using them for now,i can always exchange stuff later on.

Sounds like a plan

 

 

Have a good weekend

You too

OK,another question has popped up.

 

I have taken apart ALL my external usb drives (all Western Digital),so i can utilize the SATA on the drives themselves for unRAID,however, of the 10 drives, 9 are of the intellipark load cycle problem variety primarily 7 drives are WD20EARS and 2 are WD10EADS, and i just ordered a WD20EARX and i have an WD20EACS also,will these also suffer the intellipark problem the rest of the green drives do?

 

Do i need to run widdle3 to disable the intellipark feature on these drives to prevent crive crash due to high load cycle count? If i need to use widdle, will it affect the data on the drives,or do i need to backup the drives,and then widdle them before using them in unRAID?

wdidle should not effect the data and it would be a good idea to run it before putting the drives into unRAID.

OK,i have now run widdle3 on all drives,and they were as suspected all set to 8secs or 12secs. I disabled the idle timer all together,as i have my os control the power save feature.

 

Does unRAID also control power down on the drives,or does it read the firmware timer i just disabled?

OK,i have now run widdle3 on all drives,and they were as suspected all set to 8secs or 12secs. I disabled the idle timer all together,as i have my os control the power save feature.

 

Does unRAID also control power down on the drives,or does it read the firmware timer i just disabled?

You can set spindown on the drive yourself and unRAID will then spin it down after said amount of time being inactive.

Excellent,thank you..

 

What is minimum number of drives unraid works with? I only have 1 new drive,all the others are at capacity,and i suspect unraid would need 2 to begin with,parity and data drive,correct?

 

Also,when adding new drives and unraid needs to pre-clear them,does it do all new drives at once,or 1 at a time,so it will take weeks to prepare my 12 drives?

 

Some of my drives has seen a VERY high load cycle count 50000+ will these drives potentially die using the preclear thingy?

 

And why preclear at all,if i can recover from 1 drive failure with the parity drive? The parity drive needs to be the largest drive,or equal to largest size in array,correct?

 

And why preclear at all,if i can recover from 1 drive failure with the parity drive? The parity drive needs to be the largest drive,or equal to largest size in array,correct?

 

{Easy answer first:  Yes, the parity drive needs to be same size or larger than the largest data drive -- many use a 2TB or 3TB or whatever the "maximum" size drive you will be planning on using as parity.  The rest of the drives can be of any size.  That's why many migrated from WHS to UnRaid because of this mix/match ability, among other reasons...}

 

Interjecting here from experience.  I migrated to unRaid from WHS where I also had a bunch of drives, that I cracked out of "Mybook" enclosures, etc.

 

I can truly state that I wish I had done the PreClear process -- and as recommended, multiple passes of it -- on each and every drive before adding it.  It would have saved me many headaches moving forward.

 

It amazes me on how when you move from USB environments via the external enclosures, to raw SATA connections inside of unraid -- that what were "fine" drives and perhaps with a little glitch here and there,  become problem-children drives when accessed at a much higher "intensity" when they go native SATA and at what UnRaid {via Unix} vs. Windows can access them.... and raw SATA vs. USB->SATA did...

 

PreClear is really a "are my old, reused drives up to snuff to use here" -- and also a "is the brand new drive I bought, really a good drive to put into  my array"

 

Yes, while you can get protection via parity for a failed drive -- there just makes no logical sense to go into a large unraid configuration without knowing, to your personal satisfaction, that each and every drive you put into that array has been "passed" through several PreClear passes -- especially older drives.

 

I note several passes of PreClear -- for both old and new drives.  There has been several anecdotal evidences of drives that show their 'true colors' after 3 to 5 passes of PreClear.  It's worth it.  Take the time.  It's data you don't want to rebuild/reacquire {or can't} going on them...

 

You will specifically want to look at the "reallocated" sectors counts -- as they are now, and as they are after the preclear -- high numbers, and incrementing numbers are both a worry, and a decision made on whether to use that drive in UnRaid.

 

I speak from experience.  My first unraid server was a "test" one.  Using the freebie license, and random drives I hooked up to play with.  I loved it. Bought my Pro keys...  Put more storage in it, ran out of space, cracked another drive, and added it... etc, etc -- and before I knew it,  I had a "production" unraid filled with several drives I had never ran "Preclear" on.

 

Then when I had early problems,  I wanted to blame Unraid for the fault.  Turns out, no, I had bad drives -- old drives, which had been harboring failure warnings for a long time.  Things that Preclear and the SMARTS stats would have shown me clearly, before ever having added the drives.

 

And especially that valuable "parity" drive -- I made sure that was a brand new drive, that survived a hammering by PreClear, because it is the linchpin of my system.

 

 

 

...Chuck

 

This is the mainboard i will be using http://www.asrock.com/mb/overview.asp?model=h61del , reason is the 5 pciex slots,as i find the the sata cards with 4 or 8 ports to be extremely expensive compared to these,and i can always add a 4 or 8 port card later on should i require it.

 

The link for the sata cards are here http://www.asrock.com/mb/spec/card.asp?Model=SATA3%20Card

 

 

And another side note from experience.  I also had a bunch of two-port cards (SIL3132 based) -- not only did adding them incrementally actually cost more than just buying an 8-port card -- but SIL3132-based cards are now known to have problems on *some* systems with high-volume dual-channel access.

 

{Your proposed card is Marvel based...  but many use the 2-port SIL3132-based cards that say, Monoprice, carries for less than $20...  however, I and a few others got burned by those -- so be careful...} -- the Supermicro card is rock-solid and Unraid 101 standard here -- thousands of builds with it.

 

The supermicro 8-port card that many many many use is $109 (plus some break-out cables)

 

How many two-port cards do you need to quickly reach $109?

 

I also bought a mainboard, back then, with lots of x1 slots and only one x16 for video.  I wish I had done things differently... (Eg, several x4 or x16 slots, and onboard video) -- but I live with it, as new motherboards are cheap whenever I decide I really need to do something else.  For now, I bought a PCI video card, freed up the x16 for the SuperMicro and will be rebuilding things this weekend

What is minimum number of drives unraid works with? I only have 1 new drive,all the others are at capacity,and i suspect unraid would need 2 to begin with,parity and data drive,correct?

 

Minimum of 1 drive. The parity drive is optional. You can run unraid with 1 data drive though it will of course be unprotected due to the lack of the parity drive.

 

For a parity protected config then 2 drives minimum.

 

Also,when adding new drives and unraid needs to pre-clear them,does it do all new drives at once,or 1 at a time,so it will take weeks to prepare my 12 drives?

 

Sort of both. the preclear is a community script and runs via the command line. It will only clear one drive per execution but you could run 12 in parallel if you wanted to do all your drives at once.

 

You don't *have* to preclear them using the community script.

 

Unraid, in its stock config, will want to clear any drives itself before it uses them. Unfortunately this means having the array down whilst it does it - and as you've surmised this takes a while.

 

The community pre clear script lets you do this 'behind the scenes' so your actual array only has to be down for a couple of minutes whilst you assign the new drives. It also allows you to do burn in tests and general health checking of the drives before you put them into service. Both are useful though I personally only use it for adding drives with minimal array downtime.

 

Some of my drives has seen a VERY high load cycle count 50000+ will these drives potentially die using the preclear thingy?

 

Probably not. The preclear just writes a custom signature to the drive and then zeros it out. It can do this as many times as you want to test the drive. It's not doing anything particularly invasive to the drive but it will ensure the drive is completely written and read from. Which may expose weaknesses that you havn't yet found.

 

And why preclear at all,if i can recover from 1 drive failure with the parity drive? The parity drive needs to be the largest drive,or equal to largest size in array,correct?

 

Why preclear as above. It's inevitable that you will have to clear your drives. You can either :

 

- do it the stock unraid way. No health checks, long array downtime whilst it does it

 

- the community preclear script way. Do it out of band and health check your drives at the same time.

 

Yes parity needs to be largest or equal largest drive.

OK,thank you guys. I am however worried if warranty will apply if preclear "fails" a brand new drive,then what?

 

I will be using my brand new WD20EARX for the parity drive then (if preclear clears the drive that is),and see if i can free up 2TB of space to manually add 1 drive ad a time, man is this gonna take forever :-)..I am a very impatient person,so for me to wait weeks to see if this thing will work,and if all drives are up to snuff is gonne be extremely hard.

 

On a side note,just found out i am gonna get a tax refund,so looks like i will be buying 2 more drives,as well as the pro unraid lcense :-). I would like to still get WD,but with users having to manually turn off the 8sec park thingy,and me being unalbe to find out of 8sec park is present in the new drives,i am unsure what to get then. Seagate sucks, i also have very bad experiences with hitachi,so what is left?

OK,thank you guys. I am however worried if warranty will apply if preclear "fails" a brand new drive,then what?

 

 

Many have RMAd drives back to their purchase point if they fail preclear, or immediately are showing signs that the "reallocated" sectors are incrementing -- right out of the box.

 

It's become a general pre-warning that a drive won't stand up to unraid if it is misbehaving like this right out of the gate -- ego, the reason to run several passes of pre-clear to burn-in and do this testing.

 

I am a very impatient person,so for me to wait weeks to see if this thing will work,and if all drives are up to snuff is gonne be extremely hard.

 

LOL... You have lots of company here.  But this whole patience thing when it comes to building up your first unraid server with Preclears, etc falls into --

 

"You can pay me now, or pay me later."  {Fram oil filter commercials from the past}

 

I am sure that I have lost more time and worry about what turned out to be bad drives, than I ever would have spent having "done it right" and go through the Preclears.

 

And, if you can juggle things appropriately,  you can run more than one preclear at the same time {see the Preclear.sh thread about using the 'screen' utility to do just that}

Hehe Chuck, will do,i go the preclear route.. Regarding your comment about using the 2 port controller cards vs. the supermicro one,is pure and simple, more bang for the buck so to speak. Where i am from,the supermicro card runs about 220$ and i can get all 5 2 port cards for less than ½ that price,besides the 2 port cards are reported working,so i am confident it will satisfy my need to begin with.

 

However, at one point i do need to buy a 8 port card,as onboard and 2 port cards only add up to 14 slots, which is what i will end up at right out of the gate..

 

When i replace 1 2 port,and add 1 8 port,i will reach the maximum 20 ports,but that still leaves me running the 8 port in conjunction with 4 2 port cards..

 

I have been running 11x2TB USB drives with no backup succesfully for about a year now without any crashes (knock on wood) so running unraid with a parity drive regardless of hardware optimization will give me WAY increased speed and reliability i didnt have before,so once everything is running,hardware can always be exchanged for better/other later on

 

Again,than you all for your kind help to a total unraid noobz ;-)

What is minimum number of drives unraid works with? I only have 1 new drive,all the others are at capacity,and i suspect unraid would need 2 to begin with,parity and data drive,correct?

You can assign a single drive as a data drive and unRAID will run.

 

Also,when adding new drives and unraid needs to pre-clear them,does it do all new drives at once,or 1 at a time,so it will take weeks to prepare my 12 drives?

unRAID will not "pre-clear" them, but it will clear them after you assign them to the array.  It will clear as many drives that need to be done as it can.  The more you try and do at once the longer it could take.

 

Some of my drives has seen a VERY high load cycle count 50000+ will these drives potentially die using the preclear thingy?

any drive could die at any time.  My thought on the subject is that I would much rather have them die while doing a preclear on them then the dying while attempt to be used/set up in my unRAID array.

 

And why preclear at all,if i can recover from 1 drive failure with the parity drive? The parity drive needs to be the largest drive,or equal to largest size in array,correct?

Because, for new drives especially, they may be faulty or DOA.  the preclear script will allow you to clear the drive and set it up for use in unRAID before assigning it to the array.  If you assign a non precleared drive to the array in unRAID and there is a parity drive assigned it will need to do a lengthy clearing process and the array WILL be down for the entire time that takes.

Super. I will now await my new hardware arriving. Talk to yall wednesday as i am sure questions will arise once build is complete :-)

Excellent,thank you..

 

What is minimum number of drives unraid works with? I only have 1 new drive,all the others are at capacity,and i suspect unraid would need 2 to begin with,parity and data drive,correct?

Actually, it can work with 1, but not have parity protection for the data on that disk.  To protect the data, you need 2 drives as you guessed.

Also,when adding new drives and unraid needs to pre-clear them,does it do all new drives at once,or 1 at a time,so it will take weeks to prepare my 12 drives?

When initially defining an array, no disk needs to be pre-cleared, but all should be, as unRAID will not clear them on its own when initially setting up an array. 

 

However, unRAID does nothing to validate what you are loading on the array can be read back off the disks.  The first time you'll learn of a un-readable sector is when you  initially calculate parity.  At that time, the un-readable sector will just return a block of zeros, and that will be used to compute party for that block.  In other words, you've lost that block of data.  You cannot recover it from parity, as all it has is the knowledge it is a block of zeros.

 

Typically, one out of 5 drives has had some issues.  Your odds are not good at finding 12 perfectly working drives. 

 

When adding drives incrimentally to an existing array, either you must pre-clear a disk before assigning it to the array or unRAID will keep the array off-line while it clears the disk prior to formatting it.  (typically6 to 8 hours for a larger drive )  That is done in parallel by unRAID, but search and you'll see some hardware has a tough time writing to large numbers of initially installed drives at the same time and deadlock, slowing things to a crawl.  (Advice: once parity is established, incrementally add only one drive at a time to the array.)

 

Some of my drives has seen a VERY high load cycle count 50000+ will these drives potentially die using the preclear thingy?

the pre-clear thinggy will only load the heads three times... Once to read the disk, then, if the drive unloads them, again to write it, and lastly, again only if the heads were un-loaded, a third time to read it once more.  In other words, unless the firmware on the disks is really brain-damaged, it will probably be just one more load cycle.  As far as the disk spinning...  If you cannot get through a pre-clear cycle, yo really do not want the disk in your array.  Not if you value your time.

And why preclear at all,if i can recover from 1 drive failure with the parity drive?

True, but the re-construction can ONLY occur if it can accurately read every sector on EVERY other disk in the array.

 

Here is an example...  Let's  say sector 1000 on data disk1 is used to store your most precious document.  Let's say you are foolish enough to not have a backup elsewhere, so it is your only copy.  Let's say sector 1000 on data disk 2 is un-used, as that disk is now empty, but it did have files on it at one point that you've moved as you re-organized your data.  (it is not just all zeros)  Let's say you have established parity, so you think you can recover from a 1 disk failure.  (Oh yeah, you validated your precious picture's checksum when you wrote it, so it on disk1 is perfect.

 

Now... disk1 fails.  It now just makes a high pitched noise as it grinds itself apart mechanically.  You replace it.

 

To re-construct the precious picture on disk1 unRAID (actually, any RAID) needs to read sector 1000 of the parity disk AND sector 1000 of disk2 to calculate what it needs to write to sector 1000 on the replacement disk.  It reads the working drive's sector 1000 (parity and disk2)  does the XOR calculation to re-construct what was on sector 1000 of the failed disk.

 

If we cannot read sector 1000 of disk2, because of a un-correctable media error, it cannot accurately re-construct your precious picture.

The ability to re-construct a failed disk (accurately and correctly) depends on every sector on every disk being able to be read accurately and without error.  It is for that reason the ability of the pre-clear script to validate that ALL sectors can be written and read helps.

 

Now, that is not why I initially wrote the preclear script... The original reason I wrote it was to prevent unRAID from keeping the unRAID array off-line while it clears a disk with the array off-line as I incrementally added disks.  I started my original unRAID array with 2 of the largest drives available at that time, one for parity, the other for data.  They were 500Gig Hitachi drives and cost over $300 each.  Disks were smaller then, and far more expensive per GB.  as I added additional drives we soon learned we did not like the extended down time.  My wife and I use the array too much for it to be off line that long (4 to 6 hours or more at a time)...

 

Users on the forum quickly discovered how useful the pre-clear script was to detect those drives that were not suitable for the array, or would die an early death.    If you have the time... run through the pre-clear process.  If not, be absolutely certain you verify with checksums anything you write to the array is accurately written and perform a manual non-correcting parity "check" after initially calculating parity.

  The parity drive needs to be the largest drive,or equal to largest size in array,correct?

yes.  Correct.