Taking the plunge head first...With Unraid

[ChrisDaMan]

Hello all,

 

  I've been lurking around hear for awhile seeing what this Unraid thing is all about. I found this forum because of a video on youtube (Lime Tech) talking about Unraid. After watching videos on this about this setup and how excited the guy who was promoting it was I had to look into this further. I've been using 2 raid5 (5drive) Sans digital to serve my movies/TV shows for a couple of years. I thought I might buy another case which comes with a PCI-e card to run 2 of those boxes and begin expanding. Recently having acquired 6 WD Eminence 2TB drives and the 5 mixed inside 1.5 and 2TB in the server now which are not part of a raid. I wanted to have them all work together as one and have a rebuild capability or having to share so many folders in order to get my XBMC machines to see the whole library. So after seeing the video about Unraid and is potential as well as its green capabilities I said I have to go for this. I like the idea of the drives powering down and also the fact that they only spin up the drives needed to serve the file(s). My WD Eminence drives act the same way automatically in windows 7 which is nice but the raid drive all 5 spin if serving a file. So I read about the many systems recommended based on there drive quantities and I found the [glow=red,2,300]15 Drive Budget Box[/glow] would be good for me to go for and then as time goes on build on to it and add more drives and 1 to 3 cards depending or maybe just build another Unraid Server or change the drives out for bigger ones. (Green of course) Man just so many options to this thing.

I looked up the product recommend by Raj's in the 15 Drive Budget Box. Unfortunately the motherboard was not available so that lead me to looking for a replacement board.

 

MSI NF750-G55 AM3 NVIDIA nForce 750a SLI HDMI ATX AMD Motherboard

I went with this for the price break $30 off and the PCI-e slots (2) 16x and (2) 1X... Also 5 internal  and 1 external Sata.

Added 1 stick of Crucial 4GB 240-Pin DDR3 SDRAM DDR3 1333 (PC3 10600) Desktop Memory Model CT51264BA1339

I also purchased the 8 port Sata card recommended. And I'm glad to have room for a second one on the motherboard.

SUPERMICRO AOC-SASLP-MV8 PCI Express x4 Low Profile SAS RAID Controller

 

Now correct me if I'm wrong but from what I'm learning on this forum is that we want good mother boards with as many needed PCI-e Slots (x4 speed if possable)  ample speed processors to do the things we want to have the server do. Run the Unraid program and (Other apps...For me things like Sick beard and NZB is most important) I read this stuff takes more processing power to get done. So a single core would not do. I chose a dual core 65W AMD.

AMD Athlon II X2 260 Regor 3.2GHz Socket AM3 65W Dual-Core Desktop Processor ADX260OCGMBOX

I did a little price comparing to run it 1 month and 1 year 24/7 it wasn't that big of a difference in price. A meal here and there would pay the difference. (Gosh knows I could afford to skip a few of those ) the single core recommended in the build was a 45w(AMD Sempron 145 Sargas 2.8GHz). The Intel i-3 was out of line still for price yet an efficient processor.

 

As for the case Raj's recommended they no longer have it at NewEgg but found a replacement.

Xigmatek Utgard Mesh CPC-T90DB-U03 Black Steel / Aluminum and Mesh Bezel Computer Case

 

I'm also going with 3 of the 5x3.5 bays recommended.

Norco SS-500 5 Bay SATA / SAS Hot Swap Rack Module w/ 80mm

It's a lot to bite off now but I know I'll be glad I did in the future adding drives. As for the drives I have, they are all either green or energy savers and are 5400 to 5900 rpm.

 

For the power supply I chose to take it a step further from the recommended one and look at what power supply that was used in the 20 drive box. It was a 650w. Now thinking that I may add drives passed the 15 mark someday that I'll be glad I went with this one.

PC Power and Cooling Silencer Mk II PPCMK2S650 650W ATX12V v2.2 / EPS12V SLI Ready CrossFire Ready 80 PLUS SILVER Certified ...

 

All together $690 up front minus some rebates. (Hope I really get those lol) not to shabby for the 15 drive build. Up front cost was more then the 15 drive budget box but when there out of the product something else is there to take it's place. Thats whats great about options. But you really can waist time looking for what will work for your build. [glow=green,2,300]A Big Thank you goes out to you Raj's[/glow] for taking the time to post all that info on all those great builds. It really saves people time not having to search for computer parts that go together to make these great Unraid boxes.

 

 

I guess some of my other questions come into play about the existing Raid boxes I have and could they easily be erased and brought into the raid array? Would each box count as 1 or 5 drives in the system using the single e-sata cable that came with it?

http://www.newegg.com/Product/Product.aspx?Item=N82E16816111166

 

I did spot someone with Sans Digital boxes here but haven't asked them if they are in the Unraid setup or something else. I would think you could do it and why not use the same cards that came with them. But energy wise would you want to? PCI-e x1 small mini card with 2 ports external of the case. I see here folks recommending the internal hookup card to add 2 more drives which i may use on the motherboard in the future but starting out with the 5 on board and 8 on the card should get my feet wet in this project. I plan to allocate (1) 2TB drive for the parity and (1) drive not picked out yet for the cashe drive.

 

Have fun,

Chris

 

Looks like I'll be needing that Unraid Pro license soon...

[Rajahal]

Welcome to the community!  An excellent first post - it is always nice to see someone's exuberance when they discover unRAID and realize "yes, this is what I've been looking for!"  I'm so glad that my prototype designs were helpful for you, and that you've embarked on a solid first build.

 

You've chosen your hardware wisely.  Your motherboard looks good all-around, it has everything you need for a 15 drive budget box (including a good Gigabit NIC, which is getting harder and harder to find).  I think you've also made the right choice with your CPU since you plan to run add-ons, and with your PSU since you plan to upgrade in the future.

 

I haven't used that particular Utgard case, but it looks like a fine alternative.  I didn't realize that the case linked in my prototypes thread has been discontinued - I'll update those links later today.

 

I think you'll be quite happy with the Norco drive cages.  If you want a quieter server, the best thing you can do is to replace the 80 mm fans inside each cage.  I recommend this fan (and FrozenCPU is a good place to buy it).  In fact, I think I'll add that to my prototype builds page as well.

 

The SanDisk 5 Bay external enclosure that you linked most likely will not work on the motherboard's eSATA port as the motherboard's manufacturer page mentions nothing about port multiplier support.  However, it can't hurt to try it if you do decide to go that route.  The SanDisk has its own power supply, so the extra PSU overhead you have already invested in will be wasted.  You might consider upgrading to a 20 bay case (such as the Antec 1200) and adding a 4th Norco cage instead.  Granted, that will cost significantly more than your current investment.  5 drives on a shared SATA port will be slightly bottlenecked as well, but nothing too significant.

[revco]

That does look like a great build.  I wish I'd seen that Xigmatek case before purchasing my AZZA 910.  Even though I like the Azza's black color (inside & out), the Xigmatek looks more like a NAS and less like a computer case.  This is very similar to what I *wanted* to do, but had to cut back due to budget limitations.  What I would have liked to do is run 12 drives internally and then possibly more via an eSATA connected external 8-bay RAID case that acts as a DAS to the main unRAID box.

 

Although I'm by NO means an expert here, I would probably suggest this type of strategy to get your current and new setups to work together with minimal waste.  This should theoretically work - 5 ports on your MB, 8 on the AOC card, a 2 port SATA controller (for 15 drives in the case) and then a 2/4 port eSATA card to connect to your external drives via your SD boxes.  You probably have an eSATA card working with your Sans Digital boxes now?  You might be able to repurpose it for JBOD (depending on driver support, of course) and externally connect your existing SD boxes to the unRAID machine. In total, you'd sacrifice some drive bays in the second SD box (due to unRAID drive support limitations), but it gets you up to the limit.  If it were me, I'd try to max it out at 20 drives with just one of the SD boxes eSATA'd back to your new server.  The other SD may come in handy if Lime Tech expands the drive support.

 

Most eSATA cards with multilane support only support 4 drives per port, so you'd need to use 3 total multilane eSATA ports if you eventually wanted all 10 in the SD boxes supported, should the drive limit be expanded again.  I second the doubt that the motherboard's eSATA port would support multilane configurations...haven't found one yet that does.

 

Only thing I see with your build is you're missing a 2 port SATA controller, so you're limited to 13 drives until you get one.

 

Edit:  I don't know how many drives this MB supports, but it's worth looking into to see if you have any hard stops for total drive support.  I'd ultimately plan around this, if necessary.

[ProfQ]

Nice looking build. Welcome to the unRAID fever!

 

Depending on where you plan to place your rig,  and if noise could be an issue-You might want to replace the Norco 80mm fans in the SS-500 with something quieter. Something like the Coolink SWIF2-801 fans, or the Noctua 80mm fans should work well in the SS-500, if needed.

 

Congrats on such a good and solid start!

 

[ChrisDaMan]

Darn I got all the parts in and started putting it together. I powered up the motherboard and found out it was bad. Bios was damaged error. Went to MCI.com and learned what to do which looked like it was going to work during it's process. After flashing the bios the screen came up blank. Nothing nada. so back to Newegg it goes. so this will set me back some. And that's okay. I still have some reading to do on how the Unraid is layed out and how to transfer files make folders and overall functionality of this program. But it is what I want to do for organizing my media and serving it up on the network to my XBMC machines. Also throwing it out there would there be any good reason to have 8gig's of memory in this setup? 

[revco]

Dang, that sucks.  Out of box failures do happen...glad you're taking it in stride.

 

I think you definitely had a BIOS issue, but just as an FYI...MSI BIOS flashing procedures are notorious for having problems.  There's a 3rd party utility that you can find on the MSI user forums that does a MUCH better job at flashing an MSI BIOS.  It has built in error processing and can also run directly off a flash drive.

 

There's probably not a compelling reason to have 8 gigs of memory unless you're talking about looking into virtualization on the unRAID server.  If you're looking into that, it would be compelling to have even more memory than 8 gigs to allocate to your VM's.  There's a great tutorial in this forum section on how to run VMWare on top of unRAID.

 

Planning out your shares is important if you want to keep things organized in the long run.  Look at your data that you'll be storing on the server and categorize it at the high level.  (e.g. Music, Video, etc)  From there, decide if you need even further breakdowns of the categories and this will help you determine whether you need to play around with the various split levels.  Also, plan out how much data you'll want to store for each major category and this will help you with drive allocations and any disk sharing exclusions/inclusions you'll want to use.  Depending on how much control you want unRAID to have over drive allocations, you can really customize what goes where...even if you make all the decisions for unRAID.

 

Let us know how it goes!

 

[ChrisDaMan]

Hey Revco,

 

     Thank you for the great input. I will let you all know how this build goes. I'm reading everything I can about this great setup yet feel like i might miss something or am not sure on something(s) Now you said>>>Planning out your shares is important if you want to keep things organized in the long run.

My organization goes as far as 3 folders to start. Movies, HD Movies, TV Shows Each one will have X amount of Terra-bytes of files per folder. Or can I not do that? Especially if I'm expanding as time goes on. Adding say a 2 or 3 Terra-byte at a time. See My setup now I have 2 raid5 boxes (5 drives each) say 2 folders on each drive (all 5 drives are 1 drive letter) that hold my files. Then I have 6 Hard Drives in the Antec 900 case 5 Each drive only having 1 type of thing(Movies or TV Shows) then 7 USB 2TB drives the same way. With all these drives well organized yet so many shared folders picture adding those folder to an XBMC machine or worse try playing with say a Sony or WD media player and going through that mess of folders to find something to watch where I know you must know that XBMC once setup with those folders shared to it the library feature makes ease of use with that mess. But why keep it with  this Unraid system and the so many positives (mainly in the power part). If I can just make 3, 4 or 5 folders to share whatever data those may have and continue to add to them just like my Raid5 now then I'll be in hog heaven. If' it's not like that and I hope it is... then well...thats not so good. When I say add a drive. I'm making an assumption the files/folders on the Unraid will spread (like striping)over to the new drive or am I missing something and still have only those folders to add more stuff into. Or maybe user shares is my answer if the other way can not happen.

 

Thank you for your time,  

[Johnm]

would there be any good reason to have 8gig's of memory in this setup? 

 

Yes and no... in your case. once you start to fill up your 15 drive array, the extra memory would be very useful for running "Cache_Dirs".

otherwise your xbmc will spin up all drives every time you browes for a new movie/tv show.

 

Also the 8Gb would be handy if you tend to jump around in a movie a bit since slackware tends to buffer the file you are reading to ram.

 

I would just do it since DDR3 is almost the same price for 4Gb vs 8Gb right now... it might come in handy down the road after the price goes up.

 

Hey Revco,

 

      Thank you for the great input. I will let you all know how this build goes. I'm reading everything I can about this great setup yet feel like i might miss something or am not sure on something(s) Now you said>>>Planning out your shares is important if you want to keep things organized in the long run.

My organization goes as far as 3 folders to start. Movies, HD Movies, TV Shows Each one will have X amount of Terra-bytes of files per folder. Or can I not do that? Especially if I'm expanding as time goes on. Adding say a 2 or 3 Terra-byte at a time. See My setup now I have 2 raid5 boxes (5 drives each) say 2 folders on each drive (all 5 drives are 1 drive letter) that hold my files. Then I have 6 Hard Drives in the Antec 900 case 5 Each drive only having 1 type of thing(Movies or TV Shows) then 7 USB 2TB drives the same way. With all these drives well organized yet so many shared folders picture adding those folder to an XBMC machine or worse try playing with say a Sony or WD media player and going through that mess of folders to find something to watch where I know you must know that XBMC once setup with those folders shared to it the library feature makes ease of use with that mess. But why keep it with  this Unraid system and the so many positives (mainly in the power part). If I can just make 3, 4 or 5 folders to share whatever data those may have and continue to add to them just like my Raid5 now then I'll be in hog heaven. If' it's not like that and I hope it is... then well...thats not so good. When I say add a drive. I'm making an assumption the files/folders on the Unraid will spread (like striping)over to the new drive or am I missing something and still have only those folders to add more stuff into. Or maybe user shares is my answer if the other way can not happen.

 

Thank you for your time, 

 

I feel you on this one.. I am starting to regret my folder layout as i am running my weekly backup today to my second unRAID box...

 

I have..

DVD_Movies

Then a sub folder for each movie ==> Movie.name.year.DVD.(Then ISO if it is in ISO format)

Silver Streak (1976) DVD ==> then TS_Video or the ISO

(these still have spaces in them since they are rips that go back up to 10 years old)

 

SD_Movies (divx/xvid/wmv movies)

Then a sub folder for each movie ==> Movie.name.year.format(Xvid/divix/etc)

Double.Jeopardy.1999.XviD

Most of these are 2GB rips i made many years ago that I am trying to replace with Bluray when I they get released.

 

HD_Movies (Bluray rips and HD off my DVR)

Then a sub folder for each movie ==> Movie.Name.Year.720p-or-1080p.Source(Bluray/HD-DVD/HD-TV).Audio-format.container(MKV/ISO)

Tron.Legacy.2010.1080p.BluRay.DD5.1.x264

 

TV_Shows

then sub folders for tv show then sub-foldered by season by season.

 

A year or 2 ago, this was fine.. now it is getting a little out of hand now that i have over 1500 movies in just the root of the HD folder alone.. (I need to stop buying movies.. )

I am starting to wish i had one more level of sorting.. maybe sorted by year of the movie or genre (not really practical) or a-e,f-j,etc 

 

It is starting to make backing up and data syncing to my second unraid a bit more of a time consuming project then transparent.

 

[revco]

Wow...cached folders can take 8GB of RAM?    Unbelievable!  I suppose this also depends on the size of your array & the number of folders you're using in your array, too.

 

John's point is exactly what I'm getting at.  There comes a point when a certain number of subfolders becomes unmanageable and this is particularly an issue that plagues very large arrays.  I deal with massive SAN's, DAS and NAS for work, so I see how a lot of organizations paint themselves into a corner with poor organization.  Here's how I've laid out my shares of my music collection just to give you some food for thought:

 

Music (top level share)
-MP3
--A
---Artist-Year-Album
--B
---Artist-Year-Album
--C
---Artist-Year-Album
--etc
-FLAC
--A
---Artist-Year-Album
--B
---Artist-Year-Album
--etc

 

The idea is to keep your content easily accessible so you don't have to scroll through 10,000 folders to find what you want.  So if I want to listen to AC/DC, I can quickly drill down to the A folder, find the album I want and have it playing quickly.  I personally choose to keep my MP3's separate from FLAC's for the same reasons most people separate HD video from SD video.  Eventually, I'll have completed re-ripping my CD's to FLAC and my MP3 collection can be phased out.  If I had them all in the same folder, the phasing stage would be incredibly difficult!

 

I'm not a fan of breaking out movies/music/etc by genre because it gets to be a little hard to classify in some cases.  For example, is the movie "Taken" and action movie or a thriller?  Unless you can remember every category you file every little thing under or continually do searches, it's not a scalable design.  It's better to use obvious metadata, like the names sorted alphabetically, to categorize your data.  If you need better categorization, it's better to use some sort of catalog software that can put all the metadata into a database and make it more easily accessible...and this is completely independent of folder categorization.

 

With unRAID's design, you can easily spread shares across multiple drives like you want.  With the split levels and appropriate folder planning, you can make sure the shares will be 100% effective in distributing your data across drives like you want.

 

For more info that will help you with planning, read the following sections in this link - Allocation to Included/Excluded Disks:

 

http://lime-technology.com/wiki/index.php?title=Un-Official_UnRAID_Manual#Allocation_method

 

 

[Johnm]

Wow...cached folders can take 8GB of RAM? 

 

I suppose it could with a large array filled with small files.... but That was not what i meant.

I meant 2GB is not enough for any big add-ons and some people say 4GB is pushing it, especially if you are  running apps or doing mulltiple preclears.

With ram so cheap right now, I would upgrade. it can only go up in price...

I have 8GB on my box and it is only gets fully utilized while streaming movies to my knowledge. I guess i could look.

 

 

I'm not a fan of breaking out movies/music/etc by genre because it gets to be a little hard to classify in some cases.  For example, is the movie "Taken" and action movie or a thriller?  Unless you can remember every category you file every little thing under or continually do searches, it's not a scalable design.  It's better to use obvious metadata, like the names sorted alphabetically, to categorize your data.  If you need better categorization, it's better to use some sort of catalog software that can put all the metadata into a database and make it more easily accessible...and this is completely independent of folder categorization.

 

remember, while XBMC wont care... it sees all movie subfolders as one giant pool. it comes down to management for data-keeping server side.

 

I agree to many movies fall into multiple genres.

date is logical, unless you have no clue what year the move was made and you want to copy it to a portable drive to take on a road trip. 

that leaves title sorting.. and then the question is "the"... The Shining, Do i put that under T or under S....

 

plan for the future and a plan you understand..  all will be good!

 

 

[Rajahal]

If you plan to use cache_dirs (which is a great add-on, by the way), then I recommend 4 GB of RAM at minimum.  8 GB is fine too, the more the better.  I just know that using cache_dirs with only 2 GB of RAM on my personal server caused issues once and a while.  The only real issue was that powering down the server would sometimes take 15 - 30 minutes if cache_dirs was running (not a huge deal since I run my server 24/7, but definitely annoying if I need to do any troubleshooting or swap disks).  I talked to Joe L. (the author of cache_dirs) and he said that the issue was most likely caused by my server's RAM not being large enough to hold all of my cached directories.  I did have the cache_dirs script running at default settings, meaning that it was caching everything on the server.  I could have spent some time tweaking it to only cache movies, TV shows, etc. and I might have been able to get it down to a level where 2 GB is enough.  However, I decided that upgrading the RAM to 4 GB was a much simpler and less time consuming option.  I disable cache_dirs and ordered some new RAM.  That was a few months ago.  I have the RAM ready to go but still haven't taken the time to install it and re-enable cache_dirs!  Maybe I'll do that today, it should only take about 15 mins.

[kizer]

I'm running 4GB of Ram and I orginize my tunes just like that. Seems to work pretty well since if I'm looking for an Album that falls into the A folder I can quickly get to it.