Is Unraid for me?


Recommended Posts

Hi,

I got here via the AVS forum and read quite a bit on this unraid forum. But I have to admit that I am a linux newb, so a lot of the tech talk remains a mystery to me. Still I have the feling, Unraid could be the solution to my needs. I hope some of you could give me a quick evaluation.

 

I am looking for a media server for my home theater. Currently I have a Vista HTPC playing 90% Blu-Ray rips. That means either mounted isos or folders with rips. Playback software are mostly Arcsoft TotalMediaTheatre or Powerd DVD 8. But my collection also has HD TV Caps as .ts and some .mkvs. For those I use directshow players.

 

My collection is on sata hdds and they sit in a drawer. I use a database (DVDProfiler) to find the movies, then pick the numbered disk and use a sata docking station (like this one http://www.newegg.com/Product/Product.aspx?Item=N82E16817153066.

 

I currently estimate +20TB

 

Now to my questions:

If i setup an Unraid system and use the share level system to put all movies in a share named e.g. "Blu-rays", would I see a huge folder on my Vista machine with all movies in there as if it was one gigantic disk?

 

If each Blu-ray folder is limited to just one physical disk (not spread over 2 disks), would all the other disks kept spundown if I stream just the one film?

 

I read that one should not copy files through user shares, how do i easily transfer files from the Vista machine to the unraid system? I use Total Commander on the Vista HTPC, could that be easily used?

 

I would only want to build a server, if I can build it to be quiet and energy efficient. Power is expensive over here, so i would need unused disks to stay spundown. Is unraid a solution for me?

 

Would you think this whole thing is manageable without linux skills? I build PCs and setup regular systems without too much hassle.

 

Thanks alot for your help!

Dominik

Link to comment

If i setup an Unraid system and use the share level system to put all movies in a share named e.g. "Blu-rays", would I see a huge folder on my Vista machine with all movies in there as if it was one gigantic disk?

 

Yes.

 

If each Blu-ray folder is limited to just one physical disk (not spread over 2 disks), would all the other disks kept spundown if I stream just the one film?

 

Yes, in principle at least. In order to find what you want to play, though, you may need to browse through different folders, which may spin the corresponding disks up. Fortunately, there are ways to ensure that the file-system content remains cached in RAM, which would leave disks spun down when you browse.

 

I read that one should not copy files through user shares, how do i easily transfer files from the Vista machine to the unraid system?

 

Through user shares is fine. unRAID can also expose the physical disks directly (and can even do this so they are not listed when browsing). I personally keep a record of what type of media is where, and dump any additions to the disk where I want it, via a cache disk (see the documentation).

 

I would only want to build a server, if I can build it to be quiet and energy efficient.

 

Mine has 12 disks, and uses not more than 80 watts when the disks are asleep. And, with recently-available components that figure can easily be brought further down.

 

Would you think this whole thing is manageable without linux skills?

 

Yes. unRAID is designed to require no operating-system skills to set up, and it achieves it admirably well. It's a slightly different story if you want to tinker but a number of people here have shared how they have done the most commonly-desired things (in quite spectacular fashion as a whole) and their approaches are well documented and, in fact, are typically available through downloable scripts and more.

Link to comment

Thanks ReneV!

 

Mine has 12 disks, and uses not more than 80 watts when the disks are asleep. And, with recently-available components that figure can easily be brought further down.

 

What recently-available components do you think of? Or is this generally speaking for modern CPUs that clock down efficiently when not in use and "green" hard drives?

 

Concerning the feature of file-system content remainig  cached in  RAM, is this built into unRaid or externally? Because I read one thread mentioning a seperate program and this seemed not too easy.

 

Cheers

Dominik

 

 

 

Link to comment

I'll just point out a couple of things;

 

unRAID is limited to 15 data disks. So, there is that limitationon to consider. With WD releasing 2T drives you could store 30T though.

 

You can set the disks to appear in the shares. So, when you go to the server on the network you'd see disk1, disk2... disk15, Movies. So, you can access the disks and move movies onto each one in order. Total Commander would be great to use because it's supposed to be able to set-up a "move" que and also it can throttle the move speed. If you are moving data with a parity drive assigned then the move speed is limited by the disk access speed and not the network. Even connecting the drives directly to the new server would not move quicker.

 

unRAID is very easy to use. Create the flash drive, set the computer to boot from flash and you're working. If you can follow the instructions, I'd suggest installing unmenu because it gives you extra troubleshooting tools.

 

If you do decide on unRAID, then I would build the server with the new drives but not assign a parity drive (you can install and connect it physically but just don't assign it). Copy the data at a much faster speed then assign parity and let it build before destroying the data on any of your existing drives. I'm not sure what your plan in for the old disks but you could be planning to add them in as you go too.

 

There are ways to mount the data drives you have on the new server and copy directly there too. In theory, you could connect 1 new parity drive, one new data drive and 14 of the existing data drives. Then, keep moving data from an existing disk and then assigning that newly emptied drive to the array again and again until all the drives were in the array and never even power down.

 

Peter

 

Link to comment

Just thought I'd chime in and say that I use DVD Profiler as well, and have configured the LoadDVD plugin to figure out which drive my movie is on and play it from there - all without using user shares.  (Not that user shares are a bad way to go, I just have never had a need to use them).

 

unRAID is a great tool, and will provide you a level of recoverability if one of your drives were to fail.  I (personally) would be a bit nervous plugging and unplugging internal hard drives in and out of my computer on a regular basis.  They have explosed electronics on the bottom and I'd be nervous about static and accidents.

 

unRAID requires a drive as big or bigger than your largest data disk as "parity" to allow recoverability.  This has saved the skin of more than one user.

 

15 drives (+ parity + cache = 17) is the max right now, so if your needs are greater than that, you may need to set up 2 servers (or invest in some larger disks).  (I say that's the max, but you can mount other disks in the array as well, they just won't be parity protected).  20TB is alot.  With 1T drives you could hit only 15TB, but with 1.5T drives could get to 22.5T.  With 2TB drives you're up to 30TB.  My guess is that you're not going to be up to replacing all of your disks, so this is likely a make or break issue for you.

 

I think you'll start running into practical considerations in trying to get more than about 20 drives in a computer.  I'd like it if Tom expanded unRAID a little - maybe to 22 or 24, but anything past that is likely overload IMO, unless someone had externally and separately powered drive enclosures.  For me, the drive limit is not a serious limitation at this point in time.

 

Another point you mentioned was you wanted the server to be quiet.  Sound level is normally caused by fans, not drives.  It is advised to use sufficient cooling to keep drives in the 20s or 30s (degrees C that is).  I don't think silent operation is really achievable, but there are lots of techniques people have used to keep the sound down.  You may also find that since you don't need physical access to the server, that you can find an out-of-the-way spot for it and not have to worry as much about noise.

Link to comment

I thought this was a good example of the subject question, well answered, good enough to point to in the FAQ, so I have added it to the FAQ (link back only), here.  I added a link to the poll about how unRAID is used, and the excellent and very candid discussion "Seems to me unRAID is NOT cheap".  I would love to see more links to forum threads of a similar type, or even those of anecdotal interest as to "why unRAID".  This question was added to the opening section, after 2 questions about the pros and cons of unRAID vs other solutions.  So this should help to round out a balanced view of unRAID, help prospective users decide if unRAID is right for them.

 

Feel free to edit and add.

Link to comment

 

Mine has 12 disks, and uses not more than 80 watts when the disks are asleep. And, with recently-available components that figure can easily be brought further down.

What recently-available components do you think of? Or is this generally speaking for modern CPUs that clock down efficiently when not in use and "green" hard drives?

 

I was thinking mostly of modern CPUs and hard drives that consume little power when not in use.

 

Concerning the feature of file-system content remainig  cached in  RAM, is this built into unRaid or externally?

 

The way to do it is to regularly run 'ls -R' on the relevant folders. It is not built into unRAID out of the box, but is not hard to do. The underlying operating system makes available a service called `cron' that is used to run jobs regularly (http://www.google.com/search?rls=en&q=man+cron&ie=utf-8&oe=utf-8). Cron is  configured in a file (and with a command) called crontab and all you need to do is add a line to that --- see the documentation.

Link to comment

With 20+TB of data you have quite the collection going.  We might be able to suggest and give better advice if we knew how many HD's you have exactly.  Like others have pointed out you are going to hit the wall at 15 drives for one server (in the protected part of the array).  I would imagine that you are going to want to build this machine to accommodate most if not all of the drives that you have.  If will probably not be a quick process to get the data from your drives onto the drives in the unRaid machine.  We can probably suggest the best/easiest/quickest method but again that depends on a few things.

 

We could probably give you suggestions on pieces and parts to build the machine if we knew a budge you were trying to stay around.  But as is stands with you saying that you have 20+TB's you are probably looking at doing the build with hotswap bays and everything (or a case like this which has the bays already in it).  The case is not cheap but with 20+TB you are probably going to want a nice case to start with.

Link to comment

Thanks again for all the input!

 

With 20+TB of data you have quite the collection going.  We might be able to suggest and give better advice if we knew how many HD's you have exactly. 

 

Ok, you asked for it.  ::)

26x 500GB

4x 750GB

8x 1000GB

+ lots of DVD-Rs from the time before I chose HDDs for storage.

I have another 2x 1000 and one 1,5TB Seagate waiting to be filled, the latter finally has a new firmware. I still do not know if they are to trust.

 

The way to do it is to regularly run 'ls -R' on the relevant folders. It is not built into unRAID out of the box, but is not hard to do. The underlying operating system makes available a service called `cron' that is used to run jobs regularly

 

That sounds ok. I read this thread before http://lime-technology.com/forum/index.php?topic=2179.0. "Slocate" was suggested there, but I hardly understood how to use that.

 

Case and build:

I was thinking of a Norco 4020, after reading quite a bit about it, but size and noise led me to another idea. Currently I am more thinking about a big tower, that can be well ventilated using at least 120mm fans. The I thought, even if the current limit of unRaid is 16 drives, there are hints here, this might change, so my build needs a little overhead. I read a lot of suggestions on this forum and came up with a rough idea:

 

Lian Li PC-A77B CASE 12x 5,25 bays plus a cage for 3 HDDs in the top back

I could use 4 Lian Li EX-H34B to get well cooled and probably quiet 16 drives using the externals or

I could use 4 5in3s to get 20 crowded and probably hot quick trays

 

Supermicro MB775 I-3210 X7SBE

This board sounded nice, as it has 2 PCI-X and 6 onboard Satas.

 

Combined with 2x Supermicro Cont SATA2 AOC-SAT2-MV8 PCI-X

I would reach 22ports.

A small CPU like a E5200 PLUS 4 GB DDR2 RAM

 

Budgetwise I am unsure. The basic hardware like above gets me to something like 1200€ excluding hdds. With 5in3s even more.

 

I guess, before I start with this project, I should probably wait for the WD 2TB hdds to come out. Otherwise, I will definitely have to prioritize my collection and keep the drawer solution for the not so relevant films.

 

While waiting fro the 2 TB units, I could build a test setup with smaller drives. As I need a small file server for my business as well, I could use that to start with. Guess, the 2-pack license comes in handy there.  ;D

 

Dominik

 

Link to comment

If I was you, I would build several servers using your current HDDs. Most reasonable builds will not need updating for the foreseeable future, and if you get one server set up and working to your liking, it ought to be straightforward to replicate the build. On the issue of max number of HDDs, I'm with Lime Tech: going beyond 16 data drives seems to be a case of rapidly diminishing returns. In fact, I expect I'll be wanting to stay (well) within the 16-HDD limit even after unRAID gets dual-parity support.

 

As for cases, I'm partial to the Antec Nine Hundred for home-server purposes and I would expect the Antec Twelve Hundred to behave similarly. In particular, I use 3 5-in-3 backplanes and tape everything up except the air intake in front of the HDDs and use just the top 200mm fan at the lowest setting for cooling (incl of the CPU). It's all but silent and temperature has never been a problem.

Link to comment

If I was you, I would build several servers using your current HDDs. Most reasonable builds will not need updating for the foreseeable future, and if you get one server set up and working to your liking, it ought to be straightforward to replicate the build. On the issue of max number of HDDs, I'm with Lime Tech: going beyond 16 data drives seems to be a case of rapidly diminishing returns. In fact, I expect I'll be wanting to stay (well) within the 16-HDD limit even after unRAID gets dual-parity support.

Hmm, this is really hard to judge for me. Building a system with up to 16 disks and keeping it quiet seems a lot easier, than trying to get everything in one system. I am at over 20TB now, but this is not the end, so I might not catch up at all. Sounds plausible to me, I will think about 2 seperate systems.

 

As for cases, I'm partial to the Antec Nine Hundred for home-server purposes and I would expect the Antec Twelve Hundred to behave similarly. In particular, I use 3 5-in-3 backplanes and tape everything up except the air intake in front of the HDDs and use just the top 200mm fan at the lowest setting for cooling (incl of the CPU). It's all but silent and temperature has never been a problem.

 

I was also consiering the Antec twelve hundred, as it is very cheap compared to the Lian Li's. What set me down a bit is that it seems to be lit like a christmas tree. I would need it to be dark. Would I need to exchange all fans?

 

Those 5in3s have 80mm fans. How would you rate their noise level?

If you tape up the case, you are trying to force the incoming air through the 5in3 front, right? Is that enough cooling for the drives? Almost all my HDDs are Samsung F1s with 7200RPM. I always thought having a fan blowing actively on the disk is more efficient compared to a fan pulling air over a disk.

Link to comment

I was also consiering the Antec twelve hundred, as it is very cheap compared to the Lian Li's. What set me down a bit is that it seems to be lit like a christmas tree. I would need it to be dark. Would I need to exchange all fans?

 

I only use the top fan, which has no built-in light in my two (slightly different) Nine Hundreds. If I needed to use additional fans, I would make sure the case is out of the way, or replace the fans or try to cut the light wires. But then again, I tape-over most stand-by lights and similar ;-)

 

Those 5in3s have 80mm fans. How would you rate their noise level?

 

Mine come with 3 settings and the lower 2 settings are kind of OK. But, I only used the fans for a short while before taking them off completely. There is a noticeable difference between using them and not.

 

If you tape up the case, you are trying to force the incoming air through the 5in3 front, right? Is that enough cooling for the drives? Almost all my HDDs are Samsung F1s with 7200RPM. I always thought having a fan blowing actively on the disk is more efficient compared to a fan pulling air over a disk.

 

Yes, it's enough. The drives are asleep most of the time, during which temperature is a non-issue. When parity checking in sub-30C ambiant temps, the drives max out at high-40ies C.

Link to comment

OK, sounds like you have some good hardware ideas. I might suggest adding extra "slots" so you can plug-in more drives above the 16 drive array size. This can give you the chance to plug-in some non-array drives and copy data to/from them.

 

You have 10 1T drives. If you use these you could add 5 more 2T drives and get to 20T to start. Then, later on you would replace the 1T drives one at a time to expand the array to 30T. Or, you could start another system and just begin with the 2T drives in that system.

 

I'd seriously reconsider the Norco case and try to find an out of the way place in the house to stick it. It's very possible you will be expanding to 2 of these in a few years and if you get to that level you'll want them stored out of the way where you don't have to listen to them. Quiet and cooling 16 hard drives are not compatible goals. Starting with that case can get you 2 systems cheaper than the one.

 

Peter

 

Link to comment

I was going to guess 40.  Pretty close.

 

I would suggest "seeding" your array with the new 2x1T + 1x1.5T (as parity).  Keep the removable tray, and you should be able to insert your disks one at a time and copy the data from the NTFS formatted disk to the unRAID disks.  Once you are done, you can add the 2 1T disks (you just copied from) to the array, format them, and repeat with 2 other 1T disks.  Repeat until all the 1T disks are added.  I'd likely stop with the 1T drives (that would be 10data disks), but you could proceed to the 750Gs if you like.  I'd leave the last couple disks empty to use for new files.

 

I'd start investing in 2T drives once the price is reasonable.  You might even consider living in his hybrid mode for a while and wait for 4T drives to come on the market.  You're never going to squeeze your collection onto even 1.5T drives unless you are willing to replace everything, and even then you have near 0 room to grow.  2T are not going to be much better, and you'd hate to make a huge investment in them only to have to replace them.

 

If this is too much of a compromise, you might be able to set up a second array with your 750G and some of your 500G drives.  Maybe some of the movies aren't your favs and could stay "in the drawer" for a while longer?  You might also look into techniques to reduce the file sizes if you are ripping complete DVDs and BRs.

 

If you did this, you'd have 1 array with 3 750G and 12 500G (+1 750G parity) maxed out.  You'd have another array with 10x1T (+1 1.5T parity) with room to grow.  The latter array could grow with 5 2T drives giving you another 9.5T of space before it is maxed out.  After that, you can start replacing the smaller disks in the first array with bigger ones.

 

Ultimately, with 4T drives, you could handle your existing 20T of space with 5-6 data disks, leaving you 9-10 data disks worth of room to grow.  This sounds like where you need to be to satisfy your needs with a single array.  Avoiding spending a fortune in transition waiting til these are available is the tricky part.  (There are no promises, either, that 4T technology will arrive that quickly.)

Link to comment

Thanks again for all the input!

 

With 20+TB of data you have quite the collection going.  We might be able to suggest and give better advice if we knew how many HD's you have exactly. 

 

Ok, you asked for it.  ::)

26x 500GB

4x 750GB

8x 1000GB

+ lots of DVD-Rs from the time before I chose HDDs for storage.

I have another 2x 1000 and one 1,5TB Seagate waiting to be filled, the latter finally has a new firmware. I still do not know if they are to trust.

 

I was going to guess around the 40 mark. 

 

I would say that you probably need to decide how much you want to spend and if you want to do this all at once?  If you want to do this all at once then you are going to have to build 2 systems, and it probably would not hurt to replace some of the 500GB drives with larger ones (ie 1TB's or 1.5TB).  The WD 2TB's are a little to expensive right now but i would search around for deals on 1TB+ drives and start to upgrade the 500GB drives to 1TB+ drives.  Once the system is built it should go fairly smoothly.  It will just be a time consuming process to get the data moved from the NTFS drives to the unraid ones.  There are tools that will make that easier (unMenu).

 

For your massive amount of drives i highly suggest getting the 5in3 backplanes for whatever case you get, or getting a case (like the Norco) that already has them there.  I also suggest trying to find a place to put this server.  It's not overly difficult to run some Cat5e/Cat6 cable to were it might need to go.

 

I think your best bet is a sort of combination of the two systems for now.  I, being the computer geek i am, would want to do this all at once… kinda for the "thrill" of it.  I know that sounds incredibly geeky but since setting up my unRaid system I have been adding drives and backing up the stuff that is important.  I have been very happy with the system as a whole.

Link to comment

Again, let me thank you all for your help. I have seen quite a few forums, but none so far has treated newbies with so much help. None of you has pushed me to the search function or RTFM. This gives me the comfort to try out unRaid, knowing that I will find help when I need it. This is really appreciated.

 

Back on topic  :D

My mediaserver plans are really only the ultimate goal, I have time for that plan. And thinking about the size of my collection, I will really need big disks in the long run. There is no sense in grouping some 40 disks into one private system.

 

In my office I also have multiple HDDs laying around with lots of video data, that so far are not properly backed up. This is something I urgently need to change. My thought was, to setup an unRaid system for my office, using up some of my smaller 500GB disks from home and therefore gain some more 1TB disks from office. That way i have a setup to get comfortable with unRaid and also test some hardware for the noise and heat levels.

 

So, I will definitely buy the 2-pack license to start with.  ;D

 

Now I thought the whole morning about what case and HDD cage configuration I should finally go for. A 12-bay case with 4x 5in3 backplanes costs a fortune. On the other hand, getting a Norco 4020 in Europe seems difficult, as they have no distributor over here. Still importing one from the states is a lot cheaper than 4x 5in3 backplanes alone.

So I decided to go for the Norco4020 in the office. If I can get it reasonably quiet, I could then build a similar system for my home media server. By that time the 2TB disks have hopefully become available in europe. I think I really should start my home media server with at least 2x 2TB disks.

 

Two questions I still have:

We use Macs at the office. Are there any problems with Macs and unRaid?

The disks in unRaid have some sort of linux file system. Is there a files size limit for these, as I have some individual files of about 55GB size?

 

Again, thanks for your patience with me! I will keep you posted.

 

Cheers

Dominik

Link to comment

 

But the xcase is no longer available. I contacted them and asked about it.

 

Meanwhile I have also two mails out asking norcotek directly and an ebay seller on shipping to germany.

 

UPDATE: I received an answer from xcase: the 420 will return.We are making a few changes I expect to see it back in late March

Link to comment

Two questions I still have:

We use Macs at the office. Are there any problems with Macs and unRaid?

The disks in unRaid have some sort of linux file system. Is there a files size limit for these, as I have some individual files of about 55GB size?

 

Again, thanks for your patience with me! I will keep you posted.

 

Cheers

Dominik

 

I use my unRaid system with my Mac at home and I don't really have any problems with it.  Please see this thread and this thread for some specifics on getting everything to work perfect.

Link to comment

As for cases, I'm partial to the Antec Nine Hundred for home-server purposes and I would expect the Antec Twelve Hundred to behave similarly. In particular, I use 3 5-in-3 backplanes and tape everything up except the air intake in front of the HDDs and use just the top 200mm fan at the lowest setting for cooling (incl of the CPU). It's all but silent and temperature has never been a problem.

 

I second the Antec Nine Hundred.  Best case I've ever had bar none.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.