Unraid with multirole/speed drives

[Madoc]

Ok playing with the trial and I've almost hit the point where this will do what I need.  Loving it so far with two exceptions.

1) Networking - I want both vms and docker to get ips/networking from the same nic group as unraid (refered to in most software as 'bridge mode'/bridgeing.) I got that working for vms but having trouble getting docker to do it. All I need is confirmation it IS possible and my config just isn't quite right, I can research more /mess with it.

2) This it he one I think might end up being a deal breaker. And the hardest to explain so let me go with planned HD layout.

 

3X 4TB Ironwolf (5900 RPM)

3X 1TB WD Black (7200 RPM) (cache? btrfs raid?)

2X 256G SSD (btrfs raid1 cache?)

1X 120G SSD (vm/docker store)

(I know, I'll need the tier2 licence, I'm fine with that)

 

I didn't realize you can only have one array till I went to create the second one.

 

All the hd's are stuff I had lying around other then the ironwolf drives.

So the ironwolves are my base unraid array, one parity 2 data, its all very clear there.

The 120 vm/docker store it looks like my best bet is unassigned with a user script to run trim once and awhile (I've been researching/playing with this stuff for 2 days now) and run backups of it to the ironwolf raid periodically.

 

Now the problem comes with 'what to do with the blacks'

I want to use the blacks as a sort of 'downloads/to be organized/non critical vms' high write playground. I was origionally thinking I could set it up as a cache however, can I have multiple caches? Can a cache have parity? These drives are older and I would expect will die first, so I want 1 parity drive or raid 5 or what have you. 

 

Options I have explored:

   A) Just add them to the array with the ironwolves, it'll work.

Absolutely it will but if I'm understanding how unraid works thats going to be speed capped by the ironwolf parity drive and beat the CRAP out of it because of the high write.

   B) Nested unraid, or go back to esxi and run 2 unraid vms.

Requires two licencees and seems like WAY overkill/more management and takes away a lot of the reasons I wanted to goto unraid in the first place. The all in one-ness/simplicity.

   C) Just add the blacks to the ironwolf array and buy a 4TB 7200rpm nas drive

Still sounds like an early death for the parity drive, and a now MORE expensive parity drive.

   D) Hardware raid 5 it and leave it in unassigned.

Loses me the monitoring/notifications that led me to using unraid to replace free exsi in the first place. Lesson learned after Losing 3 iscsi drives. (2 in one raid5 (ow) and 1 in another) And this loses me spin control does it not?

   E) Hardware Raid 0 it and put it in the array.

This one was a bit more interesting, still killing brand new ironwolf to parity old drives.

  F) Use the blacks as the cache and drop the ssds, and pray for new features.

I was thinking this but a lot of the stuff I've come across is just 100 different use cases for 'multiple arrays would be nice' going back YEARS. So I'm guessing since the feature isn't there. That there is a reason for that, and its not coming any time soon.

 

Ideas I'm still exploring:

There must be something I'm missing, can I have more then one cache pool/array? 

btrfs, this sounds like software raid, which would be fine but how to get unraid to manage the drives. Which is why I keep coming back to making it the cache.

 

 

Appreciate any tips/ideas I haven't explored. Links to documentation or google terms greatly appreciated.

The first ironwolf arrived (ordered from 3 dif places with dif shipping speeds) and I started copying off my degraded SAS raid. Thats been going over 24hrs now I'm REALLY hoping I dont have to whipe this machine and start over with another solution.

(Yea I know I have no parity right now, im counting my degraded SAS as a backup, once other ironwolves arrive and data is all transfered ill start the parity stuff)

 

 

 

 

 

 

 

Edited March 11, 2020 by Madoc

[trurl]

Multiple cache pools are planned but not yet available.

 

Personally, I don't think those 1TB HDDs add much to your setup, especially if they are old. If you are already planning to run the 120G SSD Unassigned for dockers/VMs, then the 2x256 SSD cache may be enough for caching user shares after you get the initial data load done.

[Madoc]

Your right, they don't add a lot of value but its non zero. And if they do die i'll replace them.  My use case has always worked very well with having a 'fast' and 'large' array. I'm already stepping down from 15k rpm 'fast' and 7200 rpm 'large'. The main thing I want from unraid is the monitoring and telling me when something is degraded. 

I have tons of 'to be dealt with' that has to be sorted and stuff before archiving. VMS that are very sporatic use. dbs/development stuff im activly playing ith.

 

More research and reading and it looks like going btrfs raid 5 in unassigned devices for the 1TBs will give me what i want for now at least.

256g raid 1  write cache should be plenty. I rarley add more then 100g in a day, and even if I do, from my reading it won't fail it'll just write to array directly.

120 for critical vms/dockers that need the performance.

'Fast' btrfs raid 5 in unassigned.

 

 

 

 

 

[trurl]

8 hours ago, Madoc said:

I rarley add more then 100g in a day, and even if I do, from my reading it won't fail it'll just write to array directly.

Cache-yes user shares will overflow to the array if Unraid decides cache doesn't have enough space. The way it decides this is with the Minimum Free setting in Global Share Settings. Unraid doesn't know how large a file will become when it chooses a disk to write it to. If cache has less than Minimum Free, it will choose an array disk instead. Once it has chosen a disk, it will try write the entire file to that disk even if it runs out of space. You should set Minimum Free larger than the largest file you expect to write to cache.

 

Each user share has a Minimum Free setting also, and it works in similar manner. If a disk has less than minimum, Unraid will choose another disk.

 

Using those old, small disks Unassigned seems a good compromise. I always discourage people from using disks just because they have them, especially in the parity array or cache pool. Each additional disk is an additional point of failure. To reliably rebuild a disk in the parity array, Unraid must reliably read all the other disks, so if any aren't reliable, they can compromise a rebuild.

 

Make sure you set up Notifications to alert you immediately be email or other agent as soon as a problem is detected. Unraid monitors certain SMART attributes for you. The default SMART monitoring is in Disk Settings, and you can override these for specific disks by clicking on a disk to get to its settings.

[Tony_L]

Regarding Q1 you can achieve that by editing the container, choose a custom interface (mine is br0) as network type and key in ip manually. It will show you the available subnet of that interface.

[Madoc]

WARNING: LONG AND RAMBLING AND I NO LONGER REQUIRE ASSISTANCE FEEL FREE TO IGNORE
 

I have hit the milestone that my old dl380g5  server and md3200i san are shutdown and officially replaced by my unraid machine. I bought the plus licence which is currently overkill for my new disk layout. But gives me room to expand. After all, thats one of the big upsides of unraid. Out of space? add a drive!

 

Networking: Yea, I knew about that and it works. But I wanted them dynamically assigned by my router. I tried leaving the ip blank but it apparently assigns them from its own pool. I ended up just going with a static ip and adding 'address' entries to dnsmasq on my router (which is now a vm on unraid!)

 

Drives: the 128g ssd died on me. I went back and forth on a bunch of designs on what to do. 

From my old san I still have 3x 500g 15k rpm drives and 5 2tb 5200 rpm drives. SAS ofc. So I was thinking of getting a controller and popping those in.

But the more I think of it, and knowing my own usage patterns, thats just overkill. And defeats my 'save money on power' objective. SAS drives do not spin down, do not sleep and are designed to have lots of fans(that sound like jet engines) on them at all times 

 

So here is the layout I'm going with.

3x 4TB ironwolf in the array. (only 1 in atm)

2X 256 ssd write cache

NEW! 1X 512gb sdd as vm/docker store. Once unraid supports multiple caches, I'll add it as a cache as well.

 

2 of the 1TB 7200rpm hd's are in my desktop, I'm going to clean them out onto the array then get them out of there.

The other 1tb hd is currently in my array because I filled the first ironwolf and the second hasnt arrived yet.

Which tells me my current cold storage is 5TB, the 8TB capacity I'm going to have should be plenty for a long time considering I bet I could cut what I have down by at least 30% if I sat down and cleaned them out.

 

I used to play web games in boring conference calls, my new boring conference call activity is cleaning/organizing data. I'm hoping to have most of it done before the parity drive arrives wednesday.

 

I ran into another wrinkle. My Unraid machine is my old work desktop. Thinkstation D30, neat dual CPU 16 ram slot tower. Basically a server board serving as a desktop. I bought a thingy off amazon to turn the 3 5.25 bays into 5 hd bays with a fan and its my 10 bay unraid host! (11 actually with 2 ssds in one bay)

 

However! When I went to start hooking up drives I noticed something odd. 2 red (Sata 3) 2 orange(sata 2) and 5 blue (sata 2)  If I'm doing the math right the sata 2's are fine for the ironwolves but not for the ssds. I'm still deciding what to do about that. I think for now I'm just going to use my 2 sata 3's for the write cache and put everything else on sata 2. Later I might add another controller but I'm done spending money on this for now, I've bought 3 4TB ironwolves, a 512g ssd and a 1tb ssd (for my desktop, which freed up its 256 for write cache) and a new 8U wall mount rack to replace the 32U old AS400 cabinet that the old equipment was sitting in. My budget for this project is already blown, twice

 

 

(4 blues are active, 2 dongles you cant get anymore 1 activated the 5th blue sata and the other activated it and turned them all into sata2/sas which is neat but they are impossible to find now and even if you can they are more then a new controller/cables would be)