Edvard_Grieg Posted November 24, 2020 Share Posted November 24, 2020 (edited) Hey all-  So, not new to unraid, but my current CPU/Mobo is getting a bit old. I haven't kept up as much on the current tech and could use some advice. Current setup: Supermicro 3U Chassis 2x E5-2660 OG (v0/v1) Asrock EP2C602-4L/D16 64GB DDR3 ECC 3xHP EX920 1TB NVME Cache running in BTRFS RAID 5 (I know, I know, it's being converted back to RAID 1 as we speak)  Current Workloads: Plethora of dockers (10-20 running at a time), standard usenet media toolchains Plex streaming upto 4K HDR for local content and 5-10 remote streamers typically max Ubiquiti Controller  I've got some intentions of going more the self-hosted route, running Wordpress for some sites, nextcloud, photo hosting etc etc  My observation has been some recent slowdowns on the docker side, and working on resolving the cache end of things, but I keep coming back to my CPU just being antiquated, and even though I've got plenty of threads at my disposal, they aren't exactly fast.   So all that being said, what can modern tech do for me? I'd like to upgrade to something that covers my needs, but gives a bit extra headroom (either more dockers/VMs etc and/or more demanding workloads (nextcloud, more remote plex transcodes etc)  Additionally, is there anything I would need to be concerned with not being 'plug and play' with the Supermicro Chassis/backplane/power supply etc?  From a budget standpoint, I'm not opposed to used via eBay, but in general looking for good value, just don't know what to look for these days. Probably looking in the $400-$800 range for CPU/Memory/Motherboard (assuming rest will come over). Of course anything coming up with holiday sales is always a plus.  Edited November 24, 2020 by Edvard_Grieg Quote Link to comment
ChatNoir Posted November 24, 2020 Share Posted November 24, 2020 Hey, unless I'm blind you mention your cache pool but not your Array drives and if you are running a GPU. Â This would have an impact on any recommandation. 3 or 20 HDD would not required the same hardware, especially if do use a GPU for media transcoding. Â This could orient you to consumer grade or Workstation / Server grade hardware. Â Quote Link to comment
PanteraGSTK Posted November 24, 2020 Share Posted November 24, 2020 I have a similar setup running on two different servers. One is the main server running 2x 2670 from the same generation as your 2660s. I haven't noticed any such slowdowns and our usage is very similar with the exception that my cache drive is a single m.2 1TB drive. Â I do have 2x GPU that plex can use for decoding/encoding so that takes a lot off the CPU. Â That server also has 2x parity and 22 array drives. Quote Link to comment
Edvard_Grieg Posted November 25, 2020 Author Share Posted November 25, 2020 11 hours ago, ChatNoir said: Hey, unless I'm blind you mention your cache pool but not your Array drives and if you are running a GPU.  This would have an impact on any recommandation. 3 or 20 HDD would not required the same hardware, especially if do use a GPU for media transcoding.  This could orient you to consumer grade or Workstation / Server grade hardware.  112TB of usable space across 14 drives including Parity.   I'd say I'm already much more in the Server grade hardware at this point (2x Xeon, Supermicro 3U Chassis with SAS Backplane, ECC Ram, Server Class Mobo), just happens to be from a different generation. Quote Link to comment
Edvard_Grieg Posted November 25, 2020 Author Share Posted November 25, 2020 (edited) 5 hours ago, PanteraGSTK said: I have a similar setup running on two different servers. One is the main server running 2x 2670 from the same generation as your 2660s. I haven't noticed any such slowdowns and our usage is very similar with the exception that my cache drive is a single m.2 1TB drive.  I do have 2x GPU that plex can use for decoding/encoding so that takes a lot off the CPU.  That server also has 2x parity and 22 array drives. Thanks for the comments, since posting, I think I've found my Portainer container being part of the source of the slowdown (no clue why, but speeds are better when it's stopped). I'm guessing it's doing extra polling and creating extra cycles- just haven't been able to dig into the why.  I have thought potentially about the GPU path....I feel like I started noticing it when helping a friend with his rig, he had a newer AMD CPU and GPU, and just some of the basic tasks (pulling down dockers and extracting) just flew in comparison...not sure if it's just a higher single core clock speed or something else.   Similarly, found when trying to run some dockers like Nextcloud etc it just 'chugged' same with Wordpress... Edited November 25, 2020 by Edvard_Grieg Quote Link to comment
PanteraGSTK Posted November 25, 2020 Share Posted November 25, 2020 12 hours ago, Edvard_Grieg said: Thanks for the comments, since posting, I think I've found my Portainer container being part of the source of the slowdown (no clue why, but speeds are better when it's stopped). I'm guessing it's doing extra polling and creating extra cycles- just haven't been able to dig into the why.  I have thought potentially about the GPU path....I feel like I started noticing it when helping a friend with his rig, he had a newer AMD CPU and GPU, and just some of the basic tasks (pulling down dockers and extracting) just flew in comparison...not sure if it's just a higher single core clock speed or something else.   Similarly, found when trying to run some dockers like Nextcloud etc it just 'chugged' same with Wordpress... True, it may be the containers you are using.  I have close to 30 dockers, but not all are in use. Portainer is one that is there, but I don't use it unless I need to.  Pulling down containers is pretty fast for me, but I'm used to this hardware. Single core clock speed could be a factor, but I'm not sure how much.  I'd be interested to see how your performance is after you change your cache pool. Perhaps the RAID aspect is causing it to slow down? Quote Link to comment
benyaki Posted November 25, 2020 Share Posted November 25, 2020 I dont mean to hijack this thread, but if anyone would mind giving me some help picking out some new hardware, that would be great. I am in the same boat, running basically the basic stuff, just older (though less dockers and less streams for plex) and I have been out of the hardware part of this for far too long to even know what to look for. I currently have a 4u supermicro server, ECC ram, 24 bay, 2 cache drives (HDD) and dual parity. Looking to upgrade the MOBO/CPU/RAM, controller card, and probably add a GPU in the near future. Would be looking at switching cache over to a dual 512GB M.2 pool. I could gut an old dell server from ebay, but wouldnt know what is going to be sufficient, and dont know what hardware would limit other (ie 'this motherboard doesnt support GPU passthrough' or something stupid). Very sorry for the ignorance. Quote Link to comment
Edvard_Grieg Posted November 25, 2020 Author Share Posted November 25, 2020 6 hours ago, PanteraGSTK said: True, it may be the containers you are using.  I have close to 30 dockers, but not all are in use. Portainer is one that is there, but I don't use it unless I need to.  Pulling down containers is pretty fast for me, but I'm used to this hardware. Single core clock speed could be a factor, but I'm not sure how much.  I'd be interested to see how your performance is after you change your cache pool. Perhaps the RAID aspect is causing it to slow down? So...I'm finding a really like BTRFS...was able to convert from a 3 drive RAID 5 to a 3 drive RAID 1 and then remove one of the drives all without losing any data.  Right now the separate drive is being used for downloads and post-processing activities which should hopefully isolate some of the IOPS work. The 2 Drive RAID 1 BTRFS array now has everything else...once 6.9 is stable I plan to upgrade and create a separate cache pool instead of the unassigned route.  So far it's feeling a little speedier, but time will tell...definitely need to put it through its paces...probably will pick up a 4th NVME to round out the set and run the downloads and post-processing as RAID 0.   Still have some minor itches to upgrade....toning down the question a little, are there any direct CPU upgrades that would be compatible with my motherboard that I might see a decent speed improvement with?  What would be a good value GPU recommendation to offload HW Transcodes? Quote Link to comment
PanteraGSTK Posted November 28, 2020 Share Posted November 28, 2020 On 11/25/2020 at 2:34 PM, Edvard_Grieg said: So...I'm finding a really like BTRFS...was able to convert from a 3 drive RAID 5 to a 3 drive RAID 1 and then remove one of the drives all without losing any data.  Right now the separate drive is being used for downloads and post-processing activities which should hopefully isolate some of the IOPS work. The 2 Drive RAID 1 BTRFS array now has everything else...once 6.9 is stable I plan to upgrade and create a separate cache pool instead of the unassigned route.  So far it's feeling a little speedier, but time will tell...definitely need to put it through its paces...probably will pick up a 4th NVME to round out the set and run the downloads and post-processing as RAID 0.   Still have some minor itches to upgrade....toning down the question a little, are there any direct CPU upgrades that would be compatible with my motherboard that I might see a decent speed improvement with?  What would be a good value GPU recommendation to offload HW Transcodes? GTX 1050 and above will help, but getting a card that doesn't have a limit wold be better.  It looks like your board can use the e5 26xx v2 series so you have lots of options. Those CPUs are cheap on ebay depending on which one you want.  https://ark.intel.com/content/www/us/en/ark/products/series/78582/intel-xeon-processor-e5-v2-family.html Quote Link to comment
Edvard_Grieg Posted January 11, 2021 Author Share Posted January 11, 2021 On 11/28/2020 at 9:10 AM, PanteraGSTK said: GTX 1050 and above will help, but getting a card that doesn't have a limit wold be better.  It looks like your board can use the e5 26xx v2 series so you have lots of options. Those CPUs are cheap on ebay depending on which one you want.  https://ark.intel.com/content/www/us/en/ark/products/series/78582/intel-xeon-processor-e5-v2-family.html So I'm still back and forth on this, and I've been doing some system tweaking with my current system. Checking that all dockers point to /mnt/cache/appdata vs /mnt/user/appdata, moving the cache drive to RAID 1 and a separate unassigned drive etc. All these things seem to help in some ways; however, I'm still finding some sluggishness when I have a lot of dockers running.   I end up leaving the main media content dockers running, but I've got another dozen that I leave off until needed and would like to just be able to leave them running.  At one end I've been looking at the E5-2697 v2 and getting a pair cheap on ebay + some faster memory.   Alternatively I keep eyeing some more modern AMD options, but it looks like every single one is going to cost a lot more, not just the cpu, but obviously will need new motherboard, ram and possibly a GPU.  What are the thoughts here? is the 2xE5-2697v2 enough of an upgrade over 2xE5-2660 to be worth the money, or am I better off investing that towards something more modern? I've been out of the CPU game so long I'm just reading up on Ryzen vs TR vs Epyc - it appears TR and Epyc may be too much money (not sure) and I've been reading mixed things on overall stability of Ryzen on Unraid....Thoughts? Quote Link to comment
bigzaj Posted January 11, 2021 Share Posted January 11, 2021 Not going to help this along, because I'm in the same boat. Running older gen dual Xeon on FreeNAS. Looking to make a change to unRaid for ease of use. Really looking forward to see how this thread develops.  My goals are the same, upgrade to newer gen hardware but prefer enterprise reliability. Looking to expand to 150TB+, 4K UHD, ~10+ concurrent streams / transcodes (I don't allow transcoding of 4K). Would like to explore HEVC h/x265 for space savings which pushes me towards a need for GPU transcoding.  Not really sure the best migration path for hardware, etc. while changing OS as well. Quote Link to comment
Edvard_Grieg Posted January 11, 2021 Author Share Posted January 11, 2021 @bigzaj I started with FreeNAS (or something similar) about 10 years ago, and after a physical raid card failed and couldn't recover the array I switched to using unraid nearly 9 years ago.  I started with a basic Dell Optiplex Celeron desktop, and have iterated from there. Prior to the current 2xE5-2660 v0 that I'm currently running I had been running 2xX5482. That move was pretty substantial, and hoping next one is as well. Honestly, were it not for trying to run some other workloads (self-hosted cloud etc) the current CPUs would probably be ok.  I'm at ~110TB with plenty of 4K UHD, Atmos as well as HEVC etc.  Part of the way I've addressed the majority of the local use is by ensuring my endpoints don't require any transcoding by using nvidia shields that will handle any format natively. I limit remote streaming to lower resolutions from a bandwidth standpoint as my upload is not extensive. Typically I don't see more than 5-6 concurrent remote streams, and often the likelihood of multiple major transcodes is not common.  This is all to say that even with my 'older' hardware it may not be far off for meeting your needs. I'm also pretty confident you'll be quite happy with unraid as a platform. There are other system architecture considerations to make (especially with a larger library) to ensure that the right drives are performant when needed. I think many are looking forward to 6.9 which will allow for multiple cache arrays (without doing the unassigned drive dance) and thus spread out some of those storage systems that require faster storage.  For me, it's the added workloads, and ensuring any transcoding activity is non-impactful. Quote Link to comment
bigzaj Posted January 11, 2021 Share Posted January 11, 2021 I am running dual E5-2670v2's with 96gib ECC ram and 20x6TB WD Red in two vdevs.  I am really just looking for less power consumption and more flexibility. I've lost all of my data once on FreeNAS with multiple failures. My Vdevs are RaidZ2 10 disks so effectively 4 parity drives to 16 storage drives. I can handle two concurrent disk failures per vdev. However FreeNAS is just such a steep learning curve and it's a pain moving from release to release with the Jails. I've setup friends in unRaid and it is so easy. My only concerns are really data integrity (bit-rot) and speed of opening files. Quote Link to comment
Edvard_Grieg Posted January 12, 2021 Author Share Posted January 12, 2021 Nice- setup, aside from the itch to reduce power consumption you're probably closer to good than not for performance. I gave up on the power consumption piece....with your setup are you thinking of replacing your drives too?  I haven't experienced any data integrity issues with my use. Speed of opening files generally is 'fine', my use case is primarily media server and don't typically see any notable loading times.   My intent is to be running nextcloud, wordpress and others- right now it tends to add to the sluggishness when adding that in with other media processes. Quote Link to comment
bigzaj Posted January 13, 2021 Share Posted January 13, 2021 Mine is primarily for Plex, but have been expanding video editing / rendering stored on this machine. Â The goal was reduce power and replace drives, cut out 8 hdd's and a processor and run 12 x 18TB Â To be honestly, I am now considering just running this setup for another year to see where AMD + Unraid end up. Quote Link to comment
Edvard_Grieg Posted January 15, 2021 Author Share Posted January 15, 2021 On 1/12/2021 at 5:50 PM, bigzaj said: Mine is primarily for Plex, but have been expanding video editing / rendering stored on this machine. Â The goal was reduce power and replace drives, cut out 8 hdd's and a processor and run 12 x 18TB Â To be honestly, I am now considering just running this setup for another year to see where AMD + Unraid end up. That one's tricky, from half of what I read Intel with QuickSync is the way to go there (been some really good prices on the i9-10850k) but then you're no longer server class, new motherboard, memory yadda yadda yadda.... Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.