February 3, 201610 yr I'm trying to optimize my N40L server for a large WORM media storage array. Max. array size with be ~75TB holding ~1 millions files. The only plugins I intend to use are Bonienl's Dynamix series, Unassigned Devices and Pre-Clear. Most of my research on this topic brought up hits of Pre-V6 UnRaid, so I apologize if I'm being repetitive. My server came with 2GB of ECC RAM and is capable of holding 16GB. With ECC RAM at ~$8/GB, the less I can get away with the better. [*]Is 2GB of ECC RAM enough for my simple NAS duties, including my plugins listed above? [*]If 2GB of ECC RAM is "enough," what does additional RAM provide? [*]Does dual channel memory provide an actual, tangible benefit? [*]Do I need an app drive to use the plugins I have listed? Thanks for taking the time to read my post.
February 3, 201610 yr WORM = Write Once Read Many - Optical, but you don't mean that, do you? I ran my v5 unRAID server with 2GB and it was fine. Based on the usage I've seen on my v6 server you could probably get away with 2GB for basic NAS duties but I'd recommend 4GB. Dual channel memory is always a benefit but it isn't a strict requirement. You need an app drive for Dockers but not for Plugins, unless the plugin itself has a data storage requirement (like Plex or Sab).
February 4, 201610 yr Author I mentioned WORM just to emphasize that my data will be mostly static. Thinking about 75TB of optical media gives me nightmares. Thanks for the help. DDR3 RAM seems to be near historic lows so it's probably a good time to max out my server anyways. Need to ponder...
February 4, 201610 yr I am running in single channel mode with a single 8GB ECC stick. By no means is it impacting any of my array operations or anything else. In theory it might impact my plex transcoding, but that is CPU dependent for the most part, and maybe even array I/O. Inexpensive or not I went for the largest single stick I was willing to pay for to leave more room for expansion later. Even though I could have easily afforded 4x8gb I didn't waste my money because even plex isn't that much of a memory hog. Oh and as to cachedir ... I wouldn't chase supporting that with massive amounts of memory too hard. you will be better served by using a smartly configured folder structure along with cachedir tuning to get the responsiveness you need for most situations. Being able to cache down 10+ levels of directory and or your 1,000,000th file seems a fools errand. tdallen is on the money with the appdrive. Its also very easy to add later on if you decide to expand your use case.
February 4, 201610 yr Author Oh and as to cachedir ... I wouldn't chase supporting that with massive amounts of memory too hard. you will be better served by using a smartly configured folder structure along with cachedir tuning to get the responsiveness you need for most situations. Being able to cache down 10+ levels of directory and or your 1,000,000th file seems a fools errand. My server will primarily store media (i.e. movies, tv shows, music), so a large portion of those million files would be in the music share. I was thinking about assigning my music share to a single drive and then setting cache_dirs to only scan the movies/tv shows shares. Browsing the movies/tv shows library would cause one disk to spin-up on playback, whereas browsing the music library would cause one disk to spin up immediately. Do you think that is a reasonable strategy?
February 4, 201610 yr Do you think it depends on disk number or total bandwidth? In other words, would 1x 8TB drive see the same benefit as 2x 4TB drives? Dual channel seem like it would help my anemic CPU deal with the I/O of parity operations, but you're the first evidence I've seen. If you remember from the other thread, I'm the guy hoping to jam 16TB drives into an N40L. Going to reply here as not to go off topic, HP with up to 6 disks there should be no difference, when I tested on mine it was single channel and the bottleneck was the A-Link, with 8 disks it was cpu limited, so it could help, I’ll make some tests in the future when opportune.
February 4, 201610 yr Oh and as to cachedir ... I wouldn't chase supporting that with massive amounts of memory too hard. you will be better served by using a smartly configured folder structure along with cachedir tuning to get the responsiveness you need for most situations. Being able to cache down 10+ levels of directory and or your 1,000,000th file seems a fools errand. My server will primarily store media (i.e. movies, tv shows, music), so a large portion of those million files would be in the music share. I was thinking about assigning my music share to a single drive and then setting cache_dirs to only scan the movies/tv shows shares. Browsing the movies/tv shows library would cause one disk to spin-up on playback, whereas browsing the music library would cause one disk to spin up immediately. Do you think that is a reasonable strategy? Seems like a very reasonable strategy. You might look further into the cachedirs auto depth feature to see if it might make caching music still tolerable. Depending on how you have the music directories set up, it might be doable and might help to avoid spinup if you are just doing some shallow searching [shrug]. But I did in fact exclude music from cache dirs. FYI, you can do that without limiting to a single disk. You just tell cachedirs to exclude the music share.
February 4, 201610 yr Author Going to reply here as not to go off topic, HP with up to 6 disks there should be no difference, when I tested on mine it was single channel and the bottleneck was the A-Link, with 8 disks it was cpu limited, so it could help, I’ll make some tests in the future when opportune. I'm really looking forward to your results. An interesting test might be using your 1430SA and splitting 6 disks evenly between the motherboard and controller card for ~250MB/s of potential bandwidth per port. Garycase's suggestions might also be useful.
February 4, 201610 yr Author Seems like a very reasonable strategy. You might look further into the cachedirs auto depth feature to see if it might make caching music still tolerable. Depending on how you have the music directories set up, it might be doable and might help to avoid spinup if you are just doing some shallow searching [shrug]. But I did in fact exclude music from cache dirs. FYI, you can do that without limiting to a single disk. You just tell cachedirs to exclude the music share. That makes sense. My idea of limiting the music share to one disk was to prevent more from spinning up every time I enter the music library. I'll play around with settings as you suggest and see what works best.
February 4, 201610 yr Yeah your plan still makes sense in that regard. Come to think of it, I think I did the same thing with my music in addition to exlucding it from cachedir. though that exclusion was also because I frankly didn't need or care about having my music cached.
February 7, 201610 yr Since I had some time and was curious myself, here are the results, N54L parity check speed single vs dual channel, because that’s what I had available I used a SAS2LP as the additional controller. O=Onboard, S=SAS2LP Single - Dual 4 x O: 187.2 - 184.0 (*) 4 x S: 213.4 - 278.4 3 x S + 3 x O: 169.4 – 186.1 6 x S: 162.5 – 190.6 4 x S + 4 x O: 148.2 - 168.5 8 x S: 144.9 - 180.9 8 x S + 4 x O: 114.8 – 138.0 Using the 4 onboard ports only is limited by the A-link bus, but in every other configuration there’s a nice improvement with dual channel. (*) Kind of strange, although speed is very close, but ran this test twice and got exactly same result.
February 7, 201610 yr Very interesting results. Q: Are those initial speeds or full check speeds? I ask because I peak at just over 120mb/s and average over the whole check at 70mb/s So it makes me wonder if I'd see any improvement if I'm probably not saturating my on-board bus with my small drives EDIT: looks like my last partiy check was actually 87mb/s average ... but still ...
February 7, 201610 yr Very interesting results. Q: Are those initial speeds or full check speeds? I ask because I peak at just over 120mb/s and average over the whole check at 70mb/s So it makes me wonder if I'd see any improvement if I'm probably not saturating my on-board bus with my small drives Full check speeds with SSDs.
February 7, 201610 yr ohhhhhh hahaha yeah now you tell me (or now I noticed) ... I don't know that any / many spinners will see the same results. But that is indeed very interesting that you do see a clear difference with an array of SSDs
February 8, 201610 yr Author Wow, great work. Parity check speeds definitely look like they're CPU constrained. Too bad my N40L has about 65% of the processing power of the N54L, or ~750MB/s of total bandwidth if performance scales linearly. Looks like I'm stuck at A-Link speeds. Again, thanks for your help.
Archived
This topic is now archived and is closed to further replies.