1812 Posted February 14, 2017 Share Posted February 14, 2017 Ok, this may be dumb, but I have a use case that this would be really effective on. Currently I pass trough 2 unassigned 10k drives to a vm as scratch disks for audio/video editing. In the vm, they are then setup as Raid 0. Super fast. The problem becomes that the drives are then bound to that VM. I can't use the disks for any other vm nor span a disk image (work areas) for separate vm's on that pool. I think it would be better to have the host (unRaid) manage the raid, and then mount the "network" disks and use them that way. Since the vm uses paravirtualized 10GBE adapters, performance should be no issue. And multiple vm's could access them as well. Why don't I just add more drives to my current cache pool? Separation. I don't want the dockers that are running, or the mover, or anything else to interfere with performance. Plus, i'm not sure how mixing SSD's and spinners would work out. Maybe ok? I'm sure someone has done that. TLDR: Essentially I'm suggesting that we be able to have more than one pool of drives in a specifiable raid setup (0 and 10! please!) 1 7 Quote Link to comment
TSM Posted February 16, 2017 Share Posted February 16, 2017 I was thinking about something similar the other day. Except I was thinking about the unassigned devices plugin. I honestly don't now how difficult it would be, but it seems that it should be possible without too much trouble to add support for creating a software raid array in unassigned devices. Unless I'm misunderstanding your post, this would provide a similar outcome wouldn't it? Quote Link to comment
SSD Posted February 16, 2017 Share Posted February 16, 2017 I was thinking about something similar the other day. Except I was thinking about the unassigned devices plugin. I honestly don't now how difficult it would be, but it seems that it should be possible without too much trouble to add support for creating a software raid array in unassigned devices. Unless I'm misunderstanding your post, this would provide a similar outcome wouldn't it? I've asked this before. UnRaid is built on top of (or maybe more accurately hijacks) the software RAID feature of Linux. Decoupling is not going to happen. It IS possible to have a RAID set in your server, it just has to be a hardware RAID. I have a RAID0 of 2 4T drives as my parity. Quote Link to comment
JorgeB Posted February 16, 2017 Share Posted February 16, 2017 It's possible to use a btrfs pool using the unassigned devices plugin, it won't look pretty since it's not officially supported (ie, only one device shows as mounted) but it works with any raid profile: btrfs fi show /mnt/disks/Test Label: none uuid: 75c7d7f5-74e4-4662-b465-c400b7384a6c Total devices 2 FS bytes used 1.75GiB devid 1 size 232.89GiB used 2.03GiB path /dev/sdf1 devid 3 size 232.89GiB used 2.03GiB path /dev/sde1 btrfs fi df /mnt/disks/Test Data, RAID0: total=2.00GiB, used=1.75GiB System, RAID1: total=32.00MiB, used=16.00KiB Metadata, RAID1: total=1.00GiB, used=2.02MiB GlobalReserve, single: total=16.00MiB, used=0.00B ETA: You'll need to manually create the pool first (or use the cache slots). Quote Link to comment
SSD Posted February 16, 2017 Share Posted February 16, 2017 It's possible to use a btrfs pool using the unassigned devices plugin, it won't look pretty since it's not officially supported (ie, only one device shows as mounted) but it works with any raid profile: btrfs fi show /mnt/disks/Test Label: none uuid: 75c7d7f5-74e4-4662-b465-c400b7384a6c Total devices 2 FS bytes used 1.75GiB devid 1 size 232.89GiB used 2.03GiB path /dev/sdf1 devid 3 size 232.89GiB used 2.03GiB path /dev/sde1 btrfs fi df /mnt/disks/Test Data, RAID0: total=2.00GiB, used=1.75GiB System, RAID1: total=32.00MiB, used=16.00KiB Metadata, RAID1: total=1.00GiB, used=2.02MiB GlobalReserve, single: total=16.00MiB, used=0.00B ETA: You'll need to manually create the pool first (or use the cache slots). Would this allow the btrfs RAID set to appear as a drive and be used as parity? I think not but you've got me curious. Quote Link to comment
JorgeB Posted February 16, 2017 Share Posted February 16, 2017 Would this allow the btrfs RAID set to appear as a drive and be used as parity? I think not but you've got me curious. No, this would only work to have VMs/Dockers, etc. Quote Link to comment
trurl Posted February 16, 2017 Share Posted February 16, 2017 Currently I pass trough 2 unassigned 10k drives to a vm as scratch disks for audio/video editing... How large are these? Might be simpler just to replace with SSDs and add them to your regular cache pool. Sometimes it's better in the long run to just spend money. I used a 10K drive as cache (because I already had it) when I built my first unRAID but decided it was too noisy and quickly replaced it with SSD. That was back on unRAID 4.7 Quote Link to comment
1812 Posted February 16, 2017 Author Share Posted February 16, 2017 Currently I pass trough 2 unassigned 10k drives to a vm as scratch disks for audio/video editing... How large are these? Might be simpler just to replace with SSDs and add them to your regular cache pool. Sometimes it's better in the long run to just spend money. I used a 10K drive as cache (because I already had it) when I built my first unRAID but decided it was too noisy and quickly replaced it with SSD. That was back on unRAID 4.7 300GB each. Had them already. No sense in upgrading to ssd's for them because my server backplane only supports 3gbps sata (6 on sas). To add ssd's to the cache pool, I'd have to convert the existing 2 disk pool to raid 10 for better performance while maintaining redundancy, which means not just adding 2 disks for capacity but instead buying 4 new larger SSD's.... and then still sharing i/o with apps/dockers running. All my servers are in my basement, so noise is not a factor. Plus you never hear the 10k drives over the fan noise. I think I use unRaid a little different than some. Maybe unassigned devices could have a raid option that would allow it to be more easily setup.... Quote Link to comment
hermy65 Posted February 22, 2017 Share Posted February 22, 2017 I would be very interested in having multiple cache pools. I could offload vms/docker to one and leave the other for everything else which would allow me to have 2 smaller ssd pools rather than needing 1 large ssd pool Quote Link to comment
trurl Posted February 22, 2017 Share Posted February 22, 2017 2 hours ago, hermy65 said: I would be very interested in having multiple cache pools. I could offload vms/docker to one and leave the other for everything else which would allow me to have 2 smaller ssd pools rather than needing 1 large ssd pool You can make a large pool out of small SSDs. Quote Link to comment
hermy65 Posted February 22, 2017 Share Posted February 22, 2017 13 minutes ago, trurl said: You can make a large pool out of small SSDs. Yep, im more so looking to split it out though so i can have better quality ssd's running my vms/docker containers and regular ssds handling tasks like downloads, etc Quote Link to comment
1812 Posted February 22, 2017 Author Share Posted February 22, 2017 2 hours ago, hermy65 said: Yep, im more so looking to split it out though so i can have better quality ssd's running my vms/docker containers and regular ssds handling tasks like downloads, etc This post ^^^^^^ I really don't want to put 2-4 10k 300GB scratch drives, 2 Samsung Evo SSD's, 2 Kingston entry level SSD's, and 1 patriot blast all in the same pool.... that just feels dirty.... And means that the 10k drives are spun up every time anything accesses the pool. Quote Link to comment
TOoSmOotH Posted February 28, 2017 Share Posted February 28, 2017 +1 to this. Multiple cache pools is something that would be really useful. My use case is I want to have separate SSD based cache for VMs/Dockers vs a cache pool for front ending network shares which would be based on spinning disk with more capacity. Even better would be able to choose which cache pool for certain mounts in a docker. Example: the Plex docker image resides on the spinning disk cache which is mirrored and /transcode filesystem being located on a SSD based pool that is set to raid 0. Quote Link to comment
HellDiverUK Posted March 7, 2017 Share Posted March 7, 2017 The fastest 10k drives, even in RAID0 are slower than even the crappiest SSD. Throw those two things out and put in a single SSD like the 480GB Sandisk UltraII. unRAID now also supports NVMe, so you could go with something like an Intel 600p or ADATA SX3000 and stick it in a 4x PCI-E slot on an adaptor board. This will give you lightning quick storage, like 2GB/s or more in the case of the ADATA. Quote Link to comment
1812 Posted March 7, 2017 Author Share Posted March 7, 2017 (edited) 2 hours ago, HellDiverUK said: The fastest 10k drives, even in RAID0 are slower than even the crappiest SSD. Throw those two things out and put in a single SSD like the 480GB Sandisk UltraII. unRAID now also supports NVMe, so you could go with something like an Intel 600p or ADATA SX3000 and stick it in a 4x PCI-E slot on an adaptor board. This will give you lightning quick storage, like 2GB/s or more in the case of the ADATA. your assumption is that i (and everyone else) have/has more pci-e slots available, which is not the case. Also, your assertion that the fastest 10k drives in raid0 are slower than the crappiest ssd are incorrect. My kington SSD drives in this very same server read at 250+MB and write starting at 180MB and then drop off as the file write progresses. that's what 20 bucks get you for a cheap ssd. The 10k drives are 300+MB each way in raid 0. Regardless of those 2 facts, your comments do not diminish the initial request, which is to take 2 or more drives and make an independent cache pool for usage. I would love in the future to replace the 2 10k drives with quality ssd's and hit 1GBps or more depending on configuration. the premise of unRaid is working with what you've got. various drives, etc... and this request is in the spirit of that. but building further on comments above, here is another scenario that would benefit from a separate cache pool: I've just finished working on a project on the vm desktop I have setup. I copy the 200GB file to the array which goes to the cache drive first to be later moved at night. Using the para-virutlaized 10gbe adapter, it attempts to max out the 500MBps write speed of the quality ssd drive but gets choked up because that drive is also being used as a downloader ingest and plex transcode directory. So now my write speeds take a hit. Additionally, if unRaid wants small business attention, the ability to setup a separate cache pools could be beneficial. Imagine if 2 video editors were both trying to write to that single ssd... you're wasting company time and company money as they fight for i/o. You could give each their own and have no cross over issues. It's the same situation as multiple vm's and dockers sharing disk i/o in moderate to higher usage environments. Edited March 7, 2017 by 1812 1 Quote Link to comment
HellDiverUK Posted March 7, 2017 Share Posted March 7, 2017 Seek time is more important for a scratch drive, and no matter what you think, RAID0 of 10k drives will be significantly slower than a SSD. Been there, done that, got the t-shirt, the mug and the ticket stubs. I don't know why you're talking about $20 drives, they're obviously something from years ago, probably Sandforce era, which are nothing like what a budget SSD of the last year or so can do. Quote Link to comment
1812 Posted March 7, 2017 Author Share Posted March 7, 2017 ssd drives were purchased 8 months ago.. so, not that old. I'm talking about them because you said 3 hours ago, HellDiverUK said: The fastest 10k drives, even in RAID0 are slower than even the crappiest SSD. and that is not true. source: actual usage numbers presented previously of the crappiest SSD which I happen own and use (in addition to some nicer ones.) You then try to change your assertion by adding an excluding argument via the Sandforce comment, but your "crappiest SSD" claim remains disproven. I've even provided the read time on the raid0 10k drives vs the cheap ssd. If you skipped that part, the summary was that it was better. Your argument based on cheap ssd's is invalid and incorrect. This isn't an opinion, it isn't what i "think." These are actual numbers provided from real case usage scenarios and benchmarks I've done on the hardware. You also fail to address the i/o limitation based on actual use scenarios as outlined above. Probably because either it doesn't effect your usage scenario, or you realized cost becomes a factor quickly to try and mitigate that bottleneck. But to your assertion, let's take it to the logical conclusion which is using more and larger SSD's in the current pool configuration. For my needs and wants, and those of several others, I could go purchase four 2 tb quality ssd's for $2400 USD, put them in raid 10, and they'd give me 4 tb redundant with over 1GBPS read/write, and a decent amount of bandwidth since it's my only option because I have no more pci-e slots to use. That sounds like a great thing to do, you know, spend more on 4 drives than my entire server cluster cost to assemble (including the rack and double APC battery backup system.) Why 2 tb drives? I have a few vm's that run at the same time and I need a few persistent scratch disks to go with them. As also described, my file sizes are quite large and require more space on the cache drive than what would typically be available when also hosting vm's and docker downloads. So a couple 120gb drives won't cut it. From a cost factor, I could buy 8 wd red 2 tb spinners for the price of 1 good 2 tb ssd. In raid 10 it they would reach or exceed the single ssd drive read/write speeds AND provide data redundancy not found on the single ssd. not to mention that i'd have 4x the storage over the single drive, and 2x the storage over the 2400 dollar raid 10 ssd pool while saving 3/4 the total price. To use only cheap drives, with an average cost of 50-75 dollars, I would need a pool of at last thirty-three 120gb drives to get near 2 tb with no redundancy. That's 1,650-2,475 usd. I currently only have space for 23 drives on my main server so that's not doable. And also, that's ridiculous. I'm not Linus, and I don't get storinators or boxes of SSD's for free. ¯\_(ツ)_/¯ 1 Quote Link to comment
TOoSmOotH Posted March 7, 2017 Share Posted March 7, 2017 Debates over drive speeds are irrelevant to the topic here. Just looking for multiple cache pools. 1 2 Quote Link to comment
DoeBoye Posted March 7, 2017 Share Posted March 7, 2017 4 hours ago, 1812 said: I copy the 200GB file to the array which goes to the cache drive first to be later moved at night. Using the para-virutlaized 10gbe adapter, it attempts to max out the 500MBps write speed of the quality ssd drive but gets choked up because that drive is also being used as a downloader ingest and plex transcode directory. So now my write speeds take a hit. +1 I ran into this exact issue a few weeks ago. VMs were in the cache pool, and I was finding sustained disk transfers were seriously impacting performance of the VMs themselves. I ended up pulling them out of the cache pool, and installing them on separate SSD in Unassigned Devices. Now sustained I/Os have no impact on my VMs. It's a solution, and works well, but multiple cache pools would make it a cleaner, more streamlined setup. Quote Link to comment
scytherbladez Posted March 8, 2017 Share Posted March 8, 2017 15 hours ago, DoeBoye said: I ran into this exact issue a few weeks ago. VMs were in the cache pool, and I was finding sustained disk transfers were seriously impacting performance of the VMs themselves. I ended up pulling them out of the cache pool, and installing them on separate SSD in Unassigned Devices. Now sustained I/Os have no impact on my VMs. I have also moved my VM's off the SSD cache to an SSD in Unassigned Devices. I hope it's something unRaid are working on, given VM's are a promoted functionality of the software. Quote Link to comment
me.so.bad Posted March 24, 2017 Share Posted March 24, 2017 +1 for multiple pools - that would add A LOT of flexibility for a ton of usecases. no sure if multiple arrays are sane in any matter, but multiple cache pools would be very, very awesome. Quote Link to comment
1812 Posted March 26, 2017 Author Share Posted March 26, 2017 Realized another reason this would be a nice feature- When your cache drive fills up from moving moving files to the server or downloading, it pauses your VM's. SO, there's that too.... Quote Link to comment
JonathanM Posted March 27, 2017 Share Posted March 27, 2017 24 minutes ago, 1812 said: When your cache drive fills up from moving moving files to the server or downloading, it pauses your VM's. Respectfully, that's what the min free space setting is for. As long as you use a share that is set to cache:prefer for stuff that you want to live on the cache drive and cache:yes for stuff that will be moved to the array, and have a proper setting for min free space, you will never run into this specific issue. 2 Quote Link to comment
1812 Posted March 27, 2017 Author Share Posted March 27, 2017 12 minutes ago, jonathanm said: Respectfully, that's what the min free space setting is for. As long as you use a share that is set to cache:prefer for stuff that you want to live on the cache drive and cache:yes for stuff that will be moved to the array, and have a proper setting for min free space, you will never run into this specific issue. Such a civil response! I'd like to reply but I can't find my monocle. Wait, there it is. ಠ_ರೃ You are correct, but I'm lazy and don't want to adjust each of my 40+ shares to guesstimate the max file size ever each will see. And I'm not willing to clone the 40GB max size for mkv files to each one because then I have a 40GB hole in my SSD that I never get to use while waiting for the scheduled move. And that is the point of having the ssd cache, to keep from spinning up the disks unnecessarily and/or getting faster network transfer speeds. ( I did make a joke about the "civil response," but i'm messing around, and it's been a looooong day. I'm not mocking you. I just can't bring myself to erase the ascii monocle guy at the point in the evening. ) Quote Link to comment
ryoko227 Posted May 20, 2017 Share Posted May 20, 2017 (edited) +1 for multiple cache pools. For my specific use case, I have my VMs currently on my nvme drive which is loaded up via the go file at the moment. The cache pool is being utilized by my btrfs raid0 ssd pool. I like that UD "sees" my nvme, but I don't like not having temps on it, and TBH (while I already have it setup) having to add things in a round about way feels.... "dirty" I believe is the word someone else used, lol. Multiple cache pools please! Edited May 20, 2017 by ryoko227 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.