Shobo Posted May 3, 2020 Posted May 3, 2020 (edited) I have a 500GB NVMe set up for my cache drive (btrfs) and I'm experiencing some extremely poor performance. I first noticed it when I set up a development share on the cache drive for a project. I have a Linux VM setup (on a separate SSD mounted with unassigned devices) that mounted the share and the performance on both serving a website and compiling node.js was abysmal. For testing purposes I copied the data from the share to the VM's local filesystem and the compiling performance was approximately 20x faster. Night and day. I've replaced the NVMe drive to see if it had something to do with faulty hardware but that was not the case. Now I'm still very much a beginner when it comes to Unraid - so it's very possible I just have something configured incorrectly or I'm just going about this all wrong. Any help would be greatly appreciated. Edited May 3, 2020 by Shobo Quote
itimpi Posted May 3, 2020 Posted May 3, 2020 What brand of NVME drive? Some of them are much slower than you might think You should attach your system diagnostics zip file (obtained via Tools >> Diagnostics) to your next post if you want any sort of informed feedback. 1 Quote
Shobo Posted May 3, 2020 Author Posted May 3, 2020 (edited) The first one was a Samsung 960 EVO. I replaced it with a SABRENT Rocket 2280 for testing. They shouldn't be upwards of 20x slower than a regular SATA SSD. I've attached the diagnostics file. Thanks again. mori-diagnostics-20200503-1617.zip Edited May 3, 2020 by Shobo Quote
Shobo Posted May 3, 2020 Author Posted May 3, 2020 I've done some additional tests and believe my issue doesn't have anything to do with the cache but instead of mounting shares within my VM. First I tried various other shares and they all ran at approximately the same speed. Then I made a copy of my VM disk and moved it within a share that also had the project (on the array) and also set it to mount that same share. Booted it up and tested compiling from the VM's filesystem - it was snappy as I'd expected. I then tried compiling from mounted share (which again, is the same share the VM is running on) and it was upwards of 20x slower like all previous tests. This makes me believe it isn't the cache at all but something with mounting Unraid shares on my VM that's causing the issue. Quote
Shobo Posted May 4, 2020 Author Posted May 4, 2020 I've attached the output of bonnie++ on both the local filesystem of the VM as well as the mounted share. The differences are quite large so something is definitely up with these mounted shares. Quote
testdasi Posted May 4, 2020 Posted May 4, 2020 15 hours ago, Shobo said: I've done some additional tests and believe my issue doesn't have anything to do with the cache but instead of mounting shares within my VM. What do you mean by "mounting shares"? That field in the VM template where you enter the /mnt/user/share and a mount name and then mount the mountname from within the OS? If so that's a known issue for quite some time. It is theoretically possible to tune that (according to Red Hat) but nobody has ever managed to do it so performance is terrible. You would have better performance mounting smb shares instead. The best performance with NVMe is to create custom smb config to access /mnt/cache/share (to by pass shfs). Alternatively mounting the NVMe with Unassigned Device will also bypass shfs and give you best performance. 1 Quote
Shobo Posted May 4, 2020 Author Posted May 4, 2020 (edited) Well then, that is great to know. I'm quite new to Unraid so wasn't sure there was any way to do this. 1 hour ago, testdasi said: You would have better performance mounting smb shares instead. The best performance with NVMe is to create custom smb config to access /mnt/cache/share (to by pass shfs). Absolutely no idea to do this, but it'll give me a project to read into. 1 hour ago, testdasi said: Alternatively mounting the NVMe with Unassigned Device will also bypass shfs and give you best performance. I can't do this because the NVMe is my cache drive. Thanks a bunch. Edited May 4, 2020 by Shobo Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.