manikyath Posted February 26, 2022 Share Posted February 26, 2022 i've been on quite a search to figure out if i can make a VM in unraid that dynamically gets memory assigned when it needs it, or when the host doesnt need it. Either way will work really. In short, my goal is to consolidate some tasks from another server into a VM on my unraid machine. Because of some long-winded requirements i cant use a docker container, and i dont have enough RAM in my server to dedicate the amount the VM 'could' use, while still having enough memory spare for what other services on my box 'could' use. Having a shared 'pool' is fine however. For solving this problem, i got interested in the ways behind the "Initial Memory" and "Max Memory" settings in the VM settings. However, the behavior doesnt seem to make much sense in my testing. My current setup is a host machine with 32GB RAM, running unraid 6.9.2 The virtual machine is running debian 11.2, configured with 2GB Initial memory and 16GB Max memory. Other services currently running are consuming approximately The behavior i'm experiencing here is as follows: - once the VM starts, it shows to have 16GB memory allocated - as the system has started up, the memory allocation depicted by unraid, and in the VM as listed by htop, lowers down to around the initial memory configruation: 2GB in unraid, and about 1.6GB in debian (i'm assuming driver losses here.) - when i run a memory intensive application in the VM, it'll start eating into swapfile, and eventually crashes with an out of memory error, the VM never gets any additional memory past what it dropped down to after startup. - i can manually assign the VM more memory in the 2 to 16GB memory range trough terminal commands on unraid, but past the initial shrink it doesnt appear to take any automatic action in either direction. Is this how it is intended to work with unraid? Is there supposed to be any sort of automated process that allocates memory as needed, and it just appears broken for my exact setup? i've dug trough every bit of documentation i could find on the topic, and have come up short unfortunately. Quote Link to comment
tomd Posted March 7, 2022 Share Posted March 7, 2022 I am running into the exact same situation that you have discribed. I have installed about 10 different linux distros ranging from lightweight AntiX to fatty Ubuntu, Arch based, and Red Hat based... They all do what you have outlined. They are supposed to use virt-io ballooning to dynamically grow and shrink between what is specified in the VM config but they never grow: Virtio balloon device: Spoiler VM config: Spoiler Htop: Spoiler While troubleshooting, I decided to disable the virtio memory ballon PCI device to see what would happen. Remove PCI device: Spoiler Htop: Spoiler The bare metal memory usage didn't spike after removing it so it seemed promising. To continue testing, I opened a crap-ton of tabs and saw the memory usage was able to surpass the initial 2.5G, which was good: Spoiler The issue I ran into was that when I closed the browser, memory usage in the VM decreased, but the host never reclaimed any of it. Before VM start: Spoiler VM running and virtio balloon device removed: Spoiler After opening a lot of tabs then closing them: Spoiler So, in short... yeah, it's broken. no idea why, but it seems to be specific to unraid as the same VM config in proxmox grows and reclaims memory properly: Spoiler I'm new to unraid but, IMO, unraid's hypervisor is subpar... 2 Quote Link to comment
Crad Posted May 2, 2022 Share Posted May 2, 2022 Did you ever find a solution for this? Quote Link to comment
manikyath Posted May 2, 2022 Author Share Posted May 2, 2022 6 hours ago, Conrad Allan said: Did you ever find a solution for this? so far the only solution i've found is to switch to proxmox, which is a really disappointing to say on the unraid forums. Quote Link to comment
unifiedmamba Posted January 5, 2023 Share Posted January 5, 2023 (edited) Has anything even changed here? It seems like this topic isn't even recognized by the developers of unraid. It's a feature you pay for and it's definitely not working correctly, so something should be done about this. I am having the same issue and was not able to find any fix for this. Since Unraid just uses KVM and qemu which are standard, could this just be a configuration issue of one of these services? Edited January 5, 2023 by unifiedmamba Quote Link to comment
manikyath Posted January 5, 2023 Author Share Posted January 5, 2023 1 minute ago, unifiedmamba said: Has anything even changed here? It seems like this topic isn't even recognized by the developers of unraid. It's a feature you pay for and it's definitely not working correct, so something should be done about this. I've got the same issue and was not able to find any fix for this. Since Unraid just uses KVM and qemu which are standard, could this just be a configuration issue of one of these services? no change on the matter, but I also havent chased it any more. for all i know the latest release of unraid has magically resolved the issue. i've figured out (almost a year ago now) that dynamic memory is a bit weird in KVM, and their own documentation sort of implies that it's not supposed to be used this way. and that dynamic memory also doesnt work on a VM with IOMMU features. i'm assuming that the issue is one of the following: - unraid might accidentially set some flags on the IOMMU side that mess with dynamic memory. - other platforms that focus a lot more on the 'VM farm' aspect of things have just put in the R&D of making it work. given the traction this thread is getting, i think it's safe to say "there's half a douzen of us" that have encountered this issue. i've since concluded that it's better to chase my 'it must be in a VM' limitation, than to chase this oddity with dynamic memory. Quote Link to comment
unifiedmamba Posted January 6, 2023 Share Posted January 6, 2023 19 hours ago, manikyath said: for all i know the latest release of unraid has magically resolved the issue. Sadly not for me. Just tried yesterday. The documentation of unraid doesn't help either. It just mentions the feature casually but I can't find more information about it. Maybe someone "official" could help out here, it could be us using or understanding the feature wrong . Quote Link to comment
unifiedmamba Posted January 18, 2023 Share Posted January 18, 2023 Nice, that's what I call support. I'll write an email to the official support team listed on the website and post the result here. Quote Link to comment
HappyBobJr Posted January 29, 2023 Share Posted January 29, 2023 (edited) Edited January 29, 2023 by HappyBobJr 1 Quote Link to comment
unifiedmamba Posted February 12, 2023 Share Posted February 12, 2023 AndrewZ replied to my request: Quote Hey there, Unraid simply passes the appropriate flags to the KVM system to support memory ballooning. This feature is pretty much limited to Linux VMs and is something that is a very advanced feature, and you most likely require the virtio guest tools installed on the VM. Googling how this gets handled (not using search terms that are unraid specific) is your best bet. 99.99% of our users (and ourselves) have no need / use case for this feature, and your best place for support on KVM and memory ballooning on Debian is on the greater internet. Andrew ... Quote Link to comment
manikyath Posted February 12, 2023 Author Share Posted February 12, 2023 17 minutes ago, unifiedmamba said: AndrewZ replied to my request: ... by that i assume that this previous assumption was correct, then: On 1/5/2023 at 3:40 PM, manikyath said: other platforms that focus a lot more on the 'VM farm' aspect of things have just put in the R&D of making it work. Quote Link to comment
JorgeB Posted March 12, 2023 Share Posted March 12, 2023 I've been doing some testing and ballooning is working with Unraid and Windows, this is a VM set to 4GB minimum and 16GB max, before installing the balloon driver and service: Same VM after installing the ballon driver and service: Note that the used RAM reported by Windows is not correct, this is normal, see this post: https://github.com/virtio-win/kvm-guest-drivers-windows/issues/568 Couple of notes: - Install both the balloon driver and service, though the driver seems enough for this to work the service reportedly helps with performance, you can install everything by running virtio-win-gt-x64.exe from the virtio ISO root. - memory ballooning won't work if any PCI(e) device is being passed-through to that VM, this is because all the RAM must be allocated for possible DMA use. Quote Link to comment
manikyath Posted March 12, 2023 Author Share Posted March 12, 2023 1 minute ago, JorgeB said: I've been doing some testing and ballooning is working with Unraid and Windows, this is a VM set to 4GB minimum and 16GB max, before installing the balloon driver and service: Same VM after installing the ballon driver and service: Note that the used RAM reported by Windows is not correct, this is normal, see this post: https://github.com/virtio-win/kvm-guest-drivers-windows/issues/568 Couple of notes: - Install both the balloon driver and service, though the driver seems enough for this to work the service reportedly helps with performance, you can install everything by running virtio-win-gt-x64.exe from the virtio ISO root. - memory ballooning won't work if any PCI(e) device is being passed-through to that VM, this is because all the RAM must be allocated for possible DMA use. the "question" in the thread is how to make it re-release RAM to the virtual machine automagically, and if that's possible at all on unraid. i know the balloon driver is needed, i know it doesnt work if PCIe devices are passed trough, the behavior you describe is how far i got as well, but after that the RAM doesnt come back to the VM on demand, like it does with for example proxmox. Quote Link to comment
JorgeB Posted March 12, 2023 Share Posted March 12, 2023 7 hours ago, manikyath said: the "question" in the thread is how to make it re-release RAM to the virtual machine automagically Yes, and I'm sorry, I was though I was replying to a "ballooning doesn't work" thread, and that's not what this is about, I wasn't sure myself but from what I can find KVM memory balloon is not dynamic: "You don't mention which hypervisor you're using, so I'm going to guess KVM, but my answer applies to Xen too. When you set currentMemory=16384 and maxMemory=32768, you are telling KVM to boot with 32 GB of RAM and use balloon driver to reduce that to 16 GB.Once the balloon driver has reduced the RAM to 16 GB, it will never be increased again unless you explicitly tell KVM to increase. It won't adjust itself without you telling it to." And at the very top of the link I posted above: "EDIT: For future reference, virtio memory ballooning is not dynamic by default. Meaning, if you want the guest to request more memory from the hypervisor based on something like memory pressure, that would be up to you to set up (a script talking to a virto channel should work)." So I believe, and if I'm wrong someone please correct me, that it will never automatically add or remove allocated RAM to a VM, you can do it manually or with some sort of script, for example, say I want my Windows VM to usually use 2GB of RAM, but sometimes I need to run some app that requires 16GB, I set the VM to 2GB/16GB, install the balloon driver and service and after boot the VM will only use 2GB, the rest will be released to the host, when I need the extra RAM type in the Unraid console: virsh setmem "Windows 10" --size 16777216 Of course the host needs to have enough free RAM at that time, when the VM no longer requires the extra RAM type: virsh setmem "Windows 10" --size 2097152 And the VM goes back to using 2GB of RAM, releasing the rest to the host, basically this as implemented allows you to add or remove allocated RAM to a VM on the fly and with the VM running, since KVM does not support dynamically changing this I guess it would be possible for Unraid to change it based on the RAM pressure of the host/VM, but no idea if this is even possible to implement for all guest OSes, I guess users can make a feature request and if there is enough interest and it's feasible LT might do it in the future. Quote Link to comment
manikyath Posted March 13, 2023 Author Share Posted March 13, 2023 16 hours ago, JorgeB said: So I believe, and if I'm wrong someone please correct me, that it will never automatically add or remove allocated RAM to a VM, you can do it manually or with some sort of script, for example, say I want my Windows VM to usually use 2GB of RAM, but sometimes I need to run some app that requires 16GB, I set the VM to 2GB/16GB, install the balloon driver and service and after boot the VM will only use 2GB, the rest will be released to the host, when I need the extra RAM type in the Unraid console: this is the assumption i came to so far, and the platfoms where it "works" have just implemented some sort of in-house script to automate the process. Quote Link to comment
zaberchann Posted October 26, 2023 Share Posted October 26, 2023 (edited) On 3/13/2023 at 2:59 AM, JorgeB said: Yes, and I'm sorry, I was though I was replying to a "ballooning doesn't work" thread, and that's not what this is about, I wasn't sure myself but from what I can find KVM memory balloon is not dynamic: "You don't mention which hypervisor you're using, so I'm going to guess KVM, but my answer applies to Xen too. When you set currentMemory=16384 and maxMemory=32768, you are telling KVM to boot with 32 GB of RAM and use balloon driver to reduce that to 16 GB.Once the balloon driver has reduced the RAM to 16 GB, it will never be increased again unless you explicitly tell KVM to increase. It won't adjust itself without you telling it to." And at the very top of the link I posted above: "EDIT: For future reference, virtio memory ballooning is not dynamic by default. Meaning, if you want the guest to request more memory from the hypervisor based on something like memory pressure, that would be up to you to set up (a script talking to a virto channel should work)." So I believe, and if I'm wrong someone please correct me, that it will never automatically add or remove allocated RAM to a VM, you can do it manually or with some sort of script, for example, say I want my Windows VM to usually use 2GB of RAM, but sometimes I need to run some app that requires 16GB, I set the VM to 2GB/16GB, install the balloon driver and service and after boot the VM will only use 2GB, the rest will be released to the host, when I need the extra RAM type in the Unraid console: virsh setmem "Windows 10" --size 16777216 Of course the host needs to have enough free RAM at that time, when the VM no longer requires the extra RAM type: virsh setmem "Windows 10" --size 2097152 And the VM goes back to using 2GB of RAM, releasing the rest to the host, basically this as implemented allows you to add or remove allocated RAM to a VM on the fly and with the VM running, since KVM does not support dynamically changing this I guess it would be possible for Unraid to change it based on the RAM pressure of the host/VM, but no idea if this is even possible to implement for all guest OSes, I guess users can make a feature request and if there is enough interest and it's feasible LT might do it in the future. I agree with your point after delving into the details about kvm balloon. Currently, no official support for dynamic RAM. On Windows VMs, they do support automatic increase of the RAM. On Linux VMs, they support neither automatic increase nor decrease. Edited October 26, 2023 by zaberchann Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.