btagomes
-
Posts
25 -
Joined
-
Last visited
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Report Comments posted by btagomes
-
-
6 hours ago, sittingmongoose said:
Which part happened to you before?
The part of not beeing able to update some dockers showing "not avaliable" in the version
-
37 minutes ago, Dava2k7 said:
Hi btagomes
No I haven’t heard of it since now il have alook into it can you tell me where I might find it please thank you 👍🏻
Open a terminal window and type "htop" and have a look at it to see if your cpu's are really at 100%.
-
3 minutes ago, Dava2k7 said:
Hey all
Playing with my VM today I noticed I have a pinned core at 100% which jumps around from core to core now and again whilst I’m using it if I leave the VM and don’t do anything it stays pinned at 100% and then doesn’t jump around is this a GUI bug cause if It was pinned surely it would stay pinned?
Have you checked with htop? Usualy i have that 100% indication jumping around cores but i check with htop and its a false value...
-
5 hours ago, sittingmongoose said:
Seeing some weird things now. Upgraded from beta 25. My server completely crashed, hasn't done that in many months. Not seeing any errors anywhere. Had to pull the power to reboot it.
Then there are some issues with docker. A bunch of the dockers say "not available" under the version.
Andddd NZBget no longer works. But it did before the crash. Attaching diagnostics and Syslog.
syslog.txt 326.7 kB · 1 download 9900k-diagnostics-20201001-0016.zip 276.58 kB · 1 download
That happened with me before and i solved it by recreatig the "proxynet" network (created from spaceinvader's video instructions for reverse proxy)
-
1 hour ago, limetech said:
Will this work with real hdd/ssd devices as well? I was in process of reverting to libvirt v6.5, maybe doing that is still the best course until the issue is officially fixed upstream?
I don't know, i have no issues with hdd/ssd devices. I have sata and nvme ssds and they are working great, same with nvme passtrough, and array disks are working ok too. Only issues i've had are:
-Dvdrom passtrough to guest vm.
-Npt and nrip-save features on guest cpu definition.
-Novnc conection to guest (working fine in vnc trough guacamole).
So far, no more issues, my log files are clean...
-
ok, my issue with devmapper and /dev/sr0 is solved.
Changed the guest xml, instead of passtrough the cdrom as a disk i passed trough as a scsi device and now is working.
(before this upgrade to beta29, if i used the scsi method to passtrough the cdrom i could only read the first disk, if i changed disk the guest would not read the second disk, now on beta 29 this is not happening...)
now i'll try to find a way around the vnc issue...
-
ok...
After a bucket of cofee i tryed to add this lines to the cpu config on guest xml:
<feature policy='require' name='npt'/>
<feature policy='require' name='nrip-save'/>All Vm's booted except the one that had a dvd-rom passtrought, that one i had to disable till i find a work around.
-
-
I still saying, should exist some colour diferenciation in the cpu pinning menu (based on the numa architecture obtained from numactl --hardware) so anyone could understand wich cpus are closer to each memory channel.
I know that it wont be easy to implement but would be a wonderfull feature on the threadripper era.
i'm still in battle with the memory spills caused by data being transfered, and in my case the memory spills create interference noise in my onbord sound card and in my gpu causes micro stutters. It only happens after the memory spill to the other node, and if i drop caches regularly it won't happen, but that would kill the purpouse of the cache and can put my cache data in risk...
-
To better understand the particularities of NUMA assignment this diagram help a lot...
Every time the data travel trough infinity fabric it gets latency.
Inter-CCX latency is tolerable.
Inter-DIE latency is bad.
For de 1950X:
If a VM get 4 cores from 1 CCX it gets 1 memory channel and the best latency you can get from threadripper (69ns) and .
If a VM get 8 cores from 2 CCX in the same DIE it gets 2 memory channels, that means some latency (73ns) but double memory bandwith.
If a VM get more than 8 cores or if a VM have cores from 2 different DIES you will quadruple the memory bandwith but the latency gets terrible (130ns).
If you plan to use cores from 2 diferent DIES in the same VM then you would get better results if you set your memory mode to AUTO in the bios...
That way the system will manage the memory in use so you get bittersweet latency (100ns).
For the "interleave", "strict" and "prefered" modes...
If you select "strict" your VM will only boot after all the memory gets alocated (if you have the ram full than it must dump it before the vm starts) and i'm almost sure that if it cannot get enough memory it wont boot.
The "interleave" mode and the "prefered" mode are almost the same (interleave uses roundrobin)... they try to get memory from designated node but if they cannot get it they will spill to the other node, increasing latency.
Sure the strict mode is nicer but then some times the VM cannot boot...
When you use strict mode some times the machines get paused and wont resume (as they cannot acess all the alocated memory).
-
1
-
-
9 hours ago, xrstokes said:
Wow, I never even considered how the memory channels could play into all of this. Thanks so much for the detailed response. 👍
Well... It depends how you set your numa mode in bios... You can select none/auto/die/socket/channel... I get the best results when i set it to channel, and that means that the each cpu will want to talk with its respective memory channel before talking to the other channels. The 1950x have 16 cores (32 treads) in each socket, 8 cores (16 treads) in each die, 4 cores(8 treads) for each memory channel...
If you set numa to "die" the 2 memory channels from the same die will be handled as one channel... The latency will rise as there are data being transfered trough the infinit fabric to the adjacent memory channel, anyway, this latency wont be as bad as having the cores of one die talking to the memory of the other die...
-
Hi all,
I'm a newbie in the linux world, my first contact with linux was with unraid...
Looking for some virtualization platform i assembled a server with a threadripper 1950X, 4x16gb ram, an Geforce gtx 1080 for main VM (with screen, keyboard and mouse) and two geforce gtx 1050 for secondary VM's with remote access (one is all the time passedtrough to the same VM and the other is frequently passedtrough to any other VM whenever i want to use it).
i've 2 NVME SSDs for cache and 3 HDDs for the array and main VM runs from a passedtrough NVME SSD while the other VM's run from another NVME SSD (unassigned)
After playing around for some days, tweaking my unraid at the same time i was learning about linux architecture, virtualization, networking, and NUMA, my platform started to be more and more stable, every issue was beeing sorted out one by one and today i can say something about threadripper and NUMA.
I've done it every possible way and failed several times... then i started to understand that my ram allocations were spilling in to the wrong node, causing my latency to spike.
Today i've all my dockers waiting to start AFTER all the VM's are started... and that did an huge diference, even when i've CPUs 4-31 isolated.
The memory mode='strict' is tricky because it would only work right if you matched the right cores with the respective memory channel, for example, each numa node have 2 memory channels. Each channel matches to 4 cores (8 treads)...
For that reason, in the 'strict' mode my VM performance could only get right if i set it up to use 4 cores with <16 gb ram or 8 cores with <32 gb ram, if the ram values were exceded the latency would spike.
The GPU placement will only matter if all the cores of your VM are located on the wrong numa node...
If the VM have cores from 2 memory channels in the same node you wil get double the memory bandwith, with allmost the same latency of single channel placement, as long as they stay in the same node.
So... for my main VM i use 8 cores and 16 gb Ram in interleave mode, i get a good latency and good memory bandwith.
For secondary VM's i use 4 cores and 8 gb ram in strict mode, the latency is even better but the memory bandwith would be half, as expected.
I've my server working with S3 sleep (now that i've tweaked it a bit it works fine), waking from keyboard/mouse or wol, but the dockers need to be stoped before sleep so when the server wakes up, the sleep plugin makes them wait to start so the VM's get plenty of memory for their needs.
Another thing i noticed... everytime i transfer large amounts of data between network - cache or cache - array, i get numa spills and the avaliable ram on each node gets cripled.
That makes the latency spike too.
For that reason i tweaked the mover script to dropcaches after the move process ends, that way the ram gets free again and the VM's may use it.
Another weird event is that every time the server wakes from S3 sleep, the cpu clock gets nutz... that causes bad performance on the VM's too.
To solve that problem i added the "echo 1 > /sys/module/processor/parameters/ignore_ppc ; " comand in the S3 postrun script.
Forgive my english (i'm from Portugal) and forgive me if i said something unacurate, i just tryed to share my small experience.
I'm loving unraid and i've already assembled a secondary backup server (old parts) with unraid too.
I would like to recomend an upgrade to the CPU pinning page... add different colours for the cpus of each memory channel... that way it would be more intuitive to set up numa nodes correctly.
My best regards
Bruno Gomes
-
2
-
Unraid OS version 6.9.0-beta29 available
-
-
-
-
-
in Prereleases
Posted · Edited by btagomes
Mine is stable too, same conditions, couple vms, some with gpu passtrough, some with vnc and some twenty dockers running, only issue now is the novnc connection not working, all other issues are gonne and not a single crash till now.