1812

wait, you're right, just looked up the specs on your cpu, no hyperthreading. if you don't isolate cores in the syslinux.cfg file, then unRaid and the docker still have access to all 4 cores, regardless of what you do in the vm manager.

try changing your cpus to 1,3 only. Many find that keeping windows on paired thread on a core helps with lag, and the addition of having 1 thread from one core (cpu 2) with 2 on another may be causing issues (I'm not 100% convinced of this, but windows isn't my main working environment and haven't done extensive experimentation with it.) And at this point it's worth a shot. Also keep an eye on your core utilization while booting/using the vm. If you haven't isolated cores from unraid/dockers for the vm exclusively to ue, it may be that they are bogging it down. But if utilization doesn't max out while trying to do normal things (web, etc...) then that probably isn't the issue but it' not helping the situation either (it also wouldn't hurt to consider letting the vm have it's own core(s).)

I have unRaid on 4 DL380 G6 servers, add disks as needed, run a couple dockers, run a couple vm's. Not sure if unRaid will like your B320i... I'm pretty sure I used mixed rotational rates in my array. Only matters when reading/writing to that specific disk or during parity checks, where you are limited by the speed of the slowest disk. You may run into issues if you try to use a gpu. Aside from smaller specifics, that's about it for me.

Ok, so here is a problem in the system log: Tower kernel: vfio-pci 0000:07:00.1: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor . and that's the issue I had. I would say try booting the vm without audio, but the video card is still in group 1 with another device and most likely won't boot... but.. it costs nothing to try. This also pops up in the logs Feb 3 12:01:08 Tower kernel: [Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR 38d is 330) This actually shows up in my logs as well, but can be ignored from what I read online.

my next suggestion was to try and rule out if it was a slow usb response issue or slow vm/video issue, but it appears you've done that. what driver is the video card using? are you running anything in docker/plugin? what does your cpu utilization look like with the vm running?

The logical threads used are identical. How they are presented to the vm is what changes. C presents the vm a 3:2 virtual processor on a 3:2 physical D presents the vm a 6:1 virtual processor on a 3:2 physical This would illustrate differences in presenting the same paired physical cores to a vm as either a virtual ht processor (which is accepted as how you should do it) vs. presenting the virtual processor as 6 non hyper threaded cores. As it turns out, there are none. E presents the vm a 3:2 virtual processor on a 3:2 physical F presents the vm a 6:1 virtual processor on a 3:2 physical The difference with E & F is that the order of cores are mixed vs C & D, and not in what is generally accepted as the optimal topological layout. To view differences between accepted windows ht pairing format and the implied optimal one for os x when using ht pairs, You would compare tests C & E, and D & F, which also shows essentially no change. Everyone says "You have to put your ht pair on the ht thread of the physical cpu." Well.... testing implies otherwise. Regardless of how the virtual cpu was presented to the vm or even the core order, mixing what we perceived as windows ht cores doesn't matter in terms of performance/cpu benchmarks. And your observation is correct, they yielded similar scores. which is why I wrote "I don’t think virtualized windows cares about HT pairings." Windows testing implies that that defining a topology really didn't matter in benchmark scores in virtualized windows, as it appears to treat all cores presented to it as the same, unlike os x. Yes, expected results that 6 cores are better than 3 thread pairs, which is why I wrote: "In any test utilizing HT cores/threaded pairs on a physical processor, scores were consistently lower than using non-paired cores. Not that that was ever in question either, but shows consistency in testing." Test 2 & 3 are not the same in terms of assignment: Test 2 presented the vm a 3:2 virtual cpu consisting 3 physical cpus and their ht pair, laid out with the vm ht pair on the physical cpu ht thread. (see image 2 in attachment) Test 3 presented the vm a 3:2 virtual cpu consisting of 3 physical cpus and their ht pairs laid out with the the accepted windows HT pairing scheme, which then placed os x ht threads in alternating positions of the vertical row. (see number 3 in attachment) I suspect the results are similar because regardless of where the ht thread is placed in these 2 tests, both logical threads on a core were still utilized to the same percentage because they each contained 1 thread at 100% and 1 thread at less than 100%. Test 1 presented the vm a 3:2 virtual cpu consisting of 6 physical cores (no pairs used) Test 4 presented the vm a 6:1 virtual cpu consisting of 6 physical cores (no pairs used) Differences in benchmark scores of these tests show how the os x vm performs when presented a 3:2 virtual cpu vs a 6:1. And this is actually the most important test for os x, as it led to the discovery that if you give the vm a virtual HT processor, (3:2 in this case) then half of the cores will not be utilized to 100%, reducing overall performance. This is the key difference between windows and os x on kvm. And as I proposed earlier, I believe it is because windows "knows" it is running in a virtual state and does not throttle down any cores as hyper threads. OS X on the other hand, believes it is a real boy (computer) and won't push it's HT cores to 100% because in a real world setting, you can not achieve 100% utilization on 2 threads of a physical core. If you could, then the benchmark score of a single thread of a given core would be doubled when you ran the test again utilizing both its threads. But that does not happen. If it did, then every test run in this benchmark across 6 physical cores would have a similar score to the run on the 3:2 physical cpu. There is roughly an 8-12% difference between benchmarks scores of the physical cpu at 3:2 vs 6:1, with the latter scoring higher (as expected.) And if you look at the os x images (attached) that presented the vm a virtual HT processor, unRaid shows cpu usage on the ht cores to be 6-8% less than 100%. I believe the variance of a few points between these two sets of numbers (benchmark differences and displayed cpu utilization) can be explained as not expecting unRaid to be 100% precise on the dashboard utilization, since it is not a realtime graph but a snapshot, combined with the vm also using cpu resources to run itself as well as the benchmark. Does that 10% difference matter in terms of overall performance on a vm? On the small scale side of things, probably not. If you're scoring a 300 and reach a 330, it's not a big deal. But if you're running a 24 core vm and intend to use an application that will push your cores to full utilization, then there is definable difference in how long it takes to complete a given task if you're benching at 1420 vs 1291. Tell me about it.....

I meant to drop this here the other day for posterity: OS X Performance: cpu pinning, benchmarks, discussion and windows comparison https://lime-technology.com/forum/index.php?topic=56139.0 *note- no audio/video testing was done in conjunction with these tests. Using the non-accepted settings may cause less than desired audio/video issues in windows. In previous testing, using non-paired cores in os x produced no audio/video issues and increased performance. This includes running 2 separate vm's on a core's separate threaded pairs. OS X audio issues only arose when vm's were placed on the same core threads. (more info here: https://lime-technology.com/forum/index.php?topic=51915.msg533600#msg533600)

I use Krusader (downloadable via community apps) to duplicate a vm img file. I have done the following for win10/osx with success: Once img is duplicated, click the vm tab, select a template, for Primary vDisk Location change to manual and navigate to the new img location. Finish setting up the remainder of the vm template. Then launch. There maybe better ways, but this works for me.

could you not just put the files in a share folder and then mount the share? the virtual network adapter in windows 10 runs at 10gbe, so it would be fast enough. And you could set permissions to read only. I know you can do this with with apps on OS X, but not sure about windows.... it handles program installs a bit differently....

is the disk image on a cache drive/pool? in the array? What version of unRaid are you on? a little weird that it is in the app data folder (/mnt/user/appdata/Windows 10 Test/vdisk1.img) but that alone shouldn't cause the issue I think.

OK, less sleepy answer: Your problem is that this 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09) is in the same iommu group with your video card /sys/kernel/iommu_groups/1/devices/0000:00:01.0 /sys/kernel/iommu_groups/1/devices/0000:07:00.0 /sys/kernel/iommu_groups/1/devices/0000:07:00.1 You can't pass through only part of an iommu group. It's all or nothing. You said that ACS Override did not do anything. So there are 2 options that I can think of (others may have more or better info) 1. Can you move the card to another slot and see if it changes the iommu group to just only contain the video and audio components ( currently 07:00.0 & 07:00.1 but will possibly change)? 2. I had a different issue with iommu groups and a possibly faulty nvidia card https://lime-technology.com/forum/index.php?topic=54786.msg523314#msg523314 If you attach your logs, it will help diagnose the situation better, I believe the issue I had is what you have, but the fix for it might work as it achieves the desired result, separating devices out of an iommu group. So to try it, add the following to your syslinux.cfg append vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=id:10de:10c3,10de:0be3 initrd=/bzroot This will attempt to move the card components into individual iommu groups. You don't have to use MC, you can do this in the web gui under Main tab > click on the text "Flash" under boot device, scroll down and edit there, apply, then reboot. After reboot, check iommu groups and see if the devices are now separated (may be 2 different groups.) If so, try to boot vm. --> Since I was not using the audio portion of the gpu in the link I posted, I do not know what the effects may be if the video and audio are in separate iommu groups. If the vm won't boot, then remove audio and try again. But really, logs would help a great deal.(Tools>diagnostics) --edit there is also this, while not directly related (or might be, but can't tell without logs: http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=emr_na-c04781229&sp4ts.oid=5249566

Which sounds great except for the fact that you'd have to live...in...Texas....

GeForce GT 710(B) attached. note: my system identified this card as: VGA compatible controller: NVIDIA Corporation GK208 [GeForce GT 710B] (rev a1) I do not know if a card that showns as 710A or just 710 (if either of those are actually identifiers) will be the same. I chose to include the listed naming scheme in my rom dump. GeForceGT710B.dump.zip

sorry, forgot about this: GeForce GT 730 attached. 710 exported a 0kb file and is being a pain. Will try again on it in an hour. GeForceGT730.dump.zip

https://www.youtube.com/watch?v=ntjQphOSPPI https://www.youtube.com/watch?v=rDGSS1iEtq4

on my g6 servers, I had a similar, if not the same, iommu issue. It was resolved by adding the following to the syslinux.cfg to allow unsafe interrupts. append vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot I know you've said you've done it, but can you post your complete syslinux.cfg file? on second look, you also have an iLo usb controller in the same iommu group (13). If you con't pass that through as well, or find a way to split it out, it'll never work. Does ACS override not split the group up better? If not, let me know, I had another issue that I somehow managed to split out a video portion of a card away from an audio portion for passthrough, which might work for this? just woke up and realized I was looking at the onboard video....

I'll put this on my todo list. That would be fantastic and very much appreciated!

any chance to get the ability to display more than 1 ethernet interface at a time in stats?

Setting tab, Stats Settings near the bottom. change ethernet interface to bond0

I had a half rack that was in my office and just moved it to the basement with a high capacity dehumidifier. Actually ended up building a standalone clean room for it out of wood and plastic. A friend of mine said it looked like a murder room.... It houses my 4 servers, another 2 computers, a disk array, pfsense modem, a synoloy nas, cable moden, 48 port switch, and some other techno-goodies. The fans can howl all the want/need and it doesn't bother anyone in the house. Also provides silent computing for my remote vm's in the house. Was a fun project. I didn't have space for it, but made it so I could have all the goodies it holds. At some point I guess i'll have to post a photo of it in here... BUT You've got a great server there!

once you get a rack, you never go back...

I use to use mc on terminal in os x... switched to adding krusader via community apps and never looked back. Although I don't use the editor, nor have ever looked for it, I believe it has one.

testing results moved to a new thread so I can quit hijacking gridrunners: https://lime-technology.com/forum/index.php?topic=56139.new#new

-WARNING - LONG POST- I had a project cancel this am, so I was able to do get to this a little sooner than expected. And since this got long, I decided to remove it from the OS X sierra install thread and give the discussion its own space. I believe I realized why topology settings did not affect performance before in my testing a few months ago. I tested topology settings on a windows vm. And as I discuss later, you’ll see why I thought it did nothing (spoiler: it really doesn’t…) The following tests were completed to determine optimal cpu performance settings in relation to cpu pinning and topology in os x. There were not done to test lag, video/audio responsiveness, or any other element. For the previous tests in the sierra install discussion, no topology was ever defined for the vm. So for these tests, I swapped to using the Linux xml that gridrunner provided which defines topology. And there were some noticeable changes between showing the vm either a grouping of individual single thread cores vs cores with a virtual hyper-threading processor, and an interesting revelation about os x in virtual environments. SO, on to the scores! Note: no tests were done with audio/video Modified Linux xml (previous tests were modified Ubuntu xml) 8GB ram Cinebench R15 Same fresh copy of un-updated os x emulator pin 1-2 All set to host passthrough Proc 2 isolated from unRaid CPU Thread Pairings Proc 1 cpu 0 <===> cpu 12 cpu 1 <===> cpu 13 cpu 2 <===> cpu 14 cpu 3 <===> cpu 15 cpu 4 <===> cpu 16 cpu 5 <===> cpu 17 Proc 2 cpu 6 <===> cpu 18 cpu 7 <===> cpu 19 cpu 8 <===> cpu 20 cpu 9 <===> cpu 21 cpu 10 <===> cpu 22 cpu 11 <===> cpu 23 (test numbers correlate on the attached image of processor utilization in unRaid) 1. Topology 3 cores:2 threads, processors 6-11 (see image 1) not all cores not utilized to 100% 482 485 486 2.Topology 3 cores :2 threads, processors 6,18 7,19 8, 20 (HT Pairs on physical cpus) same issue, vm HT cores not pushed to 100% utilization 351 350 350 3. Topology 3 cores :2 threads, processors 6, 7, 8, 18, 19, 20 (HT Pairs on physical cpu) same issue, HT cores not pushed to 100% utilization 345 351 348 4. Topology 6 cores: 1 thread, processors 6-11, 100% cpu usage 541 535 540 5. Topology 6 cores: 1 thread, processors 6,7,8, 18,19, 20, 100% cpu usage 359 356 360 6. Topology 6 cores: 1 thread, Processors 6,18 7,19 8, 20 (ht Pairs on physical cpu) image 4 100% cpu utilization 360 363 361 and for fun: 7. Topology 2 sockets, 3 cores, 1 threads, processors 6-11, 100% cpu utilization 540 536 542 8. topology 2 sockets, 3 cores, 1 threads, processors 6,18 7,19 8, 20, 100% cpu utilization 365 364 363 ——————— The info you don’t care about - Setting the topology to 2 processors with 3 cores 1 thread each showed no real change in benchmark vs showing the vm 1 processor of 6 cores with 1 thread each. Not that this was ever debated, but I figured it wouldn’t take too long just to see what happens. In any test utilizing HT cores/threaded pairs on a physical processor, scores were consistently lower than using non-paired cores. Not that that was ever in question either, but shows consistency in testing. Now something interesting- Declaring a topology that presents an OS X vm a hyper-threaded processor results in degraded performance numbers. This performance loss is not improved regardless if you use only physical cores or physical threaded/HT core pairs. It is continually lower vs. either setting a topology of 6 cores 1 thread, or no topology definition at all (which defaults to 6:1) And I think this is why- Through viewing cpu utilization in unRaid of the cores, and presenting OS X with a virtual HT processor created via topology declaration (3:2), and assigned on 6 physical single thread cores, it displayed that every other core was not achieving 100% utilization. (see image 1.) Even as the core assignment changed to the defined virtual processor (3:2), the degradation of performance on every other core remained (see benchmarks/images 2 &3.) I believe this is because OS X does not push “HT threads” (a second thread on a perceived core) to full utilization. This also means that OS X sees vm HT pairs assigned in horizontal groups, not the vertical rows in the xml as is currently believed to be the case with windows. To achieve os x HT pairing using this method would be defined in xml as <cputune> <vcpupin vcpu='0' cpuset=‘6’/> <vcpupin vcpu='1' cpuset=’18’/> <vcpupin vcpu='2' cpuset=‘7’/> <vcpupin vcpu=‘3’ cpuset=’19’/> <vcpupin vcpu=‘4’ cpuset=‘8’/> <vcpupin vcpu=‘5’ cpuset=’20’/> </cputune> while the windows accepted standard way is <cputune> <vcpupin vcpu='0' cpuset=‘6’/> <vcpupin vcpu='1' cpuset=’7’/> <vcpupin vcpu='2' cpuset=‘8’/> <vcpupin vcpu=‘3’ cpuset=’18’/> <vcpupin vcpu=‘4’ cpuset=‘19’/> <vcpupin vcpu=‘5’ cpuset=’20’/> </cputune> I needed to determine if this behavior was indicative of my hardware, or os x. At this point, I booted up a windows 10 vm and proceeded with the same testing methodology. Windows 10 8GB ram no emulator pin (forgot, but inconsequential for testing reasons) Cinebench R15 host passthrough A. Topology 1 socket, 3 cores, 2 threads, processors 6-11, 100% cpu utilization 535 527 526 B. Topology 1 socket, 6 cores, 1 threads, processors 6-11, 100% cpu utilization 551 549 551 C. Topology 1 socket, 3 cores, 2 threads, processors 6,7,8, 18, 19, 20, (accepted windows pairing) 100% cpu utilization 376 369 368 D. Topology 1 socket, 6 cores, 1 threads, processors 6,7,8, 18, 19, 20, 100% (accepted windows pairing) cpu utilization 368 368 371 E. Topology 1 socket, 3 cores, 2 threads, processors 6,18, 7,19, 8, 20, (os x thread pairing) 100% cpu utilization 365 367 367 F. Topology 1 socket, 6 cores, 1 threads, processors 6,18, 7,19, 8, 20, (os x thread pairing) 100% cpu utilization 367 368 367 Windows testing results were as expected in terms of a vm running 2 threads on 1 core performing less than one using 1 thread per core. Also evident was that Windows does not show the reduction in core utilization regardless of topology declaration (I did not attach images because they all showed the same result.) And as such, I am unable to determine if the HT pairings that windows uses are actually different than OS X. And this may be for good reason. I don’t think virtualized windows cares about HT pairings- When testing topology declaration that would seem to match os x HT pairing (grouped) vs the accepted windows HT pairing (row) the results showed almost no difference, which is also interesting because it implies that windows treats what it believes as paired threads on a physical core being presented to it as the same. This is demonstrated by the close benchmark results that occurred when using the same physical cpus, but topology differences. And if you look at task manger>performance>cpu, it lists a total count of virtual processors, no topology definition. Windows is self aware… So trying to pair threads by defining topology for a windows vm, in terms of performance, is essentially an exercise in futility as you reap no tangible gains vs presenting the vm a grouping of cores. And as the tests showed, even mixing up what could be be considered HT pairs and putting the “wrong” pair together had essentially no change in results (Compare tests C:E, D:F.) How does this relate to os x- Since os x is not optimized for virtual environments, it still “believes” it is on bare metal and tries to compensate accordingly by not forcing 100% cpu usage on what it perceives as the second thread on a physical cpu, since there is no way to achieve 100% utilization of both threaded pairs at the same time. This is demonstrated by tests that use 3 physical cores and 2 threads on each achieving lower scores than 6 physical cores of 1 thread each. Bottom line- Based on these cumulative test results, if you want to maximize os x cpu performance, give it a group of single threaded cpus and don’t present it a virtual processor with HT pairs defined in topology. I would ask anyone else to replicate this process and see if their results match.

Leaving this here incase someone else runs into this issue. I searched and could not find any info on it. problem: vm boots but hangs after boot loader (os x) vm logs show: usb_desc_get_descriptor: 3 unknown type 33 (len 10) usb_desc_get_descriptor: 2 unknown type 33 (len 10) usb_desc_get_descriptor: 1 unknown type 33 (len 10) This appears to be an issue with my hardware and not the usb card. I do not believe it is the USB card I am using since I replaced it once believing it was faulty (inateck kt4006) which initially fixed this problem and the vm booted. But with the new card, it worked fine for a few days and then the issue popped up again. Temporary resolution was to shut down all vm's, stop array, then restart. No server reboot required. I do not know the underlying problem that causes the issue.