Jump to content

eribob

Members
 View Profile  See their activity

  • Posts

    100

  • Joined

  • Last visited

 Content Type 

Posts posted by eribob

    Vm's in 6.9 beta - do they work - kmode exception not handled

    in VM Engine (KVM)

    Posted

    I had a similar issue. Windows 10 VM boot-looping with blue screen. Ryzen 3950x. Solved it by adding the following to the end of the VM xml as suggested elswhere on this forum: 

      

      <qemu:commandline>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='host,topoext=on,invtsc=on,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff,hv-vpindex,hv-synic,hv-stimer,hv-reset,hv-frequencies,host-cache-info=on,l3-cache=off,-amd-stibp'/>
  </qemu:commandline>

     

    • Like 2

    Advice on good GPU:s for VM pass-through

    in VM Engine (KVM)

    Posted

    Dear community! 

    I am very pleased with my unraid machine and it gives me a little punch of joy in my stomach every time I open the dashboard. It is a continuous work in progress and I feel that better GPU:s are the next step for me. 

     

    Currently I have an Rx 580 4Gb (2.5 slot) and a R7 370 (2 slot). It works ok but gives me some problems. 

    1a. The Rx 580 is passed through to a Windows 10 VM and every time I restart or shut down the VM it goes into what must be a version of the reset bug. When I try to launch the VM again I see the BIOS logo but just as Windows is about to start the monitor goes to sleep and the VM pauses. I have to force stop it and run the reset script provided by Spaceinvader One to be able to start the VM again.

    1b. The card is so wide (2.5 slots) and large that it covers other PCI-E x 1 ports on my motherboard that I wants to insert USB cards into.

     

    2. The R7 370 is ok for linux VM:s and the 2 slot size is also ok, but my Catalina VM does not accept it. Maybe this can be solved with special drivers but I would rather not go there. 

     

    I am looking for 2 new GPU:s that do not need to be very powerful. I play some games but mostly CS:GO... The Rx580 4GB is adequate. So somewhere in that neighborhood or a bit more powerful is fine. Being able to buy used is only a plus. I think that new GPU:s are very over-priced in general. At least one of them needs to be AMD so that my Catalina VM will accept it. The other for the windows VM can be NVIDIA or AMD. Another requirement is that it is possible to run them smoothly without reset bugs or other issues. I want to be able to restart my VM:s! 😃

     

    One final thing: I do not want to insert a third GPU for unraid only if not absolutely necessary. As it is today I do not need that, and my CPU and motherboard are really too limited in PCI-E express lanes. 

     

    Thank you so much in advance for any advice!  

    Disk disabled due to read errors

    in General Support

    Posted

    I disabled the option in the Parity Check tuning plugin "pause and resume array operations if disks overheat". I had warning disk temperature at 45 and critical at 55 (I believe it is default, since I cant remember ever changing those values). I now raised the warning to 50 and critical to 60 as well. After disabling the "pause if overheat" the rebuild process has been progressing without problems (now on 39%). So most likely it was pausing due to temperatures approaching the warning level.

     

    Since I have important data on my array and no parity until the rebuild is finished, I want to await the rebuild process now. So I do not want to try and reproduce the error. 

     

    Thanks again for quick support!

    Disk disabled due to read errors

    in General Support

    Posted

    Hi again! 

    I am following your instructions. I replaced the disk data cable and removed it from the array. After that I re-inserted it and the disk is rebuilding. However, the rebuild process keeps getting paused with the message:  

    Parity Tuning Operation: 2020-09-02 16:05

Notification
unknown action: recon D1 (1.6% completed) Pause

    I can resume the process again when it pauses and it will run for another couple of minutes or so, but then the same thing happens again. 

     

    The system log also mentions the drives being overheated. Is that causing the recon D1 problem?  

    Sep  2 16:00:34 Monsterservern kernel: md: recovery thread: recon D1 ...
Sep  2 16:05:01 Monsterservern parity.check.tuning.php: Paused unknown action: recon D1  (1.6% completed) : Following drives overheated: 34 34 34 31 
Sep  2 16:05:01 Monsterservern kernel: mdcmd (44): nocheck PAUSE
Sep  2 16:05:01 Monsterservern kernel: 
Sep  2 16:05:02 Monsterservern kernel: md: recovery thread: exit status: -4
Sep  2 16:08:04 Monsterservern kernel: mdcmd (45): check Resume
Sep  2 16:08:04 Monsterservern kernel: md: recovery thread: recon D1 ...
Sep  2 16:10:02 Monsterservern parity.check.tuning.php: Paused unknown action: recon D1  (2.1% completed) : Following drives overheated: 34 34 34 31 
Sep  2 16:10:02 Monsterservern kernel: mdcmd (46): nocheck PAUSE
Sep  2 16:10:02 Monsterservern kernel: 
Sep  2 16:10:03 Monsterservern kernel: md: recovery thread: exit status: -4

    Perhaps I should remove the side panels from the case and attempt to continue? 

    monsterservern-diagnostics-20200902-1614.zip

    Disk disabled due to read errors

    in General Support

    Posted · Edited by eribob

    Hi! 

    My array has been working perfect until today. One of the disks were suddenly disabled due to read errors. The SMART report seem to indicate that the disk is healthy however? (It has FAILED: Never on all attributes as far as I can see). I have posted diagnostics - should I replace the disk or can this be some other kind of bug? I never had a warning from unraid about the disk before today, which is strange if it was failing.

     

    Best regards 

    Erik

    monsterservern-diagnostics-20200902-1225.zip

    Machine Check Events detected

    in General Support

    Posted

    Hi there! 

    Today my server suddenly rebooted without reason and is now doing a parity check because of "unclean shutdown". Soon after I got a error message from the "Fix common problems" plugin saying that I have "Machine Check Events detected". It recommended me to install "mcelog" via the nerdpack plugin and post diagnostics here for help. So I have done this now! 

    monsterservern-diagnostics-20200803-1006.zip

    Thank you for any assistance!

     

    My system spec is as follows, if you need it: Ryzen 9 3950x on Asrock x570 taichi. 80GB of Corsair vengeance LPX DDR4 3200MHz RAM. 

    Installing PCI-E USB card caused massive problems

    in General Support

    Posted

    Hi all wise people! 

    Yesterday I installed a PCI-E USB card to pass through to a second VM. The card is this one: https://www.inateck.com/inateck-ktu3fr-4p-usb-3-0-pci-express-karte-mit-4-usb-3-0-ports.html I chose it because the FL 1100 chipset was recommended in another thread here on this forum. 

     

    To fit it between my GPU:s I had to use a PCI-E x1 -> x1 riser cable from Kolink (https://www.caseking.de/kolink-pci-e-x1-auf-x1-riser-flachband-kabel-19-cm-grau-blau-zurc-005.html) 

     

    The card is recognised by Unraid and shows up in the VFIO-PCI config plugin in its own IOMMU group, however, when I start the server with the card installed it behaves very strange. 

     

    The cache drive starts to disconnect and reconnect with the system repeatedly, resulting in it being registered as an unassigned device, and when I try to update my VM xml I get an error message saying that the disk has entered a "read only state". I also get warnings that my cache drive is unreachable. Further another SSD that I pass through to a VM using "/dev/disk/by-it/[DRIVE ID]" is suddenly not found. 

     

    Also during startup, instead of endning with the login command line prompt and the server IP address as usual, I get the following screen: 

    IMG_5804.thumb.jpg.06b9910e81f0424b393ea27e940e6b90.jpg

     

    It never goes further, but the server is eventually reachable over the network as usual anyway. 

     

    My initial thought is that this could be a power issue, that my Corsair 750W PSU is stretched thin (i have a ryzen 3950X, 64GB DDR4, Rx580, R7 370, 4HDDs, 3 SSDs and 2 NVME drives, as well as some case fans). Do you agree or is there another possible explanation? 

     

    Against this power theory, the problem occurs even though the system is not under any substantial load however. In addition, I tried to remove the SATA power cable from the USB card, but left the PCI-E connection inserted and the problem persisted. It did not go away until I disconnected the card from the PCI-E port. 

     

    Another theory is that maybe my motherboard shares PCI lanes between some of the SATA ports and the PCI-E x1 port that I want to use for the USB card. I have an asrock taichi x570 board. 

     

    I forgot to save the diagnostics! I can re-insert the card and recreate the error if diagnostics would be helpful though. 

     

    Thanks for support in advance! 

     

     

    3950x performance

    in VM Engine (KVM)

    Posted

    Hi! 

    I finally built my 3950x unraid system and I am very happy with the performance increase so far! 

     

    I have done some benchmarks today with my windows VM, and I would just like to ask if my numbers are to be expected or if there are tweaks to the xml that could improve performance? 

     

    Precision boost overdrive is turned on in the BIOS. Cooling the cpu with noctua NH-D15.

     

    To test, I assigned all cores to one windows VM and shut down all dockers and other VM:s. It resulted in: 

    Cinebench R20: 8600-8700 multi-core
    Geekbench 5 CPU: 1186 single-core, 11470 multi-core 

     

    For actual use I have isolated the last 8 cpu:s (8-15, 24-31) and gave them to the VM along with 32GB of RAM. This resulted in:

    Cinebench R20: 4552 multi-core

     

    Bare metal performance seem to be around 9100p in CB20 and 14300 in geekbench, meaning I get around 5% decreased performance in CB20 and 20% in geekbench. Is this to be expected from the virtualization?

    Ryzen 3950x build - is this viable?

    in Hardware

    Posted

    1 hour ago, juan11perez said:

    GPU1 - Saphire RTX 570 (primary)

    You mean RX 570? 

     

    8 hours ago, testdasi said:

    Currently, the 1070 is the only GPU in the system (i.e. Unraid boots with it) and it is also passed through to the same workstation VM after boot.

    It seems like an NVIDIA card should work then as primary, after dumping the vbios. The 1070 is not so bad, at least it seems stronger than my Rx 580 and I do not need super GPU power...

     

    1 hour ago, juan11perez said:

    I dont game, sorry.

    w00t? 😃

     

    It is all coming together nicely I think! I will wait until my wallet feels full enough and then upgrade and then harass you again once everything fails...

    Ryzen 3950x build - is this viable?

    in Hardware

    Posted · Edited by eribob

    3 hours ago, testdasi said:

    Not a big deal if your RX 580 width is double-slot. A big deal if your RX 580 is the 2.5-slot width variety.

    It seems to be 2.2 slot: https://www.sapphiretech.com/en/consumer/nitro-rx-580-4g-g5#Design 

    I will look into gigabyte options! 

     

    Can you recommend a GPU that is known to function so that you can pass it trough even though it is the primary GPU in the system?

    3 hours ago, testdasi said:

    Latency is an inevitable result of Ryzen CCX/CCD design. It's not specific to any platform or OS - as long as it's a like-for-like comparison. E.g. you can't compare bare metal latency vs VM latency (which naturally is always higher). But between 2 VM or between 2 bare metal configs, Ryzen (and Threadripper) will always have higher latency.

    I see. So the latency is there even when you run bare metal? I guess my question is if the latency on a VM in a ryzen CPU server is so much

     

    higher than the latency on a intel CPU server that intel is clearly a better option if you want the ability to game on your VM?

     

    3 hours ago, testdasi said:

    Unraid does not need a graphics card at all. That's a misconception when people refer to a card that is "used" by Unraid.

    I understand that. Is there a workaround so that you can pass through your primary GPU even though the BIOS snags it at startup? EDIT: Sorry 0.5s of googling found me the spaceinvader video... So I will try to get my hands on a decent used NVIDIA gpu then.

    Ryzen 3950x build - is this viable?

    in Hardware

    Posted

    3 hours ago, juan11perez said:

    HI, I'm not knowledgeable enough to comment on some of your questions, but I'll share info on what Im running now. 

    Asus ROG STRIX X470-F, Ryzen 3900x, 64GB RAM, 3 x 4TB Seagate, 500GB NvMe, GPU1 - Saphire RTX 570, GPU2- Gigabyte GTX1650

    I can definitely run 2 vms at the same time without issue in addition to about 40 dockers.

     

    That is super cool to hear! Very close to what I am aming for. I have acctually considered the 3900x as a budget option. 

     

    How did you manage the 2 GPU:s? Which one do you have as primary? As testdasi said, it needs to be one that can be passed through to a VM despite also being used by unraid at startup? 

     

    How many CPU cores do you allocate to each VM and to the dockers? 

    Do you game on your VM:s? In that case, does it work ok with more modern titles? 

     

    The RAM is 4 x 16GB?

     

    Are there any major issues that you have had during the setup? 

     

    Thanks! 

    /Erik

    Ryzen 3950x build - is this viable?

    in Hardware

    Posted · Edited by eribob

    4 hours ago, testdasi said:

    I don't think you can do 2xGPU + 2xNVMe + 1xHBA with X470. Check the mobo owner manual carefully because in most cases, some slots are deactivated if other slots are occupied.

    I looked at a manual for the Asrock Taichi x570 board that I have been thinking about buying and this it what it says with regards to PCI-E and M.2 slots: 

    Quote

    AMD Ryzen series CPUs (Matisse): 3 x PCI Express 4.0 x16 Slots (PCIE1/PCIE3/PCIE5: single at x16 (PCIE1); dual at x8 (PCIE1) / x8 (PCIE3); triple at x8 (PCIE1) / x8 (PCIE3) / x4 (PCIE5))*

    This should mean that if I populate all of them using two GPU:s and a HBA I would get PCIE 4.0 x8 bandwidth (16 GB/s) for the GPUs and x4 speed (8 GB/s) for the HBA.

     

    The bandwidth for the GPU:s should probably be fine right? I will not be running the latest and greatest GPU:s since I am not a gamer (currently have a rx580 4GB card I bought used). 

     

    8 GB/s max for up to 8 drives that could be attached to the HBA, thats 1 GB/s per drive, which is almost 10x more than what most mechanical drives can deliver anyway, and rarely are all drives accessed at the same time right? 

     

    Quote

    1 x Hyper M.2 Socket (M2_1), supports M Key type 2242/2260/2280 M.2 SATA3 6.0 Gb/s module and M.2 PCI Express module up to Gen4x4 (64 Gb/s) (with Matisse)

    1 x Hyper M.2 Socket (M2_2), supports M Key type 2260/2280 M.2 PCI Express module up to Gen4x4 (64 Gb/s)*

    1 x Hyper M.2 Socket (M2_3), supports M Key type 2230/2242/2260/2280/22110 M.2 SATA3 6.0 Gb/s module and M.2 PCI Express module up to Gen4x4 (64 Gb/s)* *

    Quote

    If M2_3 is occupied, PCIE5 slot will be disabled

    So this means that I can not use the third M.2 slot, but I was looking for max 2 slots anyway (one for each VM is what I envision...)

     

    Quote

    Ryzen CPU lanes are used to feed the top two full-length expansion slots in the usual x16/x0 or x8/x8 format. As such, Crossfire and SLI is supported. The bottom full-length connector gets its PCIe 4.0 x4 connection from the X570 chipset and shares that link with the lowest M.2 connector. Both PCIe 4.0 x1 slots are also fed by the X570 chipset.

    So the 24 lanes provided by the CPU + the extra lanes provided by the x570 chipset should be enough for everything???? 

     

    This is a quite expensive board though. I will look at a cheaper option as well and come back with how that would work. 

     

    Another problem is, since I want to use the 2 GPU:s for the 2 VM:s, I need to be able to pass both through even though one might be used by Unraid? But I think I have read somewhere that this can be acieved? Perhaps spaceinvader one has a tutorial... 

    Ryzen 3950x build - is this viable?

    in Hardware

    Posted

    Many thanks for the answers. 

     

    Perhaps the scales are in favor of a x570 board then. I have been curious about the PCI-E 4.0 that they support. That would double the bandwidth of the only 24 PCI-E lanes that the 3950x has in theory, but I dont know if it will be true in practice... 

     

    I will look at Gigabyte boards. And do modest tweaks for improved latency then. Is the latency a problem with KVM or is it specific to unraid? I expect it has to do with KVM, just wanted to ask... Also, will the upgrade to the newer kernel in 6.9 help in any way? 

     

    Too bad threadrippers are so expensive hehe... 

    Ryzen 3950x build - is this viable?

    in Hardware

    Posted

    I am planning for a server upgrade. My current build is 2 x e5 2670 and I run my main computer as a VM from them. I am really happy with the general setup bu I feel that the 2.6-3.3ghz speed is a little too slow. For example cities skylines runs at about 15fps now that my city has grown a bit. Lowering the graphics settings does not improve that, which makes me think it is a cpu issue, a lot of computations for all the inhabitants in my city perhaps? 


    The 3950x is such a powerful chip and even though I am not editing 4K video I think this kind of build has the potential to fully utilise all the fast cores it provides! I use my computer for programming, some statistical calculations for my research, and sometimes gaming.
     

    I would like to be able to do the following:
    - Run 2 VM:s, with a dedicated gpu for each of them. I mainly use win 10 but want to have a Mac as well for fun :)
    - run my 20 or so docker containers
    - Use my SAS expansion card for additional hdd:s
    - Preferrably have at least 2 nvme drives.
    - Be able to upgrade to 128gb ram later
     

    I don’t think I need pcie gen 4 so I would like to use a much cheaper x470 board if possible (or do you see a need for x570 here?)

     

    I only need cpu motherboard and ram. The rest I will re-use. 

     

    I made the following list but I would like some input from you experts as it is a big purchase! Will the board accept 32gb ram sticks for example? Do you see any other major issues here?


    https://pcpartpicker.com/user/eribob/saved/8y3j23
     

    what about gaming on a vm on a ryzen server? I read somewhere here that there might be performance issues? I am no competitive gamer, just want to be able to play for fun occationally. 
     

    looking forward to your insights!

    Array: Start Button missing

    in General Support

    Posted

    Hi! I rebooted normally (just like someone else in this thread) and the start button is back. Just wanted to post my diagnostics in case someone was interested in investigating this bug. 

     

    I am also impressed by the speed with which you people reply in this forum! 

     

    Thanks.

×
×
  • Create New...