ryoko227

Members
  • Posts

    161
  • Joined

  • Last visited

  • Days Won

    1

Everything posted by ryoko227

  1. Updated from 6.5.3 rc1 on 2 machines with no issues to tell atm.
  2. Hmm, *thinking* interesting way to work around it! I'll take a look at that when I get home tonight and let you know how it goes. Thank you for the suggestion! EDIT- Seems to have worked out Thank you again bonienl!
  3. I have an M.2 nvme drive that is mounted in the go file and resides in Unassigned Devices. I love that I can now see temps and all the work done in this area (thank you for that!). Previously, I got help solving Array and Cache drive temp warning spam on another machine from johnie.black, but the method of clicking into the drive name and setting individually doesn't seem to be available under UD. Specifically there are not any temp or utilization threshold option/textboxes. I'm sure I could probably set this globably in disk settings, but since the drive can regularly run at/around/over 50C, I am not comfortable with the other drives having to get this hot before sending me a notification (not that they do-but incase of an issue). Is there anyway to add this into UD? Or some other way of doing this that I am missing?
  4. Updated from 6.5.1 to 6.5.3-rc1 on my Home TR system. The VMs are located on a isolated m.2, and since this was directed at certain intel cpus, didn't expect to see or see any noticable boot speed changes. No issues to note with the update.
  5. Interesting, I have the Corsair H100i V2 AIO mounted in-case (no direct clean air or exhaust) with 2-120x15mm slim fans running at 650rpm and this thing idles at 38C and goes up to maybe 42 under load (with fans jumping up to 750rpm). All BIOS controlled. Clean air she ran 32C at idle. EDIT - I'm using thermal grizzly hydro I think it was
  6. Just updated from 6.5.1 to 6.5.3-rc1 to see if this update fixed the long VM boot up times on this server. (Short answer - noticably faster) Running on a Intel Xeon E5-2670-V3 (Yokohama server) listed below in sig. Previously it would blackscreen for approximately 1-2mins with a 960gtx and "8" cores being passed through before the OVMF bios would pop up and then load Win10 as normal. After the update the bios splash pop'd up almost immediately. I can see (at the moment) no other issues.
  7. Awesome, thank you much Johnnie!
  8. I have a M.2 nvme drive that's normal op temp under load is around 48C. This regularly kicks off temp warnings whenever its under a decent load. After looking around I found mention of changing the temp thresholds in this post [SOLVED] Ongoing HDD temperature warnings. Unfortunately in 6.5.1, it doesn't seem to be in the Display Settings anymore. Where might I find this setting now?
  9. 2 servers from 6.5 to 6.5.1 with no issues to note.
  10. I don't recall playing with the sleep states either, and I don't seem to have issues beyond whats listed in previous posts. MSI x399 SLI PLUS MB.
  11. @Jcloud I'm not sure if you are still struggling with this, but maybe this might be helpful? Follow up on this mainly to keep my notes all together for anyone else who runs across this issue/post. Originally posted by Bryan Jacobs at Re: [vfio-users] "Device or Resource Busy" and later referenced by unRAID users @brando56894, @realies, and brought to my attention by @Rhynri. You can in fact enable UEFI even with the issue noted above! Simply run the commands listed below prior to starting a VM. echo 0 > /sys/class/vtconsole/vtcon0/bind echo 0 > /sys/class/vtconsole/vtcon1/bind echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind What I ended up doing to automate this for me was using the User Scripts plugin by @Squid. Something akin to this. Add New Script Name efi-framebuffer_unbind Desciption (optional) Startup script which unbinds the efi-framebuffer after array startup to allow proper GPU passthrough and driver loading in VMs. Edit script (required) #!/bin/bash echo 0 > /sys/class/vtconsole/vtcon0/bind echo 0 > /sys/class/vtconsole/vtcon1/bind echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind Scheduled At Startup of Array Then just startup your server UEFI with UEFI enabled in unRAID and you'll be back in business. Many thanks to everyone listed above for their work and efforts!!!
  12. Home Threadripper server updated without issues 1 more to go. Whole system feels more responsive and more solid. Of course, just feels that way. I have no data to back up that perception, www Thank you again LT and the guys here putting all these things together! I dont recall to be honest, I have one more server that I need to update so I will double check when I get up to the site. I usually just update everything before the unRAID update though.
  13. Updated from 6.4.1 -> 6.5 on the office server. About 5.5hours with no issues. Only hickup was the usb hotplug app which has been previously noted. Uninstalled and installed the new one with no issues. Looking good so far!
  14. There have been a few questions that I have asked that went unanswered, but to be honest... I see that as everyone here having only a finite amount time available or just simply not knowing the answer to some specific hardware questions. Aside from that, I have not run into what you are saying. The VAST majority of users have been overwhelmingly helpful and insightful. If anything, it feels like unRAID is growing. Thats just how it feels to me though.
  15. The way I am handling this is by having my SSDs setup in a btrfs RAID-0 cache pool, then syncing the information off onto the array for redundacy nightly. Of course, between syncs&movers the data on this cache pool is vunerable, but having 1TB of space available at SSD access speeds is worth it to me. I really only use it for cacheing and my Steam games library. So if you are just using it for your audio and video files, something like this might be a viable option for you.
  16. Welp, just got back from vacation and started to get the AER error again.. Feb 18 12:15:16 BEAST kernel: pcieport 0000:00:01.1: AER: Corrected error received: id=0000 Feb 18 12:15:16 BEAST kernel: pcieport 0000:00:01.1: PCIe Bus Error: severity=Corrected, type=Data Link Layer, id=0009(Receiver ID) Feb 18 12:15:16 BEAST kernel: pcieport 0000:00:01.1: device [1022:1453] error status/mask=00000040/00006000 Feb 18 12:15:16 BEAST kernel: pcieport 0000:00:01.1: [ 6] Bad TLP Did some quick searching and found this post "Threadripper & PCIe Bus Errors" Seems (as I havent gotten the error since) that a current work around is to toss pcie_aspm=off into your syslinux configuration. For reference here is a portion of my current syslinux config. I have certain cores blocked off from unRAID using isolcpus, and using rcu-nobcs based off of one of @gridrunner video or post that I forgot which, and the pcie_aspm from the post above. label unRAID OS menu default kernel /bzimage append pcie_aspm=off rcu_nocbs=0-23 isolcpus=2-11,14-23 initrd=/bzroot
  17. Not sure if it is unRAID specifically or the uefi bios loading it there, but efifb is being loaded into the same piece of memory that the primary slot GPU was trying to request. Running bios and unRAID in legacy mode fixed this issue for me as efifb is no longer being loaded. I did a quick write up over here detailing the journey, www. Hope this is helpful for someone, or that maybe the LT guys can take a look into it.
  18. <SOLVED> vfio-pci 0000:0a:00.0: BAR 3: can't reserve [mem 0xe0000000-0xe1ffffff 64bit pref] vfio-pci 0000:42:00.0: BAR 3: can't reserve [mem 0xc0000000-0xc1ffffff 64bit pref] TL;DR version: efifb is being loaded into the area of mem that is trying to be reserved by the GPU. Having bios and unRAID load into Legacy mode and not UEFI keeps this from reserving the mem location and allows the GPU to pass-through correctly. Long Version: After trying everything listed above in the previous posts, I decided to pull the whole system apart and re-seat everything. This had one benefit of correcting the AER: Multiple Corrected error received and PCIe Bus Errors. I think I had also over-tightened some screws (namely the AIO cooler and MB). After this I tried using the different paired DIMMs to see if there was some sort of a conflict there. No joy. After kind of having given up and just searching randomly for "BAR can't reserve", I came across this post [Qemu-devel] vfio-pci: Report on a hack to successfully pass through a b. In the author's situation, Robert Ou, discussed how the BOOTFB framebuffer was being placed in memory where the GPU was also trying to reserve during pass-through. I ran a cat /proc/iomem and sure enough there was efifb sitting right where my GPU was trying to reserve. I figured that efifb probably meant EFI Framebuffer and decided I would try running unRAID in Legacy mode to see if it would keep the efifb from loading into that same piece of memory, and yay me, it worked!! The VM loaded up with no issues, and immediately jumped out of that crabtastic 800x600 into glorious 4k! Also a nice piece of info to note, is after having loosened the over-tightened screws on everything. The nvme drive shows up in some of the boot options now, yay!
  19. Oh, well that is both good and bad news then. Good in that it is confirmed to work for people. Bad in that it might seem I have other problems to deal with, lol. No no no, I don't think your suggestion wasn't helpful, quite the contrary. The fact that the M.2 couldn't be updated pre all the virtualization stuff is leading me to believe that there is probably a deeper hardware issue at hand. It's speculative at the moment as I still have some card swapping tests to do, but I'm thinking there might be an issue with the upper slots. It might also imply that the M.2 slots are tied to those upper slots as you suggested. My current thinking is that the MB might have an issue, though tonight's testing should help show or rule that out (I hope). I will also check if the NVME is listed under PCIE devices and if its selectable as a boot device as well when I get home tonight. EDIT - put in 4th DIMM, no change. Current syslog and diags. Gonna try pulling the top slot PCIe and putting it into slot 4 next, see what happens. There is no PCIe devices list or storage beyond the image I showed. Also, the NVME drive doesn't show up as a boot option in either UEFI or LEGACY+UEFI boot modes, which leads me to believe the BIOS doesn't even see it. beast-syslog-20180206-1920.zip beast-diagnostics-20180206-1921.zip EDIT2 - With a GPU in PCI_E1 (slot 1) and PCI_E4 (Slot 3) I get the error message below when starting the associated VM.. Feb 6 19:20:00 BEAST kernel: vfio-pci 0000:42:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem Feb 6 19:20:00 BEAST kernel: pcieport 0000:40:03.1: AER: Multiple Corrected error received: id=0000 Feb 6 19:20:00 BEAST kernel: pcieport 0000:40:03.1: PCIe Bus Error: severity=Corrected, type=Data Link Layer, id=4019(Transmitter ID) Feb 6 19:20:00 BEAST kernel: pcieport 0000:40:03.1: device [1022:1453] error status/mask=00001100/00006000 Feb 6 19:20:00 BEAST kernel: pcieport 0000:40:03.1: [ 8] RELAY_NUM Rollover Feb 6 19:20:00 BEAST kernel: pcieport 0000:40:03.1: [12] Replay Timer Timeout After I pulled the upper most GPU (960) from the machine. I still get full POST and unRAID startup log displayed to the screen even though the only other card is in slot 3. I also no longer get the above error at VM startup, but the vfio-pci 0000:0a:00.0: BAR 3: can't reserve [mem 0xe0000000-0xe1ffffff 64bit pref] error spams across the syslog when starting the associated VM.
  20. It’s interesting that you mention that. I took your idea and jumped down the rabbit hole with some interesting, albeit unfruitful results. I started looking into NVME / PCIE conflicts and such which lead me to try and update the firmware of the NVME drive. When I tried to load the USB it would kick out this error and stop the flash program. So just to sanity check it, I put the NVME back into my x99 rig to try an update it there. No issues, took 14.6 seconds and updated successfully. Also note that I tried the x399 without the M.2 card in, and still got the same errors starting a VM as previously. After updating the NVME I decided to put it back into the x399 trying a different M.2 slot. Making note that both times, UEFI BIOS didn’t recognize anything being in either M.2 slot 1 or 2. I know older BIOS’es would only display the NVME if it was in fact a SATA M.2. So it’s unclear at this point if this is normal behavior or not. But I found it odd that it’s stating “Not Present”. (EDIT/Update - MSI contacted me and notified me that this is normal behavior) When trying to load the VM that is passing through the upper slot GPU, I still get the original error, but noticed there is also a PCIE bus error above that (may or may not be in my previous diags). I saved the current log and diags from last night, but they are at home. I will attach those when I get home tonight. I also tried plugging in the addition PCIE bus power 6-pin connector for giggles, but that was also ineffective. When I get home tonight, I will try the system with no 2ndary GPU. Starting by leaving the card in PCIE slot 3 since that currently passes through no problem. If that works try using both cards, but in slots 3 and 4. (I’m not counting the small slots as they are unused). If that works, then try a single card in slots 1 or 2. I also haven’t tried out the 3rd M.2 slot, so I may try the NVME down there as well. The system is also running with only 3 DIMMs in, so I'll put in the 4th and make sure there isn't something screwy going on about that. I will probably also write MSI today and ask them about this. Am I correct in assuming that most people can now succefully pass through an nvidia GPU in the primarily slot, using Threadripper, and unRAID 6.4.1?
  21. Made a write up over in the 6.4.1 stable thread related to the below issue, but thought a recap would be better here since this thread seems more in line with the issue. So I built my Threadripper rig and initially had issues with gpu passthrough not working at all on 6.4. Luckily for me, 6.4.1 came out literally the same night. My lower slot GPU, regardless of which one I use (960 or 1060) works 100%. Can play games and stream over Steam In-home streaming with no issues on a Win10VM. However, the upper slot (slot 1 or 2, I havent tried 3 for this) defaults back to the same address regardless of which physical slot it is in. When you try to pass through that GPU (again, does not matter which one) you get the error below. " Feb 4 02:01:02 BEAST kernel: vfio-pci 0000:42:00.0: BAR 3: can't reserve [mem 0xc0000000-0xc1ffffff 64bit pref] " Sometimes the VM will come up but be locked into 640x480 or 800x600 res. Or it might just lockup the entire server. The only differences between this rig and the Home Server listed in my sig are: unRAID 6.4.1 / MB: MSI X399 SLI Plus - Latest BIOS update / CPU: 1920x TR4 / RAM: 24GB Kingston Non-ECC DDR4 (1 dimm out for preclearing a different system) / and the 1060 listed above. Is anyone else running into this? beast-diagnostics-20180204-0233.zip Edited yesterday at 03:15 AM by ryoko227 Adding diags and some more info
  22. Making a new rig with a 1920X Threadripper in it. The 6.4.1 update resolved the issue of not being able to pass through an Nvidia GPU at all, but having an issue with slot 1. I've tried a GTX 960 4GB and a GTX 1060 6GB in the first slot, using the roms I dumped from the respective GPUs. Both of these cards work in slot 3 with no issues, and no issues in the separate xeon build. When I start a Win10VM my syslog either spams this message " Feb 4 02:01:02 BEAST kernel: vfio-pci 0000:42:00.0: BAR 3: can't reserve [mem 0xc0000000-0xc1ffffff 64bit pref] " Or will pop that message once and hard lockup the server. The only differences between this rig and the Home Server listed in my sig are: unRAID 6.4.1 / MB: MSI X399 SLI Plus - Latest BIOS update / CPU: 1920x TR4 / RAM: 24GB Kingston Non-ECC DDR4 (1 dimm out for preclearing a different system) / and the 1060 listed above. Is anyone else running into this? EDIT: pulled another DIMM just incase, but same situation. Diags attached EDIT2: Thought I would be slick and just work around the issue by dropping the cards down to slot 2 and 3. Unfortunately, it then assigns the next highest slot as slot 1 and I get the exact same issues. At least that rules out it being a bad slot, which makes me happy. beast-diagnostics-20180204-0233.zip
  23. If you have 4 SSDs setup as a BTRFS RAID 0 in your cache pool and wanted to move that over. Would you have to keep the same serial drives as the same drive assignment? Or would unRAID be able to figure that one out too? I ask as I intend to do a server migration over the next week or so.
  24. Hiya OP, as many have already attested, this is not only viable, but for many of us the perfect solution for our computing needs. If you check out my sig, you'll see that I currently have 2 servers doing exactly what you are talking about. 1 at home and 1 at work (soon to be a 3rd at a new office site). The homeserver lets me do all my gaming (2 VMs worth at the same time), video and 3d work, while at the same time running an (admittedly overpowered) HTPC, plus dockers, plugins and pretty much anything else I want. The office server lets me run a media pc VM that I do all my work on (and sometimes play ) while keeping a multi-layered on-site/off-site backup and syncing schedule going for the coworkers. I use the spare cores for testing out new VMs, linux distros, etc. when I have the time and it gives me a really good feel of how they might run on a barebones system. I use both of these servers daily, and with how good unRAID has gotten, all of the awesome plugins/dockers produced by the community; I don't think I would or even could go back to a 1cpu=1pc type of setup. The office is passing through hdmi, mouse, keyboard directly to periferals and is kind of my daily driver. My home setup is a bit more niche, in that its generally headless and I usually stream the VM to my laptop. setup 1) direct out via hdmi to tv, mouse and keyboard pushed from laptop to VM via Synergy setup 2) VM streamed to old mint181kde laptop, via NoMachine (for desktop, blender, etc.) & Steam In-Home (for gaming) unRAID 6.4 is (for me at least) the highest performance, most stable version of the OS yet. I could even update W10 without the whole 1core work around now. So if you are thinking about pulling the trigger on it, I would. I have 0 regrets. Just, if you don't already have the hardware, it might be a poor time to buy, prices on gpus and memory are RIDICULOUS right now. Thanks my 2 yen, hope it helps!
  25. Worked like a charm! Thank you very much for your efforts and hard work, its very much appreciated!