January 17, 20188 yr 15 hours ago, Jcloud said: I do Techsupport, customer service, and I'm a sarcastic SOB; I can appreciate the concept of being compensated for a newb's barrage of questions. Past and future Thanks for all your contributions to the community, Jcloud!
January 18, 20188 yr Woohoo! I finally was able to use my 1080 ti that I've had for awhile.. I've never done any gpu passthrough before and naturally I chose the threadripper to do it on.. Drove me a little nuts for awhile.. For whatever reason, my gtx980 passes through and displays on Splashtop desktop, but my 1080ti, I was getting a black screen.. After countless hours of swapping cards around, dumping and downloading bios, restarts, and changing every setting I could think of, turns out that the 1080ti won't display on Splashtop but it works just fine through RDP. Not sure why one card worked but the other didn't.. Been going through the paces all day and everything seems good so far.. But anyway.. It works!! Thanks to everyone who made it possible..
January 21, 20188 yr So it turns out that the user-space Java program isn't all that complicated. I've translated it to python 2.7, which should be runnable in UNRAID without installing Java. Just paste this code into /mnt/cache/recover_pci.py, then run python /mnt/cache/recover_pci.py. Can someone try it out for me? By default it is read-only. If you'd like to try to recover the device, change DO_RECOVERY to True at the top. Caveat emptor! # This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public # License as published by the Free Software Foundation, version 3 of the License. # # This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied # warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. # # You should have received a copy of the GNU General Public License along with this program. If not, see # <http://www.gnu.org/licenses/>. # Try to recover Zen's PCIe bridge after a secondary bus reset kicks it out of whack. # Author: HyenaCheeseHeads # Porter: Adapted to Python by David Coppit 2017-01-21 # Ref: https://www.reddit.com/r/Amd/comments/7gp1z7/threadripper_kvm_gpu_passthru_testers_needed import os import sys import time from datetime import datetime #------------------------------------------------------------------------------- VFIO_DIR = '/sys/bus/pci/drivers/vfio-pci' BUS_BYTES = [0x22, 0x10, 0x53, 0x14] # For testing on my Intel setup #BUS_BYTES = [0x86, 0x80, 0x94, 0xa2] DO_RECOVERY = False #------------------------------------------------------------------------------- def log(text): # Not printing the timezone, since that can require non-standard libs print datetime.now().strftime('%a %b %d %H:%M:%S %Y: {}'.format(text)) #------------------------------------------------------------------------------- def compute_bridge_mapping(vfio_dir): bridge_mapping = {} if not os.path.isdir(VFIO_DIR): log("!!! Cannot find the VFIO-PCI sysfs at " + VFIO_DIR) sys.exit(-1) if not (os.access(VFIO_DIR, os.R_OK) and os.access(VFIO_DIR, os.W_OK)): log("!!! This tool requires R/W access to the VFIO-PCI sysfs at " + VFIO_DIR) log("!!! Make sure to run it as root (or similar super user)") sys.exit(-1) # Detect list of devices using VFIO-PCI sysfs entry log("Detecting VFIO-PCI devices") for f in os.listdir(VFIO_DIR): if not (os.path.isdir(os.path.join(VFIO_DIR, f)) and f.startswith("00")): continue device_path = os.path.realpath(os.path.join(VFIO_DIR, f)) log("\tDevice: " + device_path) bridge_path = os.path.dirname(device_path) bridge_config_path = os.path.join(bridge_path, 'config') if not os.path.isfile(bridge_config_path): log("\t\tBridge config does not exist for {}! Skipping...".format(bridge_path)) continue cf = open(bridge_config_path, 'rb') id = cf.read(4) cf.close() if len(id) != 4: log("\t\tCould not read bridge config {}!".format(bridge_config_path)) sys.exit(-1) if [ord(c) for c in id] == BUS_BYTES: log("\t\tBridge: " + bridge_path) bridge_mapping[device_path] = bridge_path else: log("\t\t!!! Unknown bridge type for {}! Skipping...".format(bridge_path)) return bridge_mapping #------------------------------------------------------------------------------- def device_is_offline(device_path): device_config_path = os.path.join(device_path, 'config') cf = open(device_config_path, 'rb') id = cf.read(4) cf.close() if len(id) != 4: log("\t\tCould not read device config {}!".format(device_config_path)) sys.exit(-1) return ord(id[0]) == -1 and ord(id[1]) == -1 #------------------------------------------------------------------------------- # Monitor devices for bridge failure pattern def monitor_devices(bridge_mapping): log("Monitoring {} device(s)...".format(len(bridge_mapping))) while True: for device_path, bridge_path in bridge_mapping.iteritems(): if not device_is_offline(device_path): continue # Failure detected, recover bridge by rewriting its config log("Lost contact with " + device_path) if not DO_RECOVERY: log("\tSkipping recovery...") time.sleep(2) continue try: bridge_config_path = os.path.join(device_path, 'config') cfin = open(bridge_config_path, 'rb') data = cfin.read() cfin.close() log("\tRecovering " + len(data) + " bytes") cfout = open(bridge_config_path, 'wb') cfout.write(data) cfout.close() log("\tBridge config write complete") except Exception as e: log("\t!!! Exception: {}".format(e)) Thread.sleep(10) if device_is_offline(device_path): log("\tFailed to recover bridge secondary bus") else: log("\tRecovered bridge secondary bus") log("Re-acquired contact with " + device_path) time.sleep(.1) #------------------------------------------------------------------------------- print '-------------------------------------------' print 'Zen PCIe-Bridge BAR/Config Recovery Tool, rev 1, 2018, HyenaCheeseHeads' print '[Adapted to Python by David Coppit 2017-01-21]' print '-------------------------------------------' bridge_mapping = compute_bridge_mapping(VFIO_DIR) monitor_devices(bridge_mapping)
January 22, 20188 yr Just chiming in that the modified kernel @Jcloud made worked for me, but I had to scrap my existing SeaBIOS VM and create one with OVMF. SeaBIOS VM had no video output, OVMF worked as expected.
January 25, 20188 yr From the "Anybody planning a Ryzen build?" thread, it looks like there's a proper kernel fix now: https://patchwork.kernel.org/patch/10181903/ I don't suppose Jcould would be willing to build another kernel for us?
January 25, 20188 yr 4 hours ago, coppit said: I don't suppose Jcould would be willing to build another kernel for us? This is quickly being reminiscent of my place of work -- do it once and suddenly the boss considers you the expert. But since you asked so nice, I'm willing to peak and poke at it. lol Edit: Again having patch issues five out of six globs didn't fuzz, or combine. So back to edit by hand; fix my errors. Compiler warning of function overloading and functions being declared but unused -- compiler treated these warnings as errors. Tried to ignore the errors and compile, but something went wrong 50/50 on me or the code. I'll take another crack at it later, sleep. Also, sorry I missed your post @coppit about the python script, I'll see what the weekend looks like. Edited January 25, 20188 yr by Jcloud
January 25, 20188 yr That's exactly what happened withe and the DVB stuff, a learning experiment that got seriously out of hand. Sent from my LG-H815 using Tapatalk
January 25, 20188 yr I just found out this thread and I am one on the frustrated people who spent a fortune on building a threadripper system only to find out that gpu passthrough is buggy as hell. I really appreciate the good work all of you guys are doing to make things work.
January 25, 20188 yr Thank you guys for your work! I just started on using UnRaid but already got some VM's up. I would like to use my GTX 1080 on one of them which is already lying on my desk for 1 1/2 months now Can somebody quickly describe how to use the bzimage file that has been uploaded by Jcloud? Any help would be appreciated.
January 25, 20188 yr 17 minutes ago, Symon said: Can somebody quickly describe how to use the bzimage file that has been uploaded by Jcloud? Any help would be appreciated. Super easy: just copy it on top of the one in your thumb drive. I name my old one bzimage.backup or something like that so I can restore easily if need be. Edited January 25, 20188 yr by Rhynri Corrected error
January 25, 20188 yr 33 minutes ago, Symon said: Can somebody quickly describe how to use the bzimage file that has been uploaded by Jcloud? Any help would be appreciated. See OP instructions, on page one, just use my file instead of Limetech's. It's same idea, mine is just a newer file for 6.4.0-STABLE. I over generalized: cd /boot mv bzimage bzimage-orig download my file. (next steps -- need to change your path for your system from example) cp /FolderWhereJcloudBzImageIsStored/bzimage-tr6_4_0 /boot/bzimage If I've over simplified, sorry, tainted by work making it lowest common denominator. Edited January 25, 20188 yr by Jcloud
January 25, 20188 yr 1 hour ago, limetech said: Looks like this is going to work: Would you happen to have the patch file used, or am I just being that "special," and messing up something perfectly functional? I could see this being plausible. If you don't have it, or the time - no worries, I'll keep bashing my head against the proverbial desk.
January 26, 20188 yr Author 1 hour ago, Jcloud said: Would you happen to have the patch file used, or am I just being that "special," and messing up something perfectly functional? I could see this being plausible. If you don't have it, or the time - no worries, I'll keep bashing my head against the proverbial desk. Another member here found it: Patch is from Alex Williamson, so it's going to work.
January 26, 20188 yr 4 hours ago, limetech said: Patch is from Alex Williamson, so it's going to work. Just looked at the link, that's the same one @coppit linked which I tried to do, so I'll chalk it up to, "I messed up, bad, somewhere." Adding more RAM to my TR, and then I'll give it another shot afterwards. EDIT0: Finally got my s*** together, and found my errors. For your testing pleasure or sadism here is the current bzImage with proposed patch. Remember this is use at your risk and/or head pounding on desk (but hopefully not): 5e635ca335a2011dcb349326e15e151f bzimage-tr4_6_0b I just booted up on it and so far so good. bzimage-tr4_6_0b EDIT1: I still couldn't get it to .patch file over, and had to edit pci.c -- I'm guessing because I'm not looking at the same version of pci.c as author? Again, knowledge-gaps. My point, if it's of use to devs I've also included a copy of my modified pci.c. And with that, everyone have a good evening. pci_c-patch10181903 Edited January 26, 20188 yr by Jcloud Finally have the goods
January 26, 20188 yr 5 hours ago, Jcloud said: Just looked at the link, that's the same one @coppit linked which I tried to do, so I'll chalk it up to, "I messed up, bad, somewhere." Adding more RAM to my TR, and then I'll give it another shot afterwards. EDIT0: Finally got my s*** together, and found my errors. For your testing pleasure or sadism here is the current bzImage with proposed patch. Remember this is use at your risk and/or head pounding on desk (but hopefully not): 5e635ca335a2011dcb349326e15e151f bzimage-tr4_6_0b I just booted up on it and so far so good. bzimage-tr4_6_0b EDIT1: I still couldn't get it to .patch file over, and had to edit pci.c -- I'm guessing because I'm not looking at the same version of pci.c as author? Again, knowledge-gaps. My point, if it's of use to devs I've also included a copy of my modified pci.c. And with that, everyone have a good evening. pci_c-patch10181903 the bzimage provided by you successfully worked for me. thanks alot . TR 1950x on asus zenith x399
January 26, 20188 yr Quote 11 hours ago, Rhynri said: Super easy: just copy it on top of the one in your thumb drive. I name my old one bzimage.backup or something like that so I can restore easily if need be. 11 hours ago, Jcloud said: See OP instructions, on page one, just use my file instead of Limetech's. It's same idea, mine is just a newer file for 6.4.0-STABLE. I over generalized: cd /boot mv bzimage bzimage-orig download my file. (next steps -- need to change your path for your system from example) cp /FolderWhereJcloudBzImageIsStored/bzimage-tr6_4_0 /boot/bzimage If I've over simplified, sorry, tainted by work making it lowest common denominator. Thank you for your help I will try it this evenening !
January 27, 20188 yr Just tried @Jcloud's new kernel (6.4.0b) and it's a no go for me. It won't boot from the unraid boot loader.
January 27, 20188 yr Author The 6.4.1-rc1 release includes a patch that should solve Threadripper GPU passthrough issue. Please give it a try:
January 27, 20188 yr On 26.1.2018 at 3:25 AM, Jcloud said: Just looked at the link, that's the same one @coppit linked which I tried to do, so I'll chalk it up to, "I messed up, bad, somewhere." Adding more RAM to my TR, and then I'll give it another shot afterwards. EDIT0: Finally got my s*** together, and found my errors. For your testing pleasure or sadism here is the current bzImage with proposed patch. Remember this is use at your risk and/or head pounding on desk (but hopefully not): 5e635ca335a2011dcb349326e15e151f bzimage-tr4_6_0b I just booted up on it and so far so good. bzimage-tr4_6_0b EDIT1: I still couldn't get it to .patch file over, and had to edit pci.c -- I'm guessing because I'm not looking at the same version of pci.c as author? Again, knowledge-gaps. My point, if it's of use to devs I've also included a copy of my modified pci.c. And with that, everyone have a good evening. pci_c-patch10181903 This new patch worked for me as well! Thanky you! Asus Rog Zenith GTX 1080
January 29, 20188 yr Can confirm video passthrough works with Threadripper on 6.4.1rc1. Passmark Performance Test still freezes the VM before even running a benchmark, but games seem to work fine.
February 5, 20188 yr On 1/31/2018 at 10:13 AM, avenger93 said: 1950x with asus zenith gpu passthrough working on 6.4.1rc1 "Throw that Bates 4000 away, it's a piece of shit." -- Onion the Movie. (moved to "stable" if you care) Edited February 5, 20188 yr by Jcloud Got to get the joke right.
Archived
This topic is now archived and is closed to further replies.