jonp

OMMU Groups /sys/kernel/iommu_groups/0/devices/0000:00:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.1 /sys/kernel/iommu_groups/1/devices/0000:01:00.0 /sys/kernel/iommu_groups/1/devices/0000:02:00.0 /sys/kernel/iommu_groups/1/devices/0000:02:00.1 /sys/kernel/iommu_groups/2/devices/0000:00:14.0 /sys/kernel/iommu_groups/3/devices/0000:00:16.0 /sys/kernel/iommu_groups/3/devices/0000:00:16.3 /sys/kernel/iommu_groups/4/devices/0000:00:19.0 /sys/kernel/iommu_groups/5/devices/0000:00:1a.0 /sys/kernel/iommu_groups/6/devices/0000:00:1b.0 /sys/kernel/iommu_groups/7/devices/0000:00:1c.0 /sys/kernel/iommu_groups/8/devices/0000:00:1c.1 /sys/kernel/iommu_groups/9/devices/0000:00:1c.3 /sys/kernel/iommu_groups/10/devices/0000:00:1c.5 /sys/kernel/iommu_groups/11/devices/0000:00:1c.6 /sys/kernel/iommu_groups/12/devices/0000:00:1c.7 /sys/kernel/iommu_groups/13/devices/0000:00:1d.0 /sys/kernel/iommu_groups/14/devices/0000:00:1f.0 /sys/kernel/iommu_groups/14/devices/0000:00:1f.2 /sys/kernel/iommu_groups/14/devices/0000:00:1f.3 /sys/kernel/iommu_groups/14/devices/0000:00:1f.6 /sys/kernel/iommu_groups/15/devices/0000:04:00.0 /sys/kernel/iommu_groups/16/devices/0000:05:00.0 /sys/kernel/iommu_groups/17/devices/0000:06:00.0 /sys/kernel/iommu_groups/17/devices/0000:07:03.0 /sys/kernel/iommu_groups/18/devices/0000:08:00.0 /sys/kernel/iommu_groups/19/devices/0000:09:00.0 Ok, what is device 01:00.0? Look at PCI Devices at the top of that Tools -> System Devices page, and let me know which device that is. It's in the same IOMMU group as your graphics card. This can be overcome by turning on the PCI ACS Override setting from the Settings -> VM Manager page (you will need to reboot your system after applying this change).

Sorry you couldn't get it working right for your needs. With respect to USB support, there's another way to guarantee easier / more natural USB support, which is to purchase a USB controller as a PCIe card, add it to unRAID, then assign that card to the VM as a PCI device with another custom string of xml. This would give you full hot-plugging support and all devices attached should work perfectly. USB assignment is tricky when it comes to supporting a wide variety of devices. It's getting better and better, but it's still got a ways to go to work universally with all device types. USB controller pass through, however, is very solid, and can be very advantageous in giving you a more "native" experience. As far as storage performance goes, I'm wondering how you went about getting the files onto the array from within the VM in the first place. If you downloaded a file inside your VM, then copied it to the array from the VM, I wouldn't expect much improvement, as you're going through a virtual disk image layer. You could potentially improve the performance there by passing through an entire block device (or perhaps even another add-on storage controller that is dedicated to the VM). It also depends on the file systems you're using, how full those devices are, and a number of other conditions. Lastly, Windows 7 is the worst of the currently supported Windows operating systems when it comes to performance as a virtual machine. Windows 8/8.1/10 are far better performing as a VM. Windows 7, while a tried and true OS, just doesn't have the optimizations that the later iterations have for running as a VM. Does it work? Yes. Does it work fairly good? Yeah, it does. Is it as good as 8.1 or 10? Not by a longshot IMHO. And the virtual USB 3 controller I gave you as XML doesn't work with Windows 7, I know that. It seems to work solid in 8.1/10 though, and even with USB cameras, although I am debugging some issues still there. Right now, the primary use cases for VMs on unRAID are pretty simple and direct: 1- replace a desktop PC / gaming rig with a NAS+Desktop combo system, where the NAS functionality rides at a level above the desktop, so desktop issues don't cause NAS issues. 2 - run headless servers for Windows or other applications that aren't available/stable enough in Docker Containers. 3 - build a media player / NAS combo with OpenELEC There are obvious other things you can do with virtualization beyond that, but we haven't fully implemented them all just yet, so this may be a case of expecting more from us than we're ready to deliver. That said, for folks willing to work with us a bit, we can find ways through XML edits to deliver functionality, but again, Windows 7 can be a major drawback for us at times. That said, I appreciate you giving it a shot and some time to work through your issues. I do understand the need to move on though, but if you ever decide to give it another shot, we're here to help.

Can you go to Tools -> System Devices and copy / paste the IOMMU groups displayed there into the forum here?

Ok, you can delete the address line when editing the XML, it will auto adjust. You can also try changing the bus from IDE to SATA. Another tweak for your USB devices would be to adjust the USB controller in the XML by adding the following to the <controller> element: model='nec-xhci'

You can try 0.1.96 if its still up there, but here's a fall back method. Ok, let's do this just to get you going, since there is clearly something weird going on. Edit the XML of your VM. Locate the <disk> section and look for the <target> part. Change the bus from virtio to ide. This will use an emulated storage controller. Windows should install without prompting for any drivers. I still think this has to be an issue with old installation media or something.

Tbh, this isn't "too" guinea pigish (did I just make that a word?), as I've set up VMs like this before. I know it works, but the device to the guest OS will simply show up as a generic qemu/virtio storage device. With the virtio-scsi controller setup, it's only emulating the storage controller, but the physical device is assigned, so it's a more native storage device experience. Does this impact performance? Maybe, but I honestly don't know because I haven't run benchmarks yet.

Not sure of they should work because I don't have your install media. Try the older ones I mentioned in looking through the archives.

This has nothing to do with your hardware. These are virtual device drivers you are loading. Almost guaranteed its something to do with the install media. What version of the virtio drivers did you download? If you follow the link to the fedora project, you can access the virtio driver archive and try to install 0.1.100 or 0.1.96 and see if that works for you.

Where did you obtain your installation media for Windows 7? If the installer is not giving you an option to load the x64 drivers, it may be because your installer is a 32-bit installer only. I had the exact same problem. Win7 64 bit wouldn't work with the AMD64 drivers. Win7 32 bit would work with the x86 drivers to install. Since I was only going to give the VM a maximum of 4 gig anyways, I was actually better off installing as 32 bit anyways, so I didn't particularily care. FWIW I was using a geniune an ISO created from a genuine Windows Ultimate 64 and a genuine Windows Ultimate 32. Using the 32 bit OS and drivers the system installed and ran fine. 64 bit OS and 64 bit drivers wouldn't find the drive at all. Was just doing it as an experiment. My useage case on the servers, etc doesn't require any form of VM at all. Curious if this was older (pre-SP1) install media.

JonP, this is rather worrisome. Those SCSI ID's can change, just as much as the sd symbols and the ata numbers. It's true that some systems seem to be more 'stable', especially small systems with no added disk controllers, but otherwise, I can't see how hard coding the SCSI ID in will work across boots. It will at times, but not reliably. A user could reboot and it would stop working, then reboot again, and it would start working, if that session happened to match the same lsscsi list. If you have multiple diagnostics zips available, test this yourself, compare the lsscsi.txt files. I personally haven't had this issue, but yes, some may. This is why we haven't put it formally into the webGui yet. Eric and I are working on a VM device manager for the webGui. The goal here is to manage device assignments differently, where a user could specify a device for use with a VM, then we would do the legwork at runtime to determine the appropriate changes to XML for VMs. E.g., if the SCSI address changed, the /dev/disk/by-id path won't, and we can use something like udevadm to trace back to the actually SCSI ID. There is another method for passing through a block device as well which we are experimenting with, and doesn't use SCSI IDs. The difference in methods could impact performance though. It should be noted that whenever I provide these guides, they are a temporary way to establish a level of functionality that isn't supported officially in the webGui yet. I want to hear from folks experiences in using these methods so we can adjust our formal support implementation path. Baked in support would be awesome. Currently, when I reboot unRAID for whatever reason(new build here so im doing quite a lot right now), I have to go in a change the address, as the device address seems to change every reboot. Always changes between the same 2 numbers, oddly enough. Even changes when I don't touch my drives during the reboot/power off. Also, as far as performance or issues, I am not positive this is unRAID/KVM/passthrough related yet, but my SABnzbD downloads are mostly failing. WinRAR unpack issues, "unexpected end of archive". Have even downloaded some NZB's which I had recently grabbed before making the unRAIR leap. Going to install on my desktop and test it out to see what might be up. When I first booted the VM up with the disk passed through, I formatted, copied some files to/from to test, and windows gave me a filesystem corruption error(didnt jot it down). Ran it through a chkdsk and have not had that error since, though still have the odd unpack issues. Ok, if you'd like to be a guinea pig, here's another method for passing through a block device that does not utilize the virtio-scsi controller. Changing your existing VM to this may possibly not work (you may have to reinstall windows or try a few things to get it to boot again). 1 - Remove all the custom xml stuff I had you add originally. 2 - Add a new <disk> section that looks like this: To find what to put in the highlighted portion, you will need to login to your server from command line. Then type the following command: ls /dev/disk/by-id Find the one that represents your storage device (but without the -part1 at the end) and paste that in the section above that says YOUR DISK ID HERE. This should work regardless of what SCSI IDs change to after reboots.

Where did you obtain your installation media for Windows 7? If the installer is not giving you an option to load the x64 drivers, it may be because your installer is a 32-bit installer only.

JonP, this is rather worrisome. Those SCSI ID's can change, just as much as the sd symbols and the ata numbers. It's true that some systems seem to be more 'stable', especially small systems with no added disk controllers, but otherwise, I can't see how hard coding the SCSI ID in will work across boots. It will at times, but not reliably. A user could reboot and it would stop working, then reboot again, and it would start working, if that session happened to match the same lsscsi list. If you have multiple diagnostics zips available, test this yourself, compare the lsscsi.txt files. I personally haven't had this issue, but yes, some may. This is why we haven't put it formally into the webGui yet. Eric and I are working on a VM device manager for the webGui. The goal here is to manage device assignments differently, where a user could specify a device for use with a VM, then we would do the legwork at runtime to determine the appropriate changes to XML for VMs. E.g., if the SCSI address changed, the /dev/disk/by-id path won't, and we can use something like udevadm to trace back to the actually SCSI ID. There is another method for passing through a block device as well which we are experimenting with, and doesn't use SCSI IDs. The difference in methods could impact performance though. It should be noted that whenever I provide these guides, they are a temporary way to establish a level of functionality that isn't supported officially in the webGui yet. I want to hear from folks experiences in using these methods so we can adjust our formal support implementation path.

Well, an on-board network interface belongs to an Ethernet controller, which is a PCI device, even if it's on the mobo (on-board devices are still using the PCI bus). That said, you got me thinking and I am curious if we can do interface "assignment" without controller passthrough. I'll have to research this...

Ok, so to pass through a full block device to a VM, here's what you need to do. First, you need to identify the device you wish to do this with. You can find a listing of your storage devices in the webGui under Tools -> System Devices -> SCSI Devices. Once identified, note the SCSI device id in the first column. Here's an example from one of my test systems. [1:0:0:0] disk PNY USB 3.0 FD 1.00 /dev/sdb [3:0:0:0] disk ATA WDC WD20EARX-00P AB51 /dev/sdc [5:0:0:0] disk ATA ST4000DM000-1F21 CC52 /dev/sdd [6:0:0:0] disk ATA OCZ-VERTEX3 2.25 /dev/sde [8:0:0:0] disk ATA ST4000DM000-1F21 CC52 /dev/sdf [9:0:0:0] disk ATA WDC WD20EARX-00P AB51 /dev/sdg [10:0:0:0] disk ATA ST2000DL003-9VT1 CC3C /dev/sdh [11:0:0:0] disk ATA WDC WD20EARX-00P AB51 /dev/sdi The first column is the scsi ID. If I wanted to assign the /dev/sdi device (a western digital device), I would need to note the 11:0:0:0 designation. Next, I need to edit the XML for my VM. Goto the VMs tab, click on the icon for the VM we're dealing with, and then select Edit XML. NOTE: Once you do this, you will not be able to use the GUI editor to make changes to your VM without losing these edits, so before proceeding, use the GUI editor to get your memory, CPU pinning, and any other device assignments all set. In the XML, add the following code: Adjust the large bolded numbers for the four numbers in the scsi id. For most, this will only be the first number in the <adapter> section. If you are installing Windows directly to this device, you will need to load the vioscsi driver during the installation process for the installer to detect the device. Follow the same procedure as is documented in the wiki for setting up a normal Windows VM, but instead of selecting viostor, browse to the vioscsi folder. Let me know if any of this is unclear or if this doesn't work for you.

There is actually a way to pass through an entire block-level device to unRAID a virtual machine (not using a virtual disk image file at all). I'll have to write this up in a reply for you a little later today.

With unRAID 6.1, we have added official support for our OpenELEC virtual machine template. For a full set of instructions on how to download and use the VM, please see this section of the VM Guest Support wiki. This thread is for requesting support in the event graphics, sound, or other aspects of the VM aren't working properly.

If you have created a plugin for unRAID 6, please see this post about the changes required for compatibility with 6.1. If you have already made the changes necessary and tested to confirm your plugin works with 6.1, you can send a PM to a moderator (or myself) to have the thread moved from the Unverified forum to the Verified forum.

This forum is for Community Developers to be able to post new support threads for their repos or containers that are templatized for use with unRAID. Note: While only Community Developers can create new topics here, all users can reply to them. 1) You earn the status of Community Developer by not only creating containers/integrating them with CA, but by showing your commitment to providing support for them. 2) You can create a post for support for your container anywhere and then simply PM a moderator to move it to the docker containers forum. 3) After sufficient time contributing, you can request official Community Developer status by contacting support.

Let's make this simple and easy: [*]Keep it clean and be civil. Healthy debates are good, flame wars are not. [*]No links to illegal or malicious content of any kind. Breaking rule #1 may result in your post being bilged and a warning issued. Breaking rule #2 will result in your post being deleted and your account being banned.

Already saw the tentative changes to the templates (when the test templates caused havoc with the appfeed, which Kode then had to account for). Put two and two together and figured out the rest. I was impressed and think that it will be a huge boon for unRaid as a whole, and a welcome relief for everyone using containers that continually has those same problems. And a welcome relief for people trying to support everyone using containers. I get the impression there are a lot of docker users who still don't really get the whole mapping thing, but somehow got it to work by copying from others. I agree. for users, docker is one of those puzzles that once you truly understand it, its super easy, but getting to that point of understanding can be a trying experience. If we would have renamed volume mappings to share shortcuts, we probably could have saved a lot of that headache, but who knows at this point. The goal for us longer term to to make all the tech transparent to users. In the end, users want an APP, not a container, VM template, or even a plugin. Those elements are just different delivery vehicles for apps. Other than differences in hw requirements, why should they care? The point is they shouldn't have to. First phase towards this goal is to iterate what we have so its easier to work with the tools we have made. Longer term, we want our unify app management in a single manager.

As long as the OT banter doesn't have to stop Saves me from continually talking to myself in here. Sorry about that. All because I'm not quite done yet on this... And this is why I'm never quite done Always another good idea. I have a thought of how to accomplish this without imposing another schema change upon the authors, at least initially. (And maybe my thought will work well enough that it won't be necessary either) The original version of the appFeed actually had total downloads included in it per container. So it was at least possible (Not sure if it still is since the revamp of dockerHub however - Kode would know). But, personally I'm not a big fan of total downloads as a rating system as it would unfairly rate the older containers higher than the newer ones (ie: needo's / binhex's plex would blow away linuxserver's / LT's out of the water, and for all I know linuxserver / LT has the better implementation). Couple of threads buried within Docker Categorize, Community Repositories about this topic) My personal preference is for a rating system of sorts, but open to suggestions on everything. Either way, give me a week and I'll get the situation with Plex, CP, Sab, etc sorted out. Always open to your ideas squidly. I'll let you have at it. You're going to love what we are working on behind the scenes. Let's just say we are working on making it so users don't have to specify any volume mappings, port mappings, or anything for that matter. Getting pretty close to a true mobile app store type experience. Stay tuned.

We do want to incorporate and I've been in discussions with squid on this, but we need to make some other changes first. We also need to get away from having multiple copies of the same app showing up as its just confusing to users. That's a tough problem to solve, but I've been thinking we could adjust the XML schema for this to make it work where each container has an <appname> field that's different from the name of the container. E.g. binhex-plex would be Plex Media Server. So would anyone else's Plex container. Then we display Plex once on the community apps plugin, but when the user goes to add, they can select which container variant they wish to use. The tricky part here would be what the default should be. Might need to set up some analytics so we can measure which variant is most popular based on total downloads and promote that as the default, but that's a little more sophisticated.

Just an FYI, I'm stickying this thread.

This thread is awesome! I need to write a blog post featuring this after we release 6.1. Keep it up guys!!

This is a great request and one that we want to do. There may be a shorter-term solution for you as there is a libvirt-compatible mobile app that can do what you want, but we'd have to configure libvirt on unRAID to allow connections from it.