Passthrough an entire SSD directly to a windows VM?


Recommended Posts

So I'm mainly a gamer but I was inspired recently with nVidia's latest "GPU in the cloud" GRID service and the overall state of streaming games/3d over a network and it got me thinking about doing something similar on my PC on my home network.  My end goal is to be able to split my gaming desktop into two VMs each with a GPU and then my father could play video games with me on my PC either by hooking up a monitor/mouse/keyboard or by using nvidia gamestream to his laptop.  I watched LTT do the 2 gamers 1 PC video and the 7 gamers 1 PC video and decided I had to try it out and I've had great success setting up a Win10 VM on a vdisk on an unassigned drive and passing through the GPU and USB controller.  But it's still a vdisk residing on a XFS partition  and performance is not the best it could be and is a little inconsistent. I am not sure if this is possible but I would like to passthrough a Samsung Evo 250GB SSD to a Windows 10 VM directly. I would like to have it be recognized directly as a Samsung SSD and not a virtual disk. I'd like the disk to be NTFS and if I can install Samsung Magician on the VM and recognize my SSD I'll be happy. I do not mind if unRAID cannot even see the Samsung, in fact I'd prefer it.

 

 

My hardware specs are as follows:

i7 4790k, 32GB DDR3, 2x GTX 970(1 is currently out for RMA), Asrock Z97 Extreme 4 and for storage, 1x 2TB WD HDD data drive, 1x 240GB Kingston SSD data drive, 1x 480GB PNY SSD as Cache and 1x 250GB Samsung 850 EVO unassigned.

 

I do not have any parity right now, although I have ordered more drives and will remove the kingston ssd at that time.

I have plugged all my array/unRAID drives in to the 4-port Intel Z97 Sata controller on my motherboard and I have an ASM1062 sata controller built into my motherboard with 4 SATA ports (nothing connected to them though currently, although in the future I'd like to use them) and I also purchased an add-in ASM1062 PCI-e 2 SATA port controller that I've connected the Samsung to. I've successfully passed through both my GPU and my motherboard's 4-port Intel USB controller with my peripherals connected (keyboard, mouse, USB DAC) and unRAID is running off a 32GB USB 3 drive connected to the ASM1042A USB controller built into my motherboard.

 

 

My next step is I'm thinking I may switch my array disks to the built in 4 port ASM1062 and try to passthrough the Intel Z97 controller instead to the VM since I've had good luck with the Z97 USB passthrough. But I'd like to add more drives to the array in the future and I only need 1 port for the Samsung so ideally I'd like to passthrough the 2 port PCIe ASM1062 and keep the builtin 4 port ASM1062 assigned to unraid. Should I buy a different PCIe SATA controller (that isn't the same as 1 on my motherboard) if that makes things easier? If I do that, what controller card is best?

 

 

TL;DR: I wish to passthrough a Samsung 850 EVO to a Windows 10 VM in it's entirety and be recognized as a Samsung 850 Evo in the Windows Device Manager and not a virtual disk. I do not need unraid to be able to access the drive in any way shape or form and would prefer if unraid has no access to it.  I run all my other VMs off my cache drive but I'd like my gaming VM to have the absolute best performance possible with it's own drive. How can I do that? If I need a different SATA controller card I'll buy it or whatever I may need to get this to work.

 

I've listed all the relevant info I can think of below:

 

00:00.0 Host bridge [0600]: Intel Corporation 4th Gen Core Processor DRAM Controller [8086:0c00] (rev 06)

00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)

00:02.0 VGA compatible controller [0300]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller [8086:0412] (rev 06)

00:14.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB xHCI Controller [8086:8cb1]

00:16.0 Communication controller [0780]: Intel Corporation 9 Series Chipset Family ME Interface #1 [8086:8cba]

00:19.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I218-V [8086:15a1]

00:1a.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB EHCI Controller #2 [8086:8cad]

00:1c.0 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 1 [8086:8c90] (rev d0)

00:1c.3 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 4 [8086:8c96] (rev d0)

00:1c.5 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 6 [8086:8c9a] (rev d0)

00:1c.6 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 7 [8086:8c9c] (rev d0)

00:1d.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB EHCI Controller #1 [8086:8ca6]

00:1f.0 ISA bridge [0601]: Intel Corporation 9 Series Chipset Family Z97 LPC Controller [8086:8cc4]

00:1f.2 SATA controller [0106]: Intel Corporation 9 Series Chipset Family SATA Controller [AHCI Mode] [8086:8c82]

00:1f.3 SMBus [0c05]: Intel Corporation 9 Series Chipset Family SMBus Controller [8086:8ca2]

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)

01:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)

03:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)

04:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)

05:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller [1b21:1142]

 

 

/sys/kernel/iommu_groups/0/devices/0000:00:00.0

/sys/kernel/iommu_groups/1/devices/0000:00:01.0

/sys/kernel/iommu_groups/1/devices/0000:01:00.0

/sys/kernel/iommu_groups/1/devices/0000:01:00.1

/sys/kernel/iommu_groups/2/devices/0000:00:02.0

/sys/kernel/iommu_groups/3/devices/0000:00:14.0

/sys/kernel/iommu_groups/4/devices/0000:00:16.0

/sys/kernel/iommu_groups/5/devices/0000:00:19.0

/sys/kernel/iommu_groups/6/devices/0000:00:1a.0

/sys/kernel/iommu_groups/7/devices/0000:00:1c.0

/sys/kernel/iommu_groups/8/devices/0000:00:1c.3

/sys/kernel/iommu_groups/9/devices/0000:00:1c.5

/sys/kernel/iommu_groups/10/devices/0000:00:1c.6

/sys/kernel/iommu_groups/11/devices/0000:00:1d.0

/sys/kernel/iommu_groups/12/devices/0000:00:1f.0

/sys/kernel/iommu_groups/12/devices/0000:00:1f.2

/sys/kernel/iommu_groups/12/devices/0000:00:1f.3

/sys/kernel/iommu_groups/13/devices/0000:03:00.0

/sys/kernel/iommu_groups/14/devices/0000:04:00.0

/sys/kernel/iommu_groups/15/devices/0000:05:00.0

[0:0:0:0] disk Lexar USB Flash Drive 1100 /dev/sda 32.0GB

[1:0:0:0] disk ATA KINGSTON SUV400S 96R9 /dev/sdb 240GB

[2:0:0:0] disk ATA WDC WD20EZRX-00D 0A80 /dev/sdc 2.00TB

[3:0:0:0] disk ATA SSD2SC480G1CS175 1101 /dev/sdd 480GB

[33:0:0:0] disk ATA Samsung SSD 850 2B6Q /dev/sdf 250GB

ls -l /dev/disk/by-id total 0

 

 

lrwxrwxrwx 1 root root 9 Jan 16 19:11 ata-KINGSTON_SUV400S37240G_50026B766A006046 -> ../../sdb

lrwxrwxrwx 1 root root 10 Jan 16 19:11 ata-KINGSTON_SUV400S37240G_50026B766A006046-part1 -> ../../sdb1

lrwxrwxrwx 1 root root 9 Jan 16 19:11 ata-SSD2SC480G1CS1754D117-514_PNY43150000099990399 -> ../../sdd

lrwxrwxrwx 1 root root 10 Jan 16 19:11 ata-SSD2SC480G1CS1754D117-514_PNY43150000099990399-part1 -> ../../sdd1

lrwxrwxrwx 1 root root 9 Jan 16 19:38 ata-Samsung_SSD_850_EVO_250GB_S21NNSAG455990F -> ../../sde

lrwxrwxrwx 1 root root 10 Jan 16 19:38 ata-Samsung_SSD_850_EVO_250GB_S21NNSAG455990F-part1 -> ../../sde1

lrwxrwxrwx 1 root root 10 Jan 16 19:38 ata-Samsung_SSD_850_EVO_250GB_S21NNSAG455990F-part2 -> ../../sde2

lrwxrwxrwx 1 root root 10 Jan 16 19:38 ata-Samsung_SSD_850_EVO_250GB_S21NNSAG455990F-part3 -> ../../sde3

lrwxrwxrwx 1 root root 10 Jan 16 19:38 ata-Samsung_SSD_850_EVO_250GB_S21NNSAG455990F-part4 -> ../../sde4

lrwxrwxrwx 1 root root 9 Jan 16 19:11 ata-WDC_WD20EZRX-00D8PB0_WD-WCC4M1534466 -> ../../sdc

lrwxrwxrwx 1 root root 10 Jan 16 19:11 ata-WDC_WD20EZRX-00D8PB0_WD-WCC4M1534466-part1 -> ../../sdc1

lrwxrwxrwx 1 root root 9 Jan 16 19:11 usb-Lexar_USB_Flash_Drive_AA96DJ30BZACPDQZ-0:0 -> ../../sda

lrwxrwxrwx 1 root root 10 Jan 16 19:11 usb-Lexar_USB_Flash_Drive_AA96DJ30BZACPDQZ-0:0-part1 -> ../../sda1

lrwxrwxrwx 1 root root 9 Jan 16 19:38 wwn-0x15333806917760667650x -> ../../sde

lrwxrwxrwx 1 root root 10 Jan 16 19:38 wwn-0x15333806917760667650x-part1 -> ../../sde1

lrwxrwxrwx 1 root root 10 Jan 16 19:38 wwn-0x15333806917760667650x-part2 -> ../../sde2

lrwxrwxrwx 1 root root 10 Jan 16 19:38 wwn-0x15333806917760667650x-part3 -> ../../sde3

lrwxrwxrwx 1 root root 10 Jan 16 19:38 wwn-0x15333806917760667650x-part4 -> ../../sde4

lrwxrwxrwx 1 root root 9 Jan 16 19:11 wwn-0x259297387829550989x -> ../../sdd

lrwxrwxrwx 1 root root 10 Jan 16 19:11 wwn-0x259297387829550989x-part1 -> ../../sdd1

lrwxrwxrwx 1 root root 9 Jan 16 19:11 wwn-0x4839972860202209281x -> ../../sdc

lrwxrwxrwx 1 root root 10 Jan 16 19:11 wwn-0x4839972860202209281x-part1 -> ../../sdc1

Link to comment

It's been a good day so here you go

 

This is what my pass-through command looks like

/dev/disk/by-id/ata-Crucial_CT525MX300SSD1_163313AAD0A9

 

Also edit your VM templet and make sure your Primary vDisk bus is set to SATA this will let the Windows install see the physical drive.

 

Screenshot included

 

Capture.PNG.eac21bbb122a7203a9324fd7d4daa570.PNG

  • Like 1
Link to comment

Awesome, that picture cleared it right up.  I had tried the /dev/disk/by-id but I was using virtio as the bus.  I'll try it was sata as soon as I get home. I'm sure this question gets asked a lot around here, but I'm glad someone was here to help.  I may be a VM noob right now but this unRAID stuff is blowing my mind and teaching me tons of stuff.  I'll try to pay it forward if/when I can!

Link to comment
  • 1 year later...
  • 10 months later...
11 hours ago, steve1977 said:

Did you succeed to get this working? I am interested in the same for the same reason.

 

What's the easiest to install Windows directly on my SSD (not as vdisk). I tried above, but it still asks me for a vdisk size and still creates a vdisk.

The "easiest" way to pass through an entire SSD is to have a NVMe ssd. There are posts all over about it on the forum and if your motherboard had seperate IMMOU groups for every NVMe it works like a charm.

 

I have a setup where if I don't boot to my unraid USB it boots from my primary NVMe drive right into windows with my 2nd video card disabled for power saving, if I restart and boot from my unraid USB it starts up two windows VMs one of which is literally the exact same NVMe drive I run windows off without virtualization.

Link to comment
  • 7 months later...

The above replies contained 2 solutions for 2 problems i had figuring out/playing with unraid.
-The /disk/by-id/  solution

-The info on the NVMe's

 

Note: Even with 2 NVMe's on your desktop motherboard in the same IOMMU group (shown seperate)

but with the same ID you use for blacklisting. I still got one 100% passedthrough and shown as the

real device in taskmanager, while the other NVMe is used as cache in my Array.

 

I use a 1Tb and 2Tb Samsung 970 Evo Plus on a ROG MAXIMUS XI HERO (WI-FI), Version Rev 1.xx

 

It is all possible, only thing NOT possible is for example on my motherboard.

To split up the 6 SATA ports on the mobo so you can passthrough 1 port so the TaskManager on

the VM will see the real Device Name. That only works with NVMe('s), for example my mobo.

 

But if you follow the disk/by-id/ata- solutions given, it will be detected on a new VM without

loading drivers to see the disk. The performance is good (SET it in unraid Menu at SATA device),

but it will be seen as QEMU HARDDISK. The NVM's will say the real name like on a BareMetal situation.

Link to comment
  • 1 month later...
On 7/16/2019 at 1:32 AM, FlyingPenguin said:

The "easiest" way to pass through an entire SSD is to have a NVMe ssd. There are posts all over about it on the forum and if your motherboard had seperate IMMOU groups for every NVMe it works like a charm.

 

I have a setup where if I don't boot to my unraid USB it boots from my primary NVMe drive right into windows with my 2nd video card disabled for power saving, if I restart and boot from my unraid USB it starts up two windows VMs one of which is literally the exact same NVMe drive I run windows off without virtualization.

Thanks. This is very cool. Let me try the same. So, you basically don't do anything then unplugging your Unraid USB and then it boots directly into your VM? I'd love to do the same. A few questions:

 

1) If passing through an NVME (on mobo or in IDE card), do I still select SATA instead of Virtio?

 

2) If passing through a seconary disk as well, can this somehow be detected when booting directly into the primary bootdisk?

 

3) Any Windows registration issue as the hardware will be different when booting directly vs. booting as VM?

 

4) How did you succeed to disable your 2nd GPU? In the "normal" case, Windows would just have access to everything, right? I.e., all USB ports, all GPUs, etc.

 

5) Does this also work for a Mac VM?

 

This is really cool and I had no idea that this would work.

Link to comment
6 minutes ago, trurl said:

Of course you won't have access to any of your Unraid storage when Unraid isn't booted.

Not to the array, but I could still access the disks? I should see them in disk manager?

 

I could not start dockers, so that's very unfortunate.

 

And of course, this would not be a viable option for good. But it could be an option if I need to make some changes to my RGB fans (I currently don't have a way to turn off the light). Also, I could use it for gaming at full baremetal performance (using all cores and all memory with single overhead). I don't game a lot, so this could be an option for the very few times that I do.

 

Any thoughts on my questions though? It still feels a bit too good to be true ;-)

Link to comment
25 minutes ago, Carbongrip said:

For NVME drives that are in their own IOMMU group just add it to the vfio.pci list and check it as a pci devices in VM creation. NVME are really just pcie cards so no need to add them as a drive, Windows will see a pcie drive that it can install on.

Using this plugin? Also works for Mac VM?

 

 

 

Link to comment
32 minutes ago, Carbongrip said:

For NVME drives that are in their own IOMMU group just add it to the vfio.pci list and check it as a pci devices in VM creation. NVME are really just pcie cards so no need to add them as a drive, Windows will see a pcie drive that it can install on.

Just tried the plugin and am no longer sure whether all my NVME drives are NVME... One says NVME, the other two do not:

 

SATA controller: Lite-On IT Corp. / Plextor M6e PCI Express SSD 

Non-Volatile memory controller: Intel Corporation SSD Pro 7600p/760p/E 6100p Series (rev 03)

Link to comment
22 hours ago, steve1977 said:

Not to the array, but I could still access the disks? I should see them in disk manager?

Possibly, but if you access them in any way from windows, it will invalidate parity. Also, windows can't read the file system, so even if the disks show up, you couldn't see any files on them.

Link to comment
4 hours ago, jonathanm said:

Possibly, but if you access them in any way from windows, it will invalidate parity. Also, windows can't read the file system, so even if the disks show up, you couldn't see any files on them.

Got it, thanks! Let me give it a shot and let's see whether it works. Will start with my NVME drive (Intel 760p). I remember reading that there are some issues with this drive and Unraid, so will see whether it works.

Link to comment

I tried my luck. Passthrough worked well with the plugin and after reboot, I could see the passthrough disk under "Other PCI devices".

 

I selected "none" as primary vdisk and then ticked the passthrough disk. When starting the VM, I get to the UEFI shell and then see a "mapping table" and then a "Shell prompt".

 

Any thoughts?

Link to comment
  • 1 year later...

Hi everyone, 

 

Hope you can help me here with this. I can't get my windows 10 install running in a VM again. 

 

What do I have:

I have a ssd with a Windows 10 install that I use for booting directly into win10.

 

What do I want:

I want to use this ssd and win10 to run it as a VM in unraid if I need it.

 

What is my problem:

I always run in to this blue screen with inaccessable boot drive. I tryed already all possibilities with the config. Etc. It is always the same. Blue screen or nothing at all. 

 

Can you help me here? Maybe I missed something

 

Thx

 

Screenshot_20210818-113922_Chrome.jpg

Screenshot_20210818-113802_Chrome.jpg

Screenshot_20210818-113157_Chrome.jpg

Link to comment
  • 2 months later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.