2 weeks ago, I started testing unRAID to see if my system can pass-through GPU and other devices efficiently.

The GPU pass-through worked only when I added 2nd GPU which I already had and worked perfectly and gaming performance and sound I noticed no difference at all as I had assigned all the CPU to the VM.

I still need to work on reducing the assigned CPUs while not causing audio latency, but I'll work on it later.

 

I first tested GPU pass-through by creating a new VM with a virtual disk, when I found that the GPU pass-through is working.

I then wanted to do some real tests by playing some games, so instead of installing games in the new virtual machine.

I thought it would be best to boot my main Windows 10 OS from the physical disk directly into the VM which has all the games and everything ready.

I researched and found sata pass-through is not suggested and it's easier to add disks as block device.

 

So, I removed the new virtual disk and assigned the physical disk as block device (Seagate 500GB HDD), it took a while for Windows 10 re-configure devices but it was up and running without any errors at all and I couldn't feel any difference while using the same.

I ran some passmark performance tests, the disk performance was way better I then realized it must be caching because HDD's can't definitely work better than SSD's. CPU and memory performance was pretty close to real performance.

 

Once this worked perfectly, I had to add 2nd disk as it had most of my games, one of it being Shadow of Mordor which is known to use GPU at fullest.

So, I assigned 2nd disk (Seagate 3 TB HDD), it was detected properly preserving the drive letters.

I was able to play the game flawlessly with no performance drop @60 FPS on 2560x1080 Ultra-wide resolution.

This got me so excited, I was really happy to see.

 

After this worked, I wanted to do some rendering tests and the software was in my another disk.

I had disabled all the additional/data disks when I started with this to avoid any possible data loss.

So, to add it I had to shutdown the VM and restart the host OS, then go to BIOS to enable the disk.

When I enabled it, unraid gave me error about no of physical disk and that the array couldn't be started on which the virtual machine configuration file was present.

So, I restarted the host OS again and disabled the physical disk with the game in BIOS.

After the restart I assigned the other 3TB disk as 2nd disk to the VM.

 

I then started the VM, and the disk was detected but I noticed that the drive letters had changed and the partition labels were same as the previous 3TB disk which was with games.

This was really weird and I got worried as to what has happened. I first though it could be something to do with caching, but then it shouldn't cause issues across reboots as caching is temporary.

I then shutdown the vm and host OS. Then started my main Windows 10 OS without any virtualization and all disks enabled.

I couldn't see drive letter for the 3TB disk with issue and when I went to disk management it had warning with disk unique id collision.

I enabled it anyways and found that it had same drive letters as the first 3tb disk and almost all the folders/data was missing.

That's when I stopped with the unraid testing and have been recovering data as and when I could, I still lost around 600 GB of data as the disk was almost full.

And I may also be dealing with some corruption in the existing/recovered data which I am not fully aware of yet.

 

I know, it wasn't right for me to use the physical disks like that but I was really excited about this and didn't give much thought to what issues could arise and anyways its my home pc and things happen.

 

I haven't gone back to unraid virtualization testing yet but plan to go soon again.

But before that I want to know if anyone has experienced this issue or why it happened ?

It looked like the 2nd disk wasn't unique for the guest OS as the block devices are listed as qemu disk in the device manager.

How can I avoid this from happening again are there any special settings I should have added to the VM configuration or is it best if I don't use block device at all.

 

I am really looking forward to use unraid linux as it appears to performs quite well. But I need to know what went wrong.

 

My system configuration is as below.

CPU: Core i7-6850K (Broadwell-E 6-core - with HT 12 threads)

Motherboard: ASUS X99-Deluxe II

GPU1: ZOTAC GTX 460 1GB (PCIe Slot 1 running at x16) - Default and assigned to host automatically.

GPU2: ASUS STRIX GTX 970 4GB (PCIe Slot 3 running at x16) - Assigned to guest OS.

Storage devices as below:

1 x 8 GB USB pen drive - Used for Unraid (I thought I was making this USB disk for installation but noticed it directly boot into unraid linux, so I used it as is.)

2 x 500 GB Seagate HDD - One has my main Windows 10 OS which is working fine still. 2nd disk was used to create unraid array without any parity disks. This had the virtual machine configuration data as well and the virtual disk.

3 x 3 TB Seagate HDD SV35 Series - One with the game as described, 2nd disk got corrupted, 3rd disk never used within unraid. Two of them have identical model number (ST3000VX000-1ES166) are were used within unraid, and the unused one has model ST3000VX000-1CU166

All the above disks have NTFS partitions except USB has FAT32 for obvious reasons.

All 3TB disks are formatted using GPT with exactly same partition layout 500 GB + 2.24TB

Also, I would like to add when I checked GPT details I found that GUID on corrupted disk was same as the other 3TB disk.

No reply yet 

I read your initial post but I got lost. If you can explain succinctly what you are trying to achieve and what is going wrong and if you attach your diagnostics zip then someone might be able to help.

Hi John,

 

Thanks. I'll explain in short here and unfortunately I won't be able to provide diagnostic log as I erased the USB with unraid linux.

I am just concerned why my 3TB HDD got corrupted and how it can be prevented.

I added 500 GB HDD (physical block device) to the VM as primary disk (boot disk) which already had Windows 10 that was running in the host machine.

I added 3TB HDD (physical block device virtio) to the VM as secondary disk which worked fine. (Model no of disk ST3000VX000-1ES166, I have total two of these with exactly same model number)

 

After that I restarted the whole system to disable the above 3TB HDD in BIOS and attach another 3TB HDD.

I had to disable it in BIOS because Unraid trial version has 3 disks limit. (One disk was being used for unraid array where VM config files are present, and one disk with Windows 10 which leaves me with only 1 more disk that I can use )

 

I added 3TB HDD (physical block device virtio) to the VM as secondary disk which showed the same drive letters in Windows as the previous one and none of the folders were accessible when I try to browse.

(Yes this is the 2nd disk with exactly same model number.)

 

I then realized something has gone terribly wrong.

I shutdown unraid boot into the same Windows 10 OS (500 GB HDD) and find that the basic structure of the 2nd disk I had put was overwritten with the first one and it ended up having same GPT GUID as the first 3TB disk.

 

I am wondering why Windows 10 guest os didn't see this 2nd 3TB HDD as unique disk and somehow overwrote it with the partition structure of previous disk in the process ? Is it possible that when I added it as secondary disk virtio didn't generate some unique id for this disk ?

I used the disk by /dev/disk/by-id/ method to provide the block device.

 

I hope this explains well, please let me know. If anyone doesn't get it. I think this is quite a serious issue.

I may try to reproduce it with smaller physical disks if possible when I have more time.

So you're trying to pass through hard disks to the virtual machine, not SATA controllers? It sounds like Windows got confused and caused the damage. I think you need to recreate the situation and provide the diagnostics. There's nothing to work on otherwise. I trust you have backups.

Hi John,

 

No, I didn't have backups I was able to recover most of the data and didn't lost any important data.

So, overall its fine but took long time to go through the recovered data and sort it.

 

I'll try to recreate the situation when I have more free time probably within next 2 or 3 weeks.

 

For now I am providing links to some articles suggesting similar issue.

https://github.com/rockstor/rockstor-core/issues/673 (Check other issues mentioned in the article as well)

https://forum.rockstor.com/t/proxmox-serial-number-passthrough/319

http://blog.zugschlus.de/archives/882-Block-devices-in-KVM-guests.html (Not sure if this is related)