Can UnRaid OS handle 3 VMs on one NVME SSD split as 3 partitions?

[slipstream]

Hi All,

Can the UnRaid OS handle three VMs on a single NVME SSD split as 3 partitions? My plan is to buy a 1TB Samsung 980 NVME SSD which is going to be mounted to a NVME PCI-E adapter card like the one shown in the JPG below. Addtionally, I want to install VMs for the Ubuntu OS, the Windows10 OS and the MacOS on this 1TB NVME SSD.  Will the UnRaid OS support such a setup?

 

 

By looking at my VM Setup Form screen capture below, I realize I may have to change the SATA option to NVME.  However, I am not 100% certain how the "Primary vDisk Location" parameter needs to change so it correctly supports such a NVME setup.  Mainly, what does the "AUTO" option need to change to so any one of the three operating systems correctly boots up as a VM?  And what will the OS installation path look like for each one of the three Operating Systems in such a NVME setup? And will all three of my VM Operating Systems run faster by using such a NVME SSD setup?

 

 

Any opinions welcome. Thank you for your time.

 

 

Edited May 12, 2021 by slipstream

[ghost82]

If unraid has drivers for that adapter, the answer is yes, it's possible to have 3 partitions on the same disk and assign one partition to each vm.

To passthrough a partition you need to select Manual for vdisk location and point to /dev/disk/by-id/blablabla-partX (where blablabla is the id of your disk and X is the partition number)-->open a terminal and cd to  /dev/disk/by-id/ to check for proper partition identification names.

The by-id identification is preferred over /dev/sdX as the X can change after a reboot and your vm will not start.

For more information you can also look this video:

 

With this setup you need a virtual controller to attach the partition (disk) to, virtio or sata should be preferred.

What you are highlighting in your picture is the virtual controller to attach the cd-rom, nothing to do with disks.

 

Depending on your needs you may also consider running the 3 vms on vdisks (image files), all on the nvme drive (with a single partition); I think it will be very little difference (if no difference at all) in terms of performances.

Edited May 13, 2021 by ghost82

[JonathanM]

7 minutes ago, ghost82 said:

Depending on your needs you may also consider running the 3 vms on vdisks (image files), all on the nvme drive (with a single partition); I think it will be very little difference (if no difference at all) in terms of performances.

This, plus it's much easier to manage backup and migration using image files.

[slipstream]

Thank you to all for your posts.  I learned something new.  I was assuming all three operating systems require being kept separately on different partitions. I never realized all 3 OS images could mingle together on the same NVME SSD partition.  So my new plan is to do without setting up 3 partitions and just mount all three images onto a single partition on a new NVME SSD. Thanks for your help.

[ghost82]

2 hours ago, slipstream said:

Thank you to all for your posts.  I learned something new.  I was assuming all three operating systems require being kept separately on different partitions. I never realized all 3 OS images could mingle together on the same NVME SSD partition.  So my new plan is to do without setting up 3 partitions and just mount all three images onto a single partition on a new NVME SSD. Thanks for your help.

good choice, I recommend only to make sure, check and set the discard option in the settings of the vms for the virtual disks (which are sparse image files) and set the operating systems for trim, otherwise the host (unraid) will not syncronize the free space on the real disk.

To make things clear with no discard trim you can copy a 10gb file in one of your vm and the host wll see +10gb, then when you delete that 10gb the vm will see -10gb but not the host, so at the end you will fill the whole disk.

Search on google or even in this forumfor trim/discard and you will understand a lot more for sure.

Attaching the vdisk to a virtio-scsi controller for windows will enable trim, for mac os you need to use a sata controller and some additional arguments in the vm settings to set non rotational disk and enable trim fr third party hd in mac os, fr linux I don't know I never played with it.

Anyway if you will have issues just come back and ask!

[ghost82]

Well, I have to modify my comment about performance..

I run a mac os vm from about 1,5 year and I always used vdisk files on my ssd (crucial).

When using vdisk attached to a virtual sata controller, mac os sees a generic AHCI controller with 1,5 gigabit speed.

Here is the test on the vdisk file:

 

 

Today I restored my vdisk image directly to the ssd and passed through to mac os the whole controller (Marvell 88SE9230, 6 gigabit), and attached the ssd to one sata port of that controller: well, the performance seems to have increased a lot:

 

 

Passing the whole controller seems to be the best option, no data with passing partitions, I think that shouldn't be much difference with vdisks, as controller is always emulated, but I may be wrong (again)...

 

Here the "issue" is obviously the AHCI virtual controller running at 1,5 gigabit, I don't know if it is detected differently in windows or linux.

Edited May 16, 2021 by ghost82

[slipstream]

Ghost82,

Thank you for your last post showing disk speed tests. My parts from Amazon arrived on Saturday which are shown in the links below:

https://www.amazon.com/Adapter-advanced-solution-Controller-Expansion/dp/B07JKH5VTL/ref=pd_sbs_4?pd_rd_w=lZeD3&pf_rd_p=98101395-b70f-4a52-af63-8fac2c513e02&pf_rd_r=RFC1N4DQ7M5CCNPEJ2SB&pd_rd_r=d09706bc-d341-499f-a106-87f369a57164&pd_rd_wg=DY1q6&pd_rd_i=B07JKH5VTL&psc=1

 

https://www.amazon.com/Blue-SN550-1TB-NVMe-Internal/dp/B07YFFX5MD/ref=pd_bxgy_img_2/141-0873488-7450925?_encoding=UTF8&pd_rd_i=B07YFFX5MD&pd_rd_r=0d5f4af5-81a7-496e-b4e5-23f80df2f905&pd_rd_w=wXlzk&pd_rd_wg=9DKb3&pf_rd_p=fd3ebcd0-c1a2-44cf-aba2-bbf4810b3732&pf_rd_r=782K87JJMREZ3MSC8Q6T&psc=1&refRID=782K87JJMREZ3MSC8Q6T

 

I watched the SpaceInvader One video you posted in your previous post.  The screenshot below shows how my NVME app appears in my Unassigned Devices area after I click on the + button:

 

 

I set my Primary vDisk Location to "Manual" and paste the path shown below which is generates the errors shown below:

 

/dev/disk/by-id/ata-WDC_WDS100T2B0C-00PXH0_21146D802780 (nvme0n1)

 

 

 

 

After I hit the "CREATE" vm button I see the error below:

 

 

Additionally, removing the "(nvme0n1)" text string located at the end did not help. I still get a path error:

 

 

 

 

Ghost82, can you or anybody in the community please advise what is the correct Manual vDisk Location path I need to use that the UnRaid OS will not reject?  I ask because the SpaceInvader One video I watched seems to be about passing through a SSD hard drive and not passing through a NVME drive.

 

The screen captrues below shows my System Devices for my PCI NVME card. So I don't think the problem has anything to do with hardware incompatibility because the UnRaid OS successfully recognizes this NVMe hardware.

 

 

Any opinions will be greatly appreciated because I am stuck in the mud.

 

 

 

[ghost82]

9 hours ago, slipstream said:

Ghost82, can you or anybody in the community please advise what is the correct Manual vDisk Location path I need to use that the UnRaid OS will not reject?

 

Hi, when using vdisk as primary disk location choose a place on your nvme disk to save the vdisk.img file (a path to it to save the file, first mount the drive!). Something like /path/to/nvme/vdisk.img

Edited May 17, 2021 by ghost82

[slipstream]

Ghost82,

Thank you for your post.  Let me break it down.  SpaceInvader One says the beginning of the path needs to look like this:   /dev/disk/by-id/ata-

Then it needs to be followed by my NVME id which is:  WDC_WDS100T2B0C-00PXH0_21146D802780 (nvme0n1)

 

My Unassigned Devices app confirms the NVMe is MOUNTED as shown in the screen shot above.  However, my Unassigned Devices app does not show me a path where my NVMe is at.  Also my Unassigned Devices app generates a "(nvme0n1)" text string at the end of the ID.  Can somebody please clarify if this text string needs to be included in the path to my NVMe?

 

Lastly, can anybody in the community please clarify for me how this part of the path:  /dev/disk/by-id/ata-

needs to change so my Unraid OS does not reject it?

 

[ghost82]

58 minutes ago, slipstream said:

Ghost82,

Thank you for your post.  Let me break it down.  SpaceInvader One says the beginning of the path needs to look like this:   /dev/disk/by-id/ata-

Then it needs to be followed by my NVME id which is:  WDC_WDS100T2B0C-00PXH0_21146D802780 (nvme0n1)

 

My Unassigned Devices app confirms the NVMe is MOUNTED as shown in the screen shot above.  However, my Unassigned Devices app does not show me a path where my NVMe is at.  Also my Unassigned Devices app generates a "(nvme0n1)" text string at the end of the ID.  Can somebody please clarify if this text string needs to be included in the path to my NVMe?

 

Lastly, can anybody in the community please clarify for me how this part of the path:  /dev/disk/by-id/ata-

needs to change so my Unraid OS does not reject it?

 

You are not passing through the nvme drive, but you are saving vdisk img files on the nvme drive, so you are following the wrong tutorial.

The path you are writing is totally wrong and doesn't start with /dev...

Once the nvme drive is mounted by unassigned device it should mount the nvme at /mnt/disks/ (please note the final s in diskS).

So look inside that folder, find your nvme mount point folder, eventually create another folder to identify the name of your vm and inside this save the vdisk.img

The path will look something like /mnt/disks/nvmeMountPoint/myVMname/vdisk1.img

[JonathanM]

1 hour ago, slipstream said:

My Unassigned Devices app confirms the NVMe is MOUNTED as shown in the screen shot above. 

If it's mounted it CAN'T be passed through.

[slipstream]

Jonathanm,

Thank you for your post.  The reason I am mounting it using my Unassigned Devices App is because SpaceInvader One In his video linked in the post above is using his Unassigned Devices app and mounting his device.  So I'm confused why he is taking that approach.

 

Nevertheless, I will follow your advice and try the following different approach. In my System Devices I 'm going to apply a VFIO checkmark to this device and then hit the BIND button.  By doing this my hope is this NVMe device path problem will get fixed.  Thank you to all for your time.