Unraid 6.11.1 - virtiofs with Windows 10 VM

[johnsanc]

Posting this here because I don't want to clutter up the announcement thread.... but how do you get virtiofs to work with a Windows 10 VM?

 

The form composer created this for the settings:

    <filesystem type='mount' accessmode='passthrough'>
      <driver type='virtiofs' queue='1024'/>
      <binary path='/usr/libexec/virtiofsd' xattr='on'>
        <cache mode='always'/>
        <sandbox mode='chroot'/>
        <lock posix='on' flock='on'/>
      </binary>
      <source dir='/mnt/user/'/>
      <target dir='shares'/>
      <alias name='fs0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </filesystem>

 

 I tried manually adding this to the XML which got around the startup error:

  <memoryBacking>
    <nosharepages/>
    <source type='memfd'/>
    <access mode='shared'/>
  </memoryBacking>

 

But now what? I have winfsp installed and in Windows I can see there is a "VirtIO-FS Service". If I start that service nothing seems to happen. I expected to see a new drive letter mounted once the service starts. Is there some additional steps that are required?

 

@SimonF - Any thoughts?

[johnsanc]

EDIT: I was able to get this to work by removing a network drive I had mapped to Z:

Apparently this service does not manage any drive letter conflicts correctly.

[johnsanc]

So it works correctly if I mount a single share, but if multiple are included it seems to map a random share to the Z: drive.

How can you properly map multiple shares?

 

I tried the instructions here but whenever I try to start the virtiofs service the console looks ok, but then I see errors in the event viewer:

 

Console log used, where "games" is a mount tag specified in the XML

"C:\Program Files (x86)\WinFsp\bin\launchctl-x64.exe" start virtiofs viofsY games Y:

 

Event Viewer error:

virtiofs: The service VirtIO-FS has failed to start (Status=c0000001).

 

Has anyone got this to work correctly for multiple shares?

[johnsanc]

Apparently this is a very new feature that hasn't made its way into any of the standard driver ISOs yet.

https://github.com/virtio-win/kvm-guest-drivers-windows/pull/804

 

I guess I'll stick with a single share for now to avoid the headaches.

[SimonF]

13 minutes ago, johnsanc said:

Apparently this is a very new feature that hasn't made its way into any of the standard driver ISOs yet.

https://github.com/virtio-win/kvm-guest-drivers-windows/pull/804

 

I guess I'll stick with a single share for now to avoid the headaches.

thanks for the info.

[johnsanc]

Another update - Permissions are also an issue and I haven't quite figured that one out yet. 

 

I tried a few basic tests like copying a folder of files from VM to the mounted drive and it did not work. it created the folder with 775 permissions but then none of the files within the folder could be copied. If anyone has any ideas I'm all ears.

 

A few tests I did:

- create folder with smb: nobody/users, 777

- create file with smb: nobody/users, 666

- create folder with virtiofs: nobody/users, 775

- create file with virtiofs: nobody/users, 664

Edited October 8, 2022 by johnsanc

[SimonF]

12 hours ago, johnsanc said:

Another update - Permissions are also an issue and I haven't quite figured that one out yet. 

 

I tried a few basic tests like copying a folder of files from VM to the mounted drive and it did not work. it created the folder with 775 permissions but then none of the files within the folder could be copied. If anyone has any ideas I'm all ears.

 

A few tests I did:

- create folder with smb: nobody/users, 777

- create file with smb: nobody/users, 666

- create folder with virtiofs: nobody/users, 775

- create file with virtiofs: nobody/users, 664

When I tried to copy my downloads folder to the z: drive the directory was create a readonly so rest of copied failed.

 

root@computenode:/mnt/disk1/@Documents/Virtio# ls -l
total 4
dr-xr-xr-x 1 nobody users 0 Oct  9 09:20 Downloads/
drwxrwxr-x 1 nobody users 0 Oct  9 09:56 New\ folder/
-rw-rw-r-- 1 nobody users 5 Oct  9 09:56 Test4.txt
root@computenode:/mnt/disk1/@Documents/Virtio# chmod 777 Downloads/
root@computenode:/mnt/disk1/@Documents/Virtio# ls -l
total 4

Will look to see if I can find any issues with WinFSP. change permissions on dir then I was able to copy.

Edited October 9, 2022 by SimonF

[SimonF]

17 hours ago, johnsanc said:

Apparently this is a very new feature that hasn't made its way into any of the standard driver ISOs yet.

https://github.com/virtio-win/kvm-guest-drivers-windows/pull/804

 

I guess I'll stick with a single share for now to avoid the headaches.

I have asked the maintainers schedule for releases and they said approx every 3 months. So hopefully there will be a updated package this month.

[johnsanc]

Excellent thanks for the update. Yeah it just took me a few minutes to realize the mount tag parameter wasn't supported yet.

Hopefully we can get to the bottom of the permissions issues... otherwise this is basically unusable for basic Windows use cases.

 

Im not sure if its a virtiofs issue or winfsp issue, or if theres some special configuration or way to make it play nice.

 

Also, just to clarify, for my results above I used an extra SMB configuration to force the user/permissions that I want and its worked great for years. I am hoping we can do something like that with virtiofs/winfsp.

 

[global]
security = USER
guest account = nobody
public = yes
guest ok = yes
map to guest = bad user
force user = nobody
force group = users
force directory mode = 0777
force create mode = 0666
create mask = 0666

 

[20_100]

if you want to access multiple unraid shares, what about trying to share directly /mnt/user/ ?

 

wouldn't that solve the multi-share issue, while we wait for the KVM release?

 

It doesn't solve the permissions issue, which I also meet (among others)

[johnsanc]

Yes I am currently just mounting /mnt/user/ for now to keep it simple until the signed drivers support the mount tag. I changed my registry key to a different drive letter though to avoid conflicts with my other mapped network drives. That worked fine.

Info here on how to do that: https://github.com/virtio-win/kvm-guest-drivers-windows/wiki/VirtIO-FS:-Shared-file-system#options

 

So far everything seems to work great and the performance for me at least is MUCH better than SMB with a ton of small files, but the permissions issues are currently a show stopper.

 

[20_100]

Looks like the permission issue it might be related to :

 

https://github.com/virtio-win/kvm-guest-drivers-windows/issues/722

and

https://github.com/virtio-win/kvm-guest-drivers-windows/issues/660

 

which seems to be closed and merged, so it might be shipped with the same next release.

 

edit : I confirm that I can create and edit files in new directories when I use a local account instead of Active Directory account

 

edit 2 : Apparently, all my issues are solved when using a local windows account. 

 

Sidenote : I even used this local Windows user to create a windows network share, and could give permissions to AD users. That hardly worked when I tried doing that from an AD account, and there were tons of issues. All works, feels very stable at the moment.

I want to see if sharing this virtiofs drive instead of using unraid SMB share solves permission issues and some performance issues I have been struggling with for years.

 

edit 3 : see the rest of the post below, as some issues seem to persist

Edited October 9, 2022 by 20_100

[johnsanc]

I am currently using a local account with administrator privileges and it does not work for me. Are you sure you can copy a folder of files to the mounted drive? If so, how do the permissions of the copied folder/files look on the host machine after you copied them?

I hate windows permissions.

Edited October 9, 2022 by johnsanc

[20_100]

Yes, I confirm.

 

My setup so you can compare:

I mounted /mnt/user/ to Z: ; to do so I registered the Windows Service as described in the virtiofs documentation.

Nothing special at all on the unraid side, except maybe that I ran the "New permissions" unraid tool to reset my permissions when I started struggling with the issues we are discussing.

 

The VirtioFsSvc windows service hosts virtiofs.exe under the SYSTEM user.

 

When you check a share permissions, do you have Everybody and a broken sid S-1-5-0 ? 

 

 

[johnsanc]

Well this is bizarre... last night my VM froze with nothing in the logs or event viewer, so I forced restarted it today and now I just did another check and I can see that it does work correctly exactly as you show in your screenshot above. Yesterday I was getting everything with that S-1-5-0 user.

 

I have no idea what would have changed to make it work differently today...

[johnsanc]

OK so I figured out how to break it... stop your VirtIO-FS service and restart it. It will behave improperly with that broken S-1-5-0 user. Can you confirm?

I think yesterday I was running into issues because I was stopping and starting services and changing drive letters. But it looks like if you let it start up automatically and don't stop it it works correctly.

[20_100]

Indeed, when I restart the service, the folder permissions issues happen and the S-1-5-0 user too.

😞

[johnsanc]

One other thing to note... the documentation shows to register the service with a dependency, but I already had the VirtIO-FS service active before enabling any of this, and it had no dependencies, so to modify that service for good measure I did this from an elevated command prompt:

 

sc config VirtioFsSvc depend="WinFsp.Launcher/VirtioFsDrv"

 

Didn't seem to resolve the broken user on restart though.

[20_100]

It works if you use 

 

depend="WinFsp.Launcher" 

 

Then the new service correctly waits for the WinFsp.Launcher to start. It appears in the dependency hierarchy of both services correctly.

Sorry I forgot I did that

[johnsanc]

Did that resolve the restart issue for you? Or even with the dependencies you still have the restart issue?

[20_100]

Even with the corrected dependencies, the restart issue persists. S-1-5-0 anytime I restart the service

[johnsanc]

Thanks for confirming, just for reference I also reported that here: https://github.com/virtio-win/kvm-guest-drivers-windows/issues/839

[pspfreak]

On 10/9/2022 at 11:32 AM, 20_100 said:

Yes, I confirm.

 

My setup so you can compare:

I mounted /mnt/user/ to Z: ; to do so I registered the Windows Service as described in the virtiofs documentation.

Nothing special at all on the unraid side, except maybe that I ran the "New permissions" unraid tool to reset my permissions when I started struggling with the issues we are discussing.

 

The VirtioFsSvc windows service hosts virtiofs.exe under the SYSTEM user.

 

When you check a share permissions, do you have Everybody and a broken sid S-1-5-0 ? 

 

 

Hi there, did you manage to fix this issue?

 

I'm getting similar permissions. 

 

[johnsanc]

The fix for now is to automatically start the service and do not restart it.

Sounds like there's some pretty complicated logic for determining the user that should be ironed out in the next driver release.

https://github.com/virtio-win/kvm-guest-drivers-windows/issues/839

 

[R.M.H]

I set this up as a test as was getting a weird issue where the VPN would block access to file shares, I had mounted on the Win 10 VM. Had this setup exactly as I needed it and change the drive letter by adding some items to the registry. All looked fine with access to the drive added yet when I had more activity against the share, I was getting multiple IO errors. Has anyone had a similar issue or is access to the shared drive OK for those that have set this up.

 

I stopped the windows service in the end and reverted the VM config so not using it now. With a plan to look it later as this is still a new addition, and a few bugs is to be expected.