Shared Folders virtio-9p performance issues

[unifiedmamba]

Hello guys, I'm running a VM that is located on my SSD cache drive and has a few user shares mounted which are located on the array via 9p.

Mounting etc. works well, but I am experiencing very slow read/write speeds atm.

 

The shares perform well within Unraid (tested with around 150mb/s read/write) but as soon as they are used inside my debian VM the read speed of the uncached part significantly decreases to about 10-15mb/s.

Here is my fstab config:

 

backups /mnt/backups 9p trans=virtio,version=9p2000.L,_netdev,rw,msize=104857600 0 0

 

The msize entry already increased my write speed, but read speed is still very slow. Reading/writing to the VMs virtual disk is about 1200mb/s, so everything is fine with the VM itself. 

Has somebody experienced similar issues and has a clue how to fix this?

Edited January 5, 2022 by unifiedmamba

[JonathanM]

What kind of speeds do you see with cifs / smb to those same shares?

[unifiedmamba]

5 minutes ago, JonathanM said:

What kind of speeds do you see with cifs / smb to those same shares?

Around 115mb/s but this is limited by the network speed (1GBe minus overhead)

[JonathanM]

Just now, unifiedmamba said:

Around 115mb/s but this is limited by the network speed (1GBe minus overhead)

In the VM?

[unifiedmamba]

5 minutes ago, JonathanM said:

In the VM?

Oh no with my windows PC. I don't feel like using a Microsoft standard to share files between two Linux machines should be the right thing to do. There must be an issue with 9p. But I will see how I am able to mount it via SMB and will give feedback.

[unifiedmamba]

@JonathanM

 

So I've mounted the smb share via this guide https://tecadmin.net/mounting-samba-share-on-ubuntu/ without any further tweaking which results in about 135mb/s in the VM. Seems better, but not the optimal solution.

The problem now is that I still don't get full speed when downloading test files to my PC served by apache2. I now get around 50mb/s which is better than before but not the 115mb/s that are possible if I serve the file from a folder that is located on the qcow2 disk of my VM.

Additionally I have made bad experiences with the handling of small files by SMB which results in very very low speeds when transferring a few small files.
I would therefore like to fix the 9p mount to avoid the new problems that SMB could cause with my use case. 

[Jawalking]

I too have spent the last few hours messing with 9pfs and have been seeing some pretty poor performance.

 

I use the following fstab options and get about 90MiB/s read and write (tested with fio and 4K block size; 128k and 256k saw slightly better performance)

epi_af /root/9pf_mount 9p trans=virtio,_netdev,rw 0 0

 

Adding 'msize=262144' or 'msize=524288' helps a tiny bit, epically with larger blocksize workloads

 

I too would rather not use SMB, but seeing as I get about 900-1100 MiB/s using SMB (which is almost exactly what I get to the virtual disk) I think that's what I think I'm going to go with, unless I can find a 9pfs solution.

Edited January 6, 2022 by Jawalking

[Jawalking]

After my perf testing, and reading through the following, I'm just going to use SMB for now.
https://landley.net/kdocs/ols/2010/ols2010-pages-109-120.pdf

https://landley.net/kdocs/Documentation/filesystems/9p.txt

https://news.ycombinator.com/item?id=8855143

 

Also looks like most of active dev work on virtio is for the Rust version (https://gitlab.com/virtio-fs/virtiofsd). I know the kernel is moving (perhaps including more that moving) that way, and fortunately for us unRAID keeps on a pretty new kernel, so perhaps we'll get to play with VirtIO-FS in 6.10:

 

 

[unifiedmamba]

@Jawalking Do you think it would be a better solution to just create a new qcow2 or even raw image as a secondary drive and add all files that are now located in the shared folder into that one? Since I need the files to be very fast in the VM for nextcloud this sounds like a good solution for me. For use with other containers I could create a NFS share or something from the VM itself. Do you think this would work? 

Edited January 16, 2022 by unifiedmamba

[unifiedmamba]

Are there any News ob this topic?

[unifiedmamba]

Bump

[TheLinuxGuy]

Has anything changed in 2023 about better peformance in QEMU 9p?

[SimonF]

12 minutes ago, TheLinuxGuy said:

Has anything changed in 2023 about better peformance in QEMU 9p?

There is meant to be a performance improvement in 7.2 https://wiki.qemu.org/ChangeLog/7.2#9pfs

Unraid is currently on 7.1

[Aegisnir]

any new information here?  I'm maxing out at ~450MB/s with 9p.  Mounting with cifs gets me 1.1GB/s but then I run into needing to use sudo every time I need to create or a delete a file/directory.  What is the current "correct" way to do this so I have native speed when accessing my unraid shares from my vm without running into permission issues?

Edited August 9 by Aegisnir
details