Elaborating on what was posted above, 9p can be much faster than both NFS and SMB in my experience. For example, on a Debian 11.3 VM running on UnRAID, I mounted my iso share as ISO and did a quick test. The results on my NVME were not steller:
root@debian:/iso# dd if=/dev/zero of=/iso/output bs=8k count=1000k status=progress; rm -f /tmp/output
1024000+0 records in
1024000+0 records out
8388608000 bytes (8.4 GB, 7.8 GiB) copied, 84.6152 s, 99.1 MB/s
This is equivalent to ~max speed for a 1Gbps NIC. Not great.
After doing some reading though, I found that increasing the maximum bytes for the packet payload greatly improves performance and ultimately outstrips NFS & SMB. Reading the QEMU docs, I mounted as follows:
mount -t 9p -o trans=virtio ISO /iso -oversion=9p2000.L,posixacl,msize=104857600,cache=loose
Using fstab:
ISO /iso 9p trans=virtio,version=9p2000.L,posixacl,msize=104857600,cache=loose,_netdev,noauto,nobootwait,rw 0 0
Mount output (it seems 512k is the max msize in kernel 5.10.)
root@debian:/iso# mount | grep ISO
ISO on /iso type 9p (rw,relatime,dirsync,loose,access=client,posixacl,msize=512000,trans=virtio,_netdev)
The results (tested a few times):
root@debian:/iso# dd if=/dev/zero of=/iso/output bs=8k count=1000k status=progress; rm -f /iso/output
1024000+0 records in
1024000+0 records out
8388608000 bytes (8.4 GB, 7.8 GiB) copied, 6.66122 s, 1.3 GB/s
Note: This is completely anecdotal and your results may vary. For example, I used the `cache=loose` option, which is not intended for this use case and may have skewed my results. Also,
Additionally, the default msize in kernel 5.15+ is 128KiB, which should yield better performance. See links below.
Update: More recent results have been less favorable, with NFS shares averaging 313MB/s (1Gbps) and 9p performing at 191MB/s on average--all on the same array used as before. My initial findings were likely a fluke.
Further reading:
https://www.kernel.org/doc/Documentation/filesystems/9p.txt
https://wiki.qemu.org/Documentation/9psetup
https://lists.gnu.org/archive/html/qemu-devel/2021-03/msg01003.html
