[samba share is very slow on nvme cache pool]

On 2/28/2023 at 4:43 PM, JorgeB said:

Try using the disk path/share, user share always add some overhead.

When Docker mounts a path to /mnt/cache/xxx, there is no shfs process usage.

New folder with 70k files

via docker

sftpgo-ftp rclone--transfers=16 11m14.9s
sftpgo-sftp rclone--transfers=16 11m7.4s
syncthing 10m52s

 

via unraid build-in sftp
host-sftp 2m6s

 

Transferring many files via docker is still slow, docker's wirte operation is direct to disk. However, via native sftp, data is cached in ram and written to disk.

[samba share is very slow on nvme cache pool]

via docker:

drakkan/sftpgo:

ftp  64s
sftp 50s

 

crazymax/pure-ftpd:

14s

[samba share is very slow on nvme cache pool]

With unraid smb all default settings:

sftp is 7.9x faster than smb

sftp also get lower cpu usage

 

nvme-cow:
TotalRead  = 62 MiB
TotalWrite = 62 MiB
TotalFiles = 2,926 (335)
TotalTime  = 01:19
TransRate  = 0.81 MB/s
FileRate   = 36.6 files/s

nvme-nocow:
TotalRead  = 62 MiB
TotalWrite = 62 MiB
TotalFiles = 2,926 (335)
TotalTime  = 01:37
TransRate  = 0.67 MB/s
FileRate   = 30.1 files/s
hdd*1(xfs)
TotalRead  = 62 MiB
TotalWrite = 62 MiB
TotalFiles = 2,926 (335)
TotalTime  = 01:34
TransRate  = 0.69 MB/s
FileRate   = 30.9 files/s

 

smb
win11->unraid(nvme raid1 cache pool)
nvme-cow: 01:19
nvme-nocow: 01:37
hdd*1(xfs): 01:34

win11(pc)->win10(pc-vm): 39s
sftp
win11->unraid(nvme raid1 cache pool)
cow=nocow=hdd*1(xfs)=10s

[What is the difference between virtio and virtio-net ?]

Default 'virtio-net' opinions provide awful performance.

'virtio-net' also gets much higher cpu usage

VM -> Host

virtio-net :

[ ID] Interval Transfer Bitrate Retr

[ 5] 0.00-10.00 sec 2.25 GBytes 1.93 Gbits/sec 0 sender

[ 5] 0.00-10.00 sec 2.25 GBytes 1.93 Gbits/sec receiver

 

virtio :

[ ID] Interval Transfer Bitrate Retr

[ 5] 0.00-10.00 sec 30.7 GBytes 26.4 Gbits/sec 0 sender

[ 5] 0.00-10.00 sec 30.7 GBytes 26.4 Gbits/sec receiver

 

proxmox seems to just use 'virtio' as the default without causing any stability problem