thymon Posted March 7, 2023 Share Posted March 7, 2023 (edited) Hello, Same here Test iperf3 unraid --> unraid-backup (30Mbits) root@Unraid-Backup:~# iperf3 -s ----------------------------------------------------------- Server listening on 5201 (test #1) ----------------------------------------------------------- Accepted connection from 10.253.0.1, port 54148 [ 5] local 10.253.0.2 port 5201 connected to 10.253.0.1 port 54162 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 2.26 MBytes 19.0 Mbits/sec [ 5] 1.00-2.00 sec 2.63 MBytes 22.1 Mbits/sec [ 5] 2.00-3.00 sec 2.75 MBytes 23.1 Mbits/sec [ 5] 3.00-4.00 sec 3.00 MBytes 25.2 Mbits/sec [ 5] 4.00-5.00 sec 3.09 MBytes 25.9 Mbits/sec [ 5] 5.00-6.00 sec 3.00 MBytes 25.1 Mbits/sec [ 5] 6.00-7.00 sec 2.96 MBytes 24.8 Mbits/sec [ 5] 7.00-8.00 sec 2.81 MBytes 23.5 Mbits/sec [ 5] 8.00-9.00 sec 2.81 MBytes 23.6 Mbits/sec [ 5] 9.00-10.00 sec 2.26 MBytes 19.0 Mbits/sec [ 5] 10.00-10.09 sec 224 KBytes 21.5 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.09 sec 27.8 MBytes 23.1 Mbits/sec receiver root@Unraid:~# iperf3 -c 10.253.0.2 Connecting to host 10.253.0.2, port 5201 [ 5] local 10.253.0.1 port 54162 connected to 10.253.0.2 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 3.91 MBytes 32.8 Mbits/sec 0 337 KBytes [ 5] 1.00-2.00 sec 2.64 MBytes 22.1 Mbits/sec 0 350 KBytes [ 5] 2.00-3.00 sec 2.76 MBytes 23.1 Mbits/sec 0 337 KBytes [ 5] 3.00-4.00 sec 3.13 MBytes 26.2 Mbits/sec 0 369 KBytes [ 5] 4.00-5.00 sec 3.19 MBytes 26.7 Mbits/sec 0 326 KBytes [ 5] 5.00-6.00 sec 3.19 MBytes 26.7 Mbits/sec 0 315 KBytes [ 5] 6.00-7.00 sec 2.39 MBytes 20.1 Mbits/sec 0 310 KBytes [ 5] 7.00-8.00 sec 3.19 MBytes 26.8 Mbits/sec 0 313 KBytes [ 5] 8.00-9.00 sec 2.39 MBytes 20.1 Mbits/sec 0 297 KBytes [ 5] 9.00-10.00 sec 2.39 MBytes 20.1 Mbits/sec 0 307 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 29.2 MBytes 24.5 Mbits/sec 0 sender [ 5] 0.00-10.09 sec 27.8 MBytes 23.1 Mbits/sec receiver rsync with 1Go "testfile" over Wireguard VPN at 3Mbits.... (I tested all possible options for rsync and SSH without improvement)... root@Unraid:~# rsync -az --progress testfile 10.253.0.2:/mnt/cache([email protected]) testfile 50,987,008 4% 3.04MB/s 0:05:20 So, what other solution do you use to backup to another remote Unraid server? Edited March 7, 2023 by thymon Quote Link to comment
weirdcrap Posted March 8, 2023 Author Share Posted March 8, 2023 14 hours ago, thymon said: Hello, Same here Test iperf3 unraid --> unraid-backup (30Mbits) root@Unraid-Backup:~# iperf3 -s ----------------------------------------------------------- Server listening on 5201 (test #1) ----------------------------------------------------------- Accepted connection from 10.253.0.1, port 54148 [ 5] local 10.253.0.2 port 5201 connected to 10.253.0.1 port 54162 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 2.26 MBytes 19.0 Mbits/sec [ 5] 1.00-2.00 sec 2.63 MBytes 22.1 Mbits/sec [ 5] 2.00-3.00 sec 2.75 MBytes 23.1 Mbits/sec [ 5] 3.00-4.00 sec 3.00 MBytes 25.2 Mbits/sec [ 5] 4.00-5.00 sec 3.09 MBytes 25.9 Mbits/sec [ 5] 5.00-6.00 sec 3.00 MBytes 25.1 Mbits/sec [ 5] 6.00-7.00 sec 2.96 MBytes 24.8 Mbits/sec [ 5] 7.00-8.00 sec 2.81 MBytes 23.5 Mbits/sec [ 5] 8.00-9.00 sec 2.81 MBytes 23.6 Mbits/sec [ 5] 9.00-10.00 sec 2.26 MBytes 19.0 Mbits/sec [ 5] 10.00-10.09 sec 224 KBytes 21.5 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.09 sec 27.8 MBytes 23.1 Mbits/sec receiver root@Unraid:~# iperf3 -c 10.253.0.2 Connecting to host 10.253.0.2, port 5201 [ 5] local 10.253.0.1 port 54162 connected to 10.253.0.2 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 3.91 MBytes 32.8 Mbits/sec 0 337 KBytes [ 5] 1.00-2.00 sec 2.64 MBytes 22.1 Mbits/sec 0 350 KBytes [ 5] 2.00-3.00 sec 2.76 MBytes 23.1 Mbits/sec 0 337 KBytes [ 5] 3.00-4.00 sec 3.13 MBytes 26.2 Mbits/sec 0 369 KBytes [ 5] 4.00-5.00 sec 3.19 MBytes 26.7 Mbits/sec 0 326 KBytes [ 5] 5.00-6.00 sec 3.19 MBytes 26.7 Mbits/sec 0 315 KBytes [ 5] 6.00-7.00 sec 2.39 MBytes 20.1 Mbits/sec 0 310 KBytes [ 5] 7.00-8.00 sec 3.19 MBytes 26.8 Mbits/sec 0 313 KBytes [ 5] 8.00-9.00 sec 2.39 MBytes 20.1 Mbits/sec 0 297 KBytes [ 5] 9.00-10.00 sec 2.39 MBytes 20.1 Mbits/sec 0 307 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 29.2 MBytes 24.5 Mbits/sec 0 sender [ 5] 0.00-10.09 sec 27.8 MBytes 23.1 Mbits/sec receiver rsync with 1Go "testfile" over Wireguard VPN at 3Mbits.... (I tested all possible options for rsync and SSH without improvement)... root@Unraid:~# rsync -az --progress testfile 10.253.0.2:/mnt/cache([email protected]) testfile 50,987,008 4% 3.04MB/s 0:05:20 So, what other solution do you use to backup to another remote Unraid server? You can setup a VM in UnRAID or really make a wireguard server on any other device besides directly in the UnRAID OS. I set it up on my site routers and it performs way better than UnRAID ever did. Quote Link to comment
Steven Schulze Posted March 25, 2023 Share Posted March 25, 2023 Hallo, I'm new to this forum but I've been using Unraid since version 6.9.2 and I really like the Unraid system and I encountered the same problem as you describe here in this thread. I would like to share my research with you! First, I would like to mention that I run two Unraid servers, with Unraid OS Version 6.11.5. The servers have a Wireguard connection to each other, my Main Server is connected with LTE to the Internet and my Backup is connected to DSL/G. Fast, don´t ask why, it´s like that! Also, the distance between the two servers is round ~300km. but now to my results weirdcrap wrote the following: Quote “What I don't think it is: I don't think my issue is with WireGuard or my ISP speeds on either end. While the transfer is crawling along over SSH at sub-par speeds I can easily browse to NODE over WireGuard from my Windows or Mac computer and pick any file to copy over the tunnel and I can fully saturate the sending servers upload with no issues while SSH is choking in the background:” weirdcrap has right when he writes that the problem is not Wireguard or the ISP, but if Wireguard is not causing the problem, what is causing the problem? when we look at the structure how we want it to transfer the data, is Wireguard as a first layer for protection, we want it that Wireguard encrypt our data, Wireguard does this!!! in the tunnel then, works a combination of both programs and protocols Rsync and SSH. Normally the combination of Rsync and SSH is not the problem, especially in local use, since Rsync has no encryption. but SSH is not the most efficient protocol and here I come to the cause of our problem, because SSH or sftp (Secure File Transfer Protocol) is not the best option for transferring data, especially large data! Yes, SSH have a encryption but not efficient enough for transfer Large Data, this can be used as an additional security option, with the disadvantage of poor transmission. My solution was that I identified SSH as a possible cause of the problem! but wait a minute if you don't use SSH, SMB, FTP how does you transfer data! this is a legitimate question and the answer to it is "rsync as daemon"! All protocols like SSH, SMB, FTP produce a large overhead that is added to the packets, which means that Wireguard or our eth0 or similar has to split the packets, that's not good because it affects the performance negatively! so, I have running “rsync as daemon” on my backup server. The setup is super easy and you can secure the daemon with appropriate arguments. I used the following guide: 1. How to Set Up an Rsync Daemon on Your Linux Server 2. How to Use Rsync to Copy/Sync Files Between Servers 3. Rsync Manuel Page The instructions are for Ubuntu, but it also works with Unraid without any problems!👍 ------------------------------------------------------------------------------- my configs on backup site: ------------------------------------------------------------------------------- User script: Rsync-Daemon-Autostart #!/bin/bash cp /boot/rsync-config/rsyncd.conf /etc/ cp /boot/rsync-config/rsyncd.secrets /etc/ sleep 1s chmod 600 /etc/rsyncd.conf chmod 600 /etc/rsyncd.secrets sleep 1s rsync –daemon ------------------------------------------------------------------------------- rsyncd.conf pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock log file = /var/log/rsync.log port = ***** [TEST] path = /path/to/your/data comment = blabla read only = false write only = false auth users = rsync hosts allow = x.x.x.x, x.x.x.x hosts deny = * secrets file = /etc/rsyncd.secrets timeout = 300 … ------------------------------------------------------------------------------- rsyncd.secrets rsync1:passwd1 rsync2:passwd2 rsync3:passwd3 ------------------------------------------------------------------------------- You can use the instructions in the Guides and the Manuel page from rsync for more information. ------------------------------------------------------------------------------- my script at Main site: #!/bin/bash BACKUP_SERVER=[IP other SERVER] PORT=[PORT of the Daemon] USER=[USER] U_PASS=/boot/rsync_pass SHARE1=/Path/to/your/Data # preparations # the cache is transferred to the disks mover # Reset the wg connection wg-quick down [your WG-interface-Name] && wg-quick up [your WG-interface-Name] # wait until everything is done sleep 60s target=$1 count=$( ping -c 5 $BACKUP_SERVER| grep icmp* | wc -l ) if [ $count -eq 5 ] then echo "1" $(date "+%d.%m.%Y - %T") "on" echo "BACKUP SHARE1" rsync -rtauvPh --stats --password-file=$U_PASS $SHARE1/* rsync://$USER@$BACKUP_SERVER:$PORT/SHARE1/ echo "Pause 15s" sleep 15s … #for more, the command can be used multiple times else echo "0" $(date "+%d.%m.%Y - %T") "OFF" fi ------------------------------------------------------------------------------- I think that was it, today I have Transfer Data for 8 hours with full Bandwidth without slow down! I've been working on this solution for 3 months, I hope you can reproduce my results! Sorry for my bad English! Regards from Germany Steven 1 Quote Link to comment
atvking Posted October 16, 2023 Share Posted October 16, 2023 (edited) On 3/7/2023 at 5:30 AM, thymon said: Hello, Same here Test iperf3 unraid --> unraid-backup (30Mbits) root@Unraid-Backup:~# iperf3 -s ----------------------------------------------------------- Server listening on 5201 (test #1) ----------------------------------------------------------- Accepted connection from 10.253.0.1, port 54148 [ 5] local 10.253.0.2 port 5201 connected to 10.253.0.1 port 54162 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 2.26 MBytes 19.0 Mbits/sec [ 5] 1.00-2.00 sec 2.63 MBytes 22.1 Mbits/sec [ 5] 2.00-3.00 sec 2.75 MBytes 23.1 Mbits/sec [ 5] 3.00-4.00 sec 3.00 MBytes 25.2 Mbits/sec [ 5] 4.00-5.00 sec 3.09 MBytes 25.9 Mbits/sec [ 5] 5.00-6.00 sec 3.00 MBytes 25.1 Mbits/sec [ 5] 6.00-7.00 sec 2.96 MBytes 24.8 Mbits/sec [ 5] 7.00-8.00 sec 2.81 MBytes 23.5 Mbits/sec [ 5] 8.00-9.00 sec 2.81 MBytes 23.6 Mbits/sec [ 5] 9.00-10.00 sec 2.26 MBytes 19.0 Mbits/sec [ 5] 10.00-10.09 sec 224 KBytes 21.5 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.09 sec 27.8 MBytes 23.1 Mbits/sec receiver root@Unraid:~# iperf3 -c 10.253.0.2 Connecting to host 10.253.0.2, port 5201 [ 5] local 10.253.0.1 port 54162 connected to 10.253.0.2 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 3.91 MBytes 32.8 Mbits/sec 0 337 KBytes [ 5] 1.00-2.00 sec 2.64 MBytes 22.1 Mbits/sec 0 350 KBytes [ 5] 2.00-3.00 sec 2.76 MBytes 23.1 Mbits/sec 0 337 KBytes [ 5] 3.00-4.00 sec 3.13 MBytes 26.2 Mbits/sec 0 369 KBytes [ 5] 4.00-5.00 sec 3.19 MBytes 26.7 Mbits/sec 0 326 KBytes [ 5] 5.00-6.00 sec 3.19 MBytes 26.7 Mbits/sec 0 315 KBytes [ 5] 6.00-7.00 sec 2.39 MBytes 20.1 Mbits/sec 0 310 KBytes [ 5] 7.00-8.00 sec 3.19 MBytes 26.8 Mbits/sec 0 313 KBytes [ 5] 8.00-9.00 sec 2.39 MBytes 20.1 Mbits/sec 0 297 KBytes [ 5] 9.00-10.00 sec 2.39 MBytes 20.1 Mbits/sec 0 307 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 29.2 MBytes 24.5 Mbits/sec 0 sender [ 5] 0.00-10.09 sec 27.8 MBytes 23.1 Mbits/sec receiver rsync with 1Go "testfile" over Wireguard VPN at 3Mbits.... (I tested all possible options for rsync and SSH without improvement)... root@Unraid:~# rsync -az --progress testfile 10.253.0.2:/mnt/cache([email protected]) testfile 50,987,008 4% 3.04MB/s 0:05:20 So, what other solution do you use to backup to another remote Unraid server? Mbits/sec does not equal MB/s... 1MB is 8Mbs (8 bits to a byte) Therefore your rsync transfer speed was 24.32 Mbits/s (pretty much spot on your iperf3 test) Mbits/s is often also written as Mb/s. The uppercase vs lowercase B is an important differentiation . Edited October 16, 2023 by atvking typo Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.