Skip to content
View in the app

A better way to browse. Learn more.

Unraid

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

Rsync and wget very slow, only to array and only from internet sources

Featured Replies

I have a very odd issue where my downloads via either wget or rsync from internet sources AND to my array are only having speeds of approximately 14MB/s, where in other cases I am able to write to the array at either full speed or the near full speed of my ISP download. 

 

When speed is only approximately 14MB/s:

 - I'm writing to array via rsync or wget (internet sources) to path for example /mnt/user/isos

 

When speed is full, either ISP download 85MB/s, or full spinny HDD speeds at average 125MB/s:

 - I'm writing directly to HDD spinny disks via rsync or wget (internet sources) to paths for example /mnt/disk1/isos /mnt/disk2/isos and so on

 - I'm writing directly to NVMe cache via rsync or wget (internet sources) to path for example /mnt/cache/downloads

 - I'm writing to array via rsync (LOCAL source) from example /mnt/user/isos/ubuntu.iso to /mnt/user/isos/linux/ubuntu.iso

 

In short, it seems I can only get approximately 14MB/s speeds when the following two conditions exist: source data is from internet AND the destination is within the array

 

Additional note: The cases above where I was able to still get full speeds (non array destinations) form internet sources, I used same download file source so I don't think peering is in play here, seems its specifically the array and  internet sources via rsync and wget

 

rsync command I'm using with flags is as follows:
rsync -aHAXxv --numeric-ids --info=progress2 --chmod=D0777,F0666 --bwlimit=85M

 

Diagnostics is attached, and was created while I was attempting the rsync from internet source which only runs at maximum of 14MB/s

 

Any help on this is greatly appreciated as I need to transfer 50+TB of data via rsync from my offsite server to my local unraid server, and the 70MB/s or so difference is going to make a world of a difference for me :)

tower-diagnostics-20230913-2209.zip

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.