Slow Internal Disk Transfers

[MandalorePatriot]

Hi everyone,

 

Rebuilt my unRAID server after not having used it for some time. Currently I am using a Dell R720XD for my server, with the below disk configuration. The server backplane is connected to a LSI 9207-8i for disk passthrough. Precleared all disks before adding to array and no issues or slow speeds found. Also running Tips and Tweaks as well as Fix Common Problems.

 

Transfers from external to array or from cache to array work great, speeds are as expected. But internal transfers between disks start great then drop down to either stalling or very low kb/s, before picking back up then slowing down again. Trying to transfer some data off my original drive which is a bit full. I tried using the Unbalance plugin and it said I needed to run the Docker New Safe Permissions tool. I did that but Unbalance only transfers the folders, even though it says it's moved like 1.5TB.

 

Next I tried using the basic Linux copy or move commands. While it completed OK, it still took almost 14 hours, which is a bit longer than I expected. After this, I've been testing with the Dolphin docker and same experience, starts great, slows down, picks up, slows down, etc.

 

Throughout all this, I've been transferring data by using /mnt/disk1 to /mnt/disk2 or similar. Is this the correct method to transfer? If so, any idea what could be causing my issues? Thanks in advance for looking!

[JonathanM]

In my experience, disk to disk inside of the array are roughly 1/8 of the raw disk speed of the slowest disk involved. When you say slow, what exact speeds are you seeing? You could try turning on turbo write, which changes how things are calculated, but for disk to disk, I don't think it will help.

 

Here is a rough outline of what's happening during a move.

Source disk is read.

Destination disk is written, which is updated in parity, both file and metadata, which involves different physical areas of both disks, causing seek delays as both writes are finished.

Source disk metadata is updated as the file is deleted, which writes to yet another area on both the source disk and parity.

 

As you can see, the parity disk is VERY busy during disk to disk moves. You may get better results with a copy followed by a delete, at least then the parity won't be thrashing back and forth quite as badly.

[MandalorePatriot]

36 minutes ago, jonathanm said:

In my experience, disk to disk inside of the array are roughly 1/8 of the raw disk speed of the slowest disk involved. When you say slow, what exact speeds are you seeing? You could try turning on turbo write, which changes how things are calculated, but for disk to disk, I don't think it will help.

 

Here is a rough outline of what's happening during a move.

Source disk is read.

Destination disk is written, which is updated in parity, both file and metadata, which involves different physical areas of both disks, causing seek delays as both writes are finished.

Source disk metadata is updated as the file is deleted, which writes to yet another area on both the source disk and parity.

 

As you can see, the parity disk is VERY busy during disk to disk moves. You may get better results with a copy followed by a delete, at least then the parity won't be thrashing back and forth quite as badly.

Thanks for the response. I do have Turbo Write on, but that seems to only affect going into the array. It did not improve disk to disk, as you mentioned.

 

That's sad to know about the disk to disk speed, but I guess it makes sense. What benefit is the Unbalance plugin then and any idea why it only transferred the folders?

 

Anyway, I moved enough data off my original drive so it is no longer in a warning state. Forgot to include disk details, which are below. Next additions are another parity disk and another cache disk.

 

Parity - 10TB

Array - 2x 10TB, 3x 8TB

Cache - 1x 500GB SSD

[JonathanM]

57 minutes ago, MandalorePatriot said:

What benefit is the Unbalance plugin then and any idea why it only transferred the folders?

If it completes without error, it works fine. Problem is, if it's interrupted, either by the user or by an error, it leaves things in a half complete state. There should be a replay somewhere in the plugin to allow it to finish, either after a user stop, or after correcting whatever error, usually file permissions.

 

59 minutes ago, MandalorePatriot said:

I moved enough data off my original drive so it is no longer in a warning state.

The warning is purely for your information, no harm is done by filling a disk within reason. I typically leave a GB or two free, so the file system has room to do maintenance if needed, but there is no need to keep disks "balanced". You can adjust the warning levels to your taste.

 

The only reason to keep a disk relatively empty is if you are anticipating future deletions and writes. Purely archival stuff, write once read many, should be filled as full as practical.