Very slow transfer speed to array (3 -8 MB/s)


Faba

Recommended Posts

Hy,

 

I'm trying to move my linux fileserver over to Unraid but hit a very large bump in the road. When copying to the array im getting mostly 3-8 MB/s writing speed. Here is a little background about the server.

 

- 10 x 8TB Seagate Archive disks (not the fastest I know) 

- Connect to an Adaptec 71605e (tested with an older an current firmware)

- 5 of those disks are in an array 4 Data + 1 Parity

- the other 5 are still simple ext4 formatted

- there is no network involved I'm using the unraid terminal to copy between them

- I get about 80-100 MB/s using the Non-array disks and ubuntu or the unraid terminal

- When copying data from non-array to array it starts with about 110 MB/s and quickly drops to about 8 MB/s

- Also installing the Krusader Docker container took about 5-10 min to extract the images and writes speed also very low

 

I've attached my diagonstics if something is missing let me know.

 

Thanks

 

Faba

 

P.S.: I bought Unraid OS Pro ;)

 

 

 

 

sassrv-diagnostics-20181019-0727.zip

Edited by Faba
Link to comment

Assuming you're transferring large files nothing really jumps out, except you might have one or more cooked disks, most of your disks overheated past max temp, which is 60C, example:

 

Lifetime    Min/Max Temperature:     22/72 Celsius

If you run a parity check is the speed normal?

 

Forgot to say, you want to keep your disks below 40C, 45C tops.

Edited by johnnie.black
Link to comment

I was battling heat problems in the summer and reworked the cooling. 

 

image.thumb.png.5ee8b3c6d59ffe9582240d0c95558765.png

 

Parity look OK after running for some time.

 

I always copying files at least a couple of GBs so I can achieve so kind of speed.

 

Didn't even notice it went up to 72C

Edited by Faba
Link to comment
1 hour ago, Faba said:

So I did install the speed test docker container and here are the results for all five array disks plus one not in the array for comparison:

Those look normal, though the test only scans a very small percentage of the surface, but likely the problem isn't the disks, you need to do some testing to rule things out.

Link to comment
15 minutes ago, Faba said:

Running a preclear on one of the disk almost an hour on average 190 MB/s, I'll wait and see 

That's about normal also, it should decrease as it goes through the slower inner cylinders, but something must be going on to explain the low write speeds and even the low parity check speed.

Link to comment

I started a preclear on the rest of the disks of the array. All connected to the same controller 2 run at about 183 MB/s, 3 at 10 MB/s 

514967861_Screenshot2018-10-20at14_50_51.thumb.png.3204113f8a3834c472394a7f07b98df7.png

 

Something is going like you said, but still don't know what

 

I did switch the cables between a fast and a slow disk but nothing changed, so I wouldn't blame the cable

Edited by Faba
Link to comment

Did another test to figure out what's going on. I removed one of the disk which ran very slow during preclear. Formatted to et4 mounted to ubuntu and copied a file to it.

 

sent 18.32G bytes  received 35 bytes  97.72M bytes/sec
total size is 18.32G  speedup is 1.00

 

Thats the same source from my inital post. 

 

so to summarise when I copy to the disk with 4-5 MB/s when in the array when its not I have 97 MB/s on average. Seems to me an unraid issue rather than a physical disk issue.

 

Link to comment

Preclear isn't really a good benchmarking tool, especially if running multiple instances, you need to do some tests to try and find the problem, for example use those disks in another computer with an Unraid trial key, use different disks on the current server, etc, until you find whats causing the problem.

 

I have a server with those same disks and can write at about 60MB/s using the normal writing mode and up to 180MB/s with turbo write, have you tried turbo write to see if you get more normal results?

Link to comment
  • 2 weeks later...
  • 9 months later...

Having similar issues with my 4 x 4TB disks + 1x 8TB parity.

Transfers run at 17 MB/s over a 10G or 1G connection (tested two different network cards).

If I passthrough one of the adapters to Windows I get expected transfer speeds.

Parity runs at 131MB/s so I know the disk isn't an issue and the "DiskSpeed" benchmarks are good.

 

Edited by DanW
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.