Hy,

 

I'm trying to move my linux fileserver over to Unraid but hit a very large bump in the road. When copying to the array im getting mostly 3-8 MB/s writing speed. Here is a little background about the server.

 

- 10 x 8TB Seagate Archive disks (not the fastest I know) 

- Connect to an Adaptec 71605e (tested with an older an current firmware)

- 5 of those disks are in an array 4 Data + 1 Parity

- the other 5 are still simple ext4 formatted

- there is no network involved I'm using the unraid terminal to copy between them

- I get about 80-100 MB/s using the Non-array disks and ubuntu or the unraid terminal

- When copying data from non-array to array it starts with about 110 MB/s and quickly drops to about 8 MB/s

- Also installing the Krusader Docker container took about 5-10 min to extract the images and writes speed also very low

 

I've attached my diagonstics if something is missing let me know.

 

Thanks

 

Faba

 

P.S.: I bought Unraid OS Pro

 

 

 

 

sassrv-diagnostics-20181019-0727.zip

Edited October 19, 20187 yr by Faba

To rule out the controller connect all array disks to the onboard SATA ports (but change ports 5 and 6 to SATA/AHCI, they're set to IDE), if still the same grab new diagnostics during a transfer.

I'll do it and post the results. Might take some time because hardware access is quiet cumbersome.

Also just to clarify, are you using Krusader for the copy or something over the network? 

I tried krusader, rsync or mc. Always on the server from internal to internal disk on the same controller

I did the test with all array disks connected to the Motherboard SATA ports with AHCI enabled. it starts like before aroung 110 MB/s and droppen down to about 16-20 MB/s with dips down to 7,5 MB/s. 

 

Logs taken during the test are attached.

 

thanks for any kind of help solving this.

 

Faba

sassrv-diagnostics-20181019-1711 (2).zip

Assuming you're transferring large files nothing really jumps out, except you might have one or more cooked disks, most of your disks overheated past max temp, which is 60C, example:

 

Lifetime    Min/Max Temperature:     22/72 Celsius

If you run a parity check is the speed normal?

 

Forgot to say, you want to keep your disks below 40C, 45C tops.

Edited October 19, 20187 yr by johnnie.black

I was battling heat problems in the summer and reworked the cooling. 

 

 

Parity look OK after running for some time.

 

I always copying files at least a couple of GBs so I can achieve so kind of speed.

 

Didn't even notice it went up to 72C

Edited October 19, 20187 yr by Faba

Those disks should start parity check at 200MB/s, so one or more may be damaged from the heat.

OK so the should be replaced in the near future? 

 

Strange the work better without unraid. Maybe I remove one from the array and test them.

31 minutes ago, Faba said:

OK so the should be replaced in the near future? 

Personally I would replace any disk that got close to, never mind over, max temp, since it will likely fail much sooner than normal, but I can't be certain that is your problem, you should run the sped test docker.

So I did install the speed test docker container and here are the results for all five array disks plus one not in the array for comparison:

 

 

I also copied a file over wifi, starts around 20 MB/s stays there until about 37 % an then drops off to 6-8 MB/s

 

 

sassrv-diagnostics-20181020-0727 (1).zip

1 hour ago, Faba said:

So I did install the speed test docker container and here are the results for all five array disks plus one not in the array for comparison:

Those look normal, though the test only scans a very small percentage of the surface, but likely the problem isn't the disks, you need to do some testing to rule things out.

Running a preclear on one of the disk almost an hour on average 190 MB/s, I'll wait and see 

15 minutes ago, Faba said:

Running a preclear on one of the disk almost an hour on average 190 MB/s, I'll wait and see 

That's about normal also, it should decrease as it goes through the slower inner cylinders, but something must be going on to explain the low write speeds and even the low parity check speed.

I started a preclear on the rest of the disks of the array. All connected to the same controller 2 run at about 183 MB/s, 3 at 10 MB/s 

 

Something is going like you said, but still don't know what

 

I did switch the cables between a fast and a slow disk but nothing changed, so I wouldn't blame the cable

Edited October 20, 20187 yr by Faba

Did another test to figure out what's going on. I removed one of the disk which ran very slow during preclear. Formatted to et4 mounted to ubuntu and copied a file to it.

 

sent 18.32G bytes  received 35 bytes  97.72M bytes/sec
total size is 18.32G  speedup is 1.00

 

Thats the same source from my inital post. 

 

so to summarise when I copy to the disk with 4-5 MB/s when in the array when its not I have 97 MB/s on average. Seems to me an unraid issue rather than a physical disk issue.

 

Preclear isn't really a good benchmarking tool, especially if running multiple instances, you need to do some tests to try and find the problem, for example use those disks in another computer with an Unraid trial key, use different disks on the current server, etc, until you find whats causing the problem.

 

I have a server with those same disks and can write at about 60MB/s using the normal writing mode and up to 180MB/s with turbo write, have you tried turbo write to see if you get more normal results?

At the moment I'm moving the disks to other Hardware and run Unraid there to see if I can see a difference. 

 

I didn't use Turbo mode explicitly, how or where do I enable it?

https://forums.unraid.net/topic/50397-turbo-write/

Next Update:-

I took only the two "fast" disks and created a new Array Data + Partity and I'm copying 100 MB/s on average for 40 GBs. Somehow unraid doesn't like some disk. Using Ubuntu I get the same 80-100 MB/s with the "slow" disks

Note that using just two disks is always faster since Unraid treats them like a mirror, though you should be able to get similar speeds with turbo write and more disks.

I think we can close this topic I bought 3 new IronWolf Nas drives, preclear them and used them. I'm getting about 120 MB/s on average.

 

thanks

 

Faba

Having similar issues with my 4 x 4TB disks + 1x 8TB parity.

Transfers run at 17 MB/s over a 10G or 1G connection (tested two different network cards).

If I passthrough one of the adapters to Windows I get expected transfer speeds.

Parity runs at 131MB/s so I know the disk isn't an issue and the "DiskSpeed" benchmarks are good.

 

Edited August 21, 20196 yr by DanW

9 hours ago, DanW said:

Having similar issues with my 4 x 4TB disks + 1x 8TB parity.

Post diagnostics, also try Turbo write, even if just to see if there's a difference.