Very slow copying


keymaster

Recommended Posts

When unRAID writes a file to the array the data is written to the computer memory first.  That data is written at memory speed - so very fast.  The OS (unRAID) then takes the file data from this memory cache and writes it to the actual disk.  The reason it looks like 112MB/s to start is that you are seeing the memory cache on unRAID filling up with data but that changes to 25MB/s because the memory cache becomes full and has to slow down to the actual speed your disks are capable of writing.  So for the rest of the file data is written to the disk from the memory buffer and data from your network is filling the empty space made in the memory buffer.  If you would pause the file transfer to unraid you would see that writing to the disk continues for a while after you pause it as the memory buffer is cleared.  If you then resume the copy operation it would again be at a faster speed like 112MB/s until the memory buffer becomes full again.

 

Now as to why you are getting slow writes to disks:  Do you have the same write speed to every disk in array or is it only to certain disks?  Try writing to your disks directly and see if they all perform at 25MB/s.  You could copy from disk to disk with MC (MidnightCommander) from a console or putty session to test this out.

Link to comment

good morning

 

thanks for that explanation. Makes it much more clear.

 

Just one more questions. Why is on unraid that write speed at least that slow ?

I understand caching etc etc

Would it make sense to use a hardware controller ? - or a cache with a ssd ?

 

I feel good with that parity check time too - before changing writing options to array by your hints it took about 30 hours !

This morning parity nearly finished and got some smart issues mentioned as follows.

 

5    Reallocated sector count    0x0033    194    194    140    Pre-fail    Always    Never    42

197    Current pending sector    0x0032    200    200    000    Old age    Always    Never    2
198    Offline uncorrectable    0x0030    200    200    000    Old age    Offline    Never    1

 

thanks again for all your support

 

and just to point out - i love unraid, its offering a lot of unique features, but the write times are driving me crazy

Edited by doppelt
Link to comment
5 minutes ago, doppelt said:

Just one more questions. Why is on unraid that write speed at least that slow ?

 

The speeds you're seeing are not the normal speeds, with modern disks you get around 50MB/s with normal write mode and 100MB/s+ with turbo write, you're using older disks but still speeds should be much better, I would guess around 30/40MB/s and 80MB/s respectively, so there's something else going on with your hardware, confirmed by the fact you see the same low speeds writing with no parity, you'll need to make some test to try to find the issue.

Link to comment

ok,

 

copied with laptop a movie about 1.1 GB from Main PC and it had an average of about 38 MB/sec.

 

Both Systems got SSD in and copied from and to.

 

Isn´t it for a 1 GB Network not a bid slow ?

 

40MB multiplicated by 8 is around 320 Mbit.

 

got a manged HP switch(HP PS1810 8G) in network for complete internal traffic

Edited by doppelt
Link to comment

thanks to all which were involved in that disscussion.

 

I think I solved problem. It was the HP Switch !

 

copied now from Laptop, Main PC to and vice versa to Unraid Server and it went perfect. File 1.1 GB with a 112 MB/sec without weaking down.

Completly the whole file.

 

There was an option in managed switch for autonegotition and I swiched to forced 1 Gbit.

 

From that moment on full speed !

 

Hurray.

 

Don´t know what bug could be in firmware of switch, normally it should negotiate to max speed for its own.

 

Thank you all very much.

Edited by doppelt
Link to comment

good morning,

it seems that i still have problems.

I mount today an external harddrive via Unassigned Devices.

I get it to run an i can see ntfs partitions with all content.

I started copying files to check speed on it to use it for backups e.g.
Process started with about 65 MB/sec, but again within 30 sec. it weaks to about 10 MB/sec.

Its connected via usb 3 on unraid server.

Anybody an idea why that now is happening with external drives too !

thanks

Link to comment

hi guys,

 

did some more investigations about that strange speed problem.

 

Fideled arount with new drivers for usb3, sata, network on main PC and changed them all to newer versions.

 

Copied then 4,3 Gigabyte files from usb port via network on unraid NAS and speed started with 112 Mb/sec and went back to about 64 Mb/sec and remaining to finish.

 

I can comfortable live with that speed..

 

At least its very strange how complex this materia is and if you are obviously thinking its caused on unraid you will be teached that it could be on several issues with drivers e.g. on your PC and Network !

 

Again thanks for all hints and comments

Link to comment

I have the same issue and I know it is an unraid problem. I have hard drives that I known can write at 100mbs+ and its writing to the array at 39-40mb/s. I had linux working on this machine before and it is NOT the hardware and NOT the network. It was the type of share working, where ubuntu had the same issue, but changing the share type and connecting in a different way get the 100mb/s write speeds back

 

when will this issue get fixed?

Link to comment
1 hour ago, tomo said:

I have the same issue and I know it is an unraid problem. I have hard drives that I known can write at 100mbs+ and its writing to the array at 39-40mb/s. I had linux working on this machine before and it is NOT the hardware and NOT the network. It was the type of share working, where ubuntu had the same issue, but changing the share type and connecting in a different way get the 100mb/s write speeds back

 

when will this issue get fixed?

Start a new thread and attach your diagnostics zip as retrieved after writing to the array and we’ll have a look at it. :)

Link to comment
13 hours ago, tomo said:

I have the same issue and I know it is an unraid problem. I have hard drives that I known can write at 100mbs+ and its writing to the array at 39-40mb/s. I had linux working on this machine before and it is NOT the hardware and NOT the network. It was the type of share working, where ubuntu had the same issue, but changing the share type and connecting in a different way get the 100mb/s write speeds back

 

when will this issue get fixed?

Writing to the parity protected array will always be significantly slower than what an individual disk drive is capable of.    The unRAID architecture means that a normal ‘write’ actually involves 4 I/O operations.   You first get a read of the target sector on the target drive and the corresponding sector on the parity drive.   The new contents of the parity sector is calculated and you then get writes to both the target drive and the parity drive.   This means that the drive rotational speed becomes a factor as there is always a full rotation of the disks between the two read and their corresponding writes with a corresponding reduction in theoretical throughput.

Link to comment

There is one way to avoid the issue that @itimpi discusses.  You can go to:

 

      'Settings'    >>>   'Disk Settings'        and find the " Tunable (md_write_method): " and set it to 'Read/Modify/Write'.  This setting will result in a considerable increase in write speed BUT all of the array drives will spin during the write process. 

 

You can also add a cache drive to get faster write speeds.  If you are going to go this route, let me suggest that you get an ssd rather then a spinner.  Spinners will often slow down when you are transferring a large number of small files to the server due to the file allocation overhead as well as the seek time and latency of  the heads. 

Edited by Frank1940
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.