reconstruct write no longer works

[Akuno]

Today I did transfer two movies with over 100 MB/s from my windows PC to my unraid server. But now any other File will be around 10 MB/s. I did check for unraid updates and restarte the server and my PC several times. I did test my HDD speeds with the "disk speeds" plugin but nothing abnormal could be found and nothing works. It seems that it will write on one of my disks and on the parity disk at the same time. It seems like it uses "read/modify/write" mode. The shares are all set to "High-Water" so as far as I could find this should work, and did work just hours before.
Where should I go from here? What could I check? Does anyone have any idea what is going on?

[trurl]

attach diagnostics to your NEXT post in this thread

[Kilrah]

Usually it's because you have something else accessing the disks at the same time.

Edited December 25, 2022 by Kilrah

[Akuno]

31 minutes ago, trurl said:

attach diagnostics to your NEXT post in this thread

 

diagnostics-20221225-2132.zip

[trurl]

appdata, domains, system shares are all on the array.

 

What happens if you disable Docker and VM Manager in Settings?

[JorgeB]

In the diags posted there's something reading from disk3, that will disable turbo write.

[Akuno]

11 hours ago, trurl said:

appdata, domains, system shares are all on the array.

 

What happens if you disable Docker and VM Manager in Settings?

I did turn off both but I still had write speeds around 10 MB/s. Then I did shutdown the server and after a while boot it back up. Now my write speeds are back at 110 MB/s. I have no idea what caused the slowdown but it works again. Thank you for your help.

Edited December 26, 2022 by Akuno

[Akuno]

1 hour ago, JorgeB said:

In the diags posted there's something reading from disk3, that will disable turbo write.

Maybe the log files of a docker? Especially nextcloud loves to write unnecessary logs all the time. At least that is what I did find out when I was trying to get my server into sleep mode. Which was unsucsessfull because of nextcloud. I only did manage to get 2/4 disks to spinn down by moving everything nextcloud relatet on disk 3.

Edited December 26, 2022 by Akuno

[JorgeB]

Looks like more than a docker log would generate, if it happens again start turning off the docker containers one by one to see if it makes a difference. 

[pras1011]

I have the exact same problem but I am not using dockers or vm. I used to have 110MB/s plus on wireless and ethernet. I reboot and the speed is back but then after a while it drops again.

unraid-diagnostics-20231007-1921.zip

Edited October 7, 2023 by pras1011

[JorgeB]

Post new diags during a large file transfer.

[pras1011]

I am transferring a 75gb file from my PC to Unraid server. PC is connected via wifi 6 or 1GBe ethernet and server is connected to router via 2.5GBe ethernet.

 

On wired its writing at 113MB/s then it drops to about 30 to 60. I reboot the router and then its back up to 113 and now its writing perfectly at 113. 

 

unraid-diagnostics-20231008-1202.zip

Edited October 8, 2023 by pras1011

[JorgeB]

37 minutes ago, pras1011 said:

I reboot the router

That suggests a router/network problem, not Unraid.

[pras1011]

After the reboot it was writing perfectly at a constant 113 and then with 46% written it suddenly drops to 31.

unraid-diagnostics-20231008-1238.zip

Edited October 8, 2023 by pras1011

[Frank1940]

I am currently currently copying across 21TB of data (Video Media files with supporting data files (2-3 files) for each video file) from my Media Server to my Test Bed Server using rysnc.  (See specs of servers in my signature.)  The speed is a consistent 110MB/s.  However, it did slow down when writing to the areas of the 12TB drives where it was reading from the inner tracks of the 3TB drives.  So it does work properly when conditions are correct. But there are problem areas.

 

First, make sure that you actually have write-reconstruct turned on.  You can tell by looking at the Main tab of the GUI.  All of the disks will be spun up and there will be reads on all of the drives except for the target drive which will be all writes. 

 

Second, make sure that you are not using a cache drive (or pool).  (That defeats the whole purpose of using write-reconstruct!) This does not appear to be an issue with your setup...

 

Third, Unraid will use unused RAM to buffer writes to your array.  Until that RAM fills up, you are not see the speed of disks writes but of your RAM speed which is a few orders of magnitude faster than any hard disk.

 

Fourth, what is the file size?  Small size file (<1MB) write speeds are much slower than writing large files (>1GB) because the file creation process and the disk head movement times to put that information onto the disk.   It is my understand that Unraid 'User Share' file creation process is particularly slow as it adds another software layer onto the top of the standard Linux file writing process. 

Edited October 8, 2023 by Frank1940

[pras1011]

1) it's turned on.

2) I am not using a cache drive

3) OK, I never knew this but I have written 100s of gigabtye in one hit and never had a slowdown like this before

4) I transfer very large 4k films

 

My server is used for storing films and therefore incredibly basic server setup. 

Edited October 8, 2023 by pras1011

[Frank1940]

Looked through your diagnostics file and I didn't see anything.

 

 Are you using a separate switch?  (As opposed to a combined router switch...)

 

Are both the data source and the Unraid server on the same switch?

 

Are you sure the source can support the data rate required? 

 

 

[pras1011]

I am using an Asus XT12 mesh router. 

[pras1011]

I got a new server and hdds and on ethernet I get 113MB/s and then after about 30GB of writing the speed drops to 31.5MB/s. I wait till its finshed and then I immediatelty write again and it starts at 31.5MB/s. I cancel it and then give it 10 seconds and then start again and it writes at 113MB/s. But sometimes it writes at 31.5. I unplug the ethernet cable on the NUC and then plug it back in and it works at 113.

 

I am using a Intel NUC with the i225-V maybe its throttling after 30GB? 

 

 

Edited October 22, 2023 by pras1011

[itimpi]

It is quite likely that this is a symptom of the start of the write being buffered to RAM and then it slowing when it actually has to start flushing the data to the physical drives.

 

Have you tried enabling Turbo Write mode to see if that makes a difference?

 

 

[pras1011]

Reconstruct write was always on. 

 

I managed to write a lot longer than 30gb this time before the write speed crashed to 31.5MB/s. I will have to try another computer without the Intel NIC to see is this is the problem.