Slow transfer speed to array / from array

[mamaraci]

Hello! 

 

I'm experiencing slow transfer speed during files copiing operation from / to array by the LAN, from my desktop, connected wiredly (with ssd nvme in the desktop). I can consider that my unraid hardware specs are not great (Celeron J1900 + 16gb ram, 1gb lan), but with the highest speeds I saw 50MB/s (which I can live with), but during the copiing progress it seems like slowing down, going to as low as 20-10-5 or even 1MB/s closer to the end of the process. 

 

Server reboot can (temporary) ease the issue, but after a while it starts to slowing down again.

 

Diags and the progress copiing bar are attached. It's clearly visible speed decline during the process. 

 

Can you please advice what should be I'm thinking about? 

Is the only way I have is to upgrade the motherboard + processor?

 

linertower-diagnostics-20240309-1159.zip

Edited March 9 by mamaraci

[JorgeB]

Is this copying to cache or array?

 

P.S. btrfs is detecting a lot of data corruption, you should run memtest.

[mamaraci]

This is a coping to cache, sorry for the mess. But also experienced the similar while copiing from array to the desktop. 
 

so I should run memtest, ram is the most obvious candidate to be the reason of the decline, right?
 

thanks, @JorgeB 

[JorgeB]

If there's a RAM issue, it could or not be related to the slowdown, but in any case it's a high priority issue, or you can't trust the data on the server.

[mamaraci]

33 minutes ago, JorgeB said:

If there's a RAM issue, it could or not be related to the slowdown, but in any case it's a high priority issue, or you can't trust the data on the server.

thanks. would update the back-ups and then try to do memtests. server is headless, so it involves some of the equipment relocation in the apartment. 

[mamaraci]

Memtest seems to be ok.

Can you please advice, what should be my next steps on both issues? (Btfrs data corruption, but I didn’t notice any data loss, and the transfer speed).

 

thanks in advance! 

Edited March 9 by mamaraci

[JorgeB]

memtest is only definitive if it finds errors, reset the btrfs stats and monitor the pool, if more corruptions are found there's still a problem, try with just one stick of RAM, if the same try the other one, that will basically rule out a RAM problem.

[mamaraci]

Thanks for your replies.

 

So, currently I didn't had memory issues, as per screenshot above. I've followed your link, run the command (btrfs dev stats /mnt/cache), got this:

 

root@LinerTower:~# btrfs dev stats /mnt/cache
[/dev/sdb1].write_io_errs 0
[/dev/sdb1].read_io_errs 0
[/dev/sdb1].flush_io_errs 0
[/dev/sdb1].corruption_errs 313360
[/dev/sdb1].generation_errs 0

Then I've created a script for regular checks, and reset the stats - it was some like 10 days ago.

No notifications during that period. I didn't changed the cable yet, but would do in some close future, or, if any other notifications will arise.

 

Am I right in understanding, that some of the data could be moved to the array with some errors, as per this information, right? I didn't notice any facts of the data corruption yet there, rechecked the latest contributions of the data to the array.

 

Didn't check also RAM modules seprately - too difficult keeping in mind my set-up... 

 

Thanks a lot!

Edited March 23 by mamaraci

[JorgeB]

24 minutes ago, mamaraci said:

Am I right in understanding, that some of the data could be moved to the array with some errors, as per this information, right?

It's possible, difficult to say for certain.

[mamaraci]

And, again going back to my initial issue of the slow speed. 

trying to copy to the cache from my desktop ssd - 5MB/s.

Rebooted, restarted the copy - just 7-8MB/s, not higher. 

Diags before reboot is attached

 

Is that a limit? What's the weak point of my system, can you please advice? 

slow_copy_to_cache_6MB_linertower-diagnostics-20240323-1322_.zip

[JorgeB]

Start by running a single stream iperf test in both directions to confirm the LAN is working well.

[mamaraci]

Looks like ok:

 

unraid server as server:
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec   533 MBytes   447 Mbits/sec                  sender
[  4]   0.00-10.00  sec   533 MBytes   447 Mbits/sec                  receiver

reverse mode: 
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   728 MBytes   610 Mbits/sec  904             sender
[  4]   0.00-10.00  sec   725 MBytes   608 Mbits/sec                  receiver

 

[JorgeB]

Not great, but also doesn't look like the main issue, post a screenshot showing a Windows explorer transfer with large files to cache, and the diags saved at that time.

[electron286]

If the SSD(s) are getting hot, they may be slowing down.  Increased ventilation will often improve the performance of sustained data transfer on SSDs.