Hello! 

 

I'm experiencing slow transfer speed during files copiing operation from / to array by the LAN, from my desktop, connected wiredly (with ssd nvme in the desktop). I can consider that my unraid hardware specs are not great (Celeron J1900 + 16gb ram, 1gb lan), but with the highest speeds I saw 50MB/s (which I can live with), but during the copiing progress it seems like slowing down, going to as low as 20-10-5 or even 1MB/s closer to the end of the process. 

 

Server reboot can (temporary) ease the issue, but after a while it starts to slowing down again.

 

Diags and the progress copiing bar are attached. It's clearly visible speed decline during the process. 

 

Can you please advice what should be I'm thinking about? 

Is the only way I have is to upgrade the motherboard + processor?

 

linertower-diagnostics-20240309-1159.zip

Edited March 9, 20242 yr by mamaraci

Is this copying to cache or array?

 

P.S. btrfs is detecting a lot of data corruption, you should run memtest.

This is a coping to cache, sorry for the mess. But also experienced the similar while copiing from array to the desktop. 
 

so I should run memtest, ram is the most obvious candidate to be the reason of the decline, right?
 

thanks, @JorgeB 

If there's a RAM issue, it could or not be related to the slowdown, but in any case it's a high priority issue, or you can't trust the data on the server.

33 minutes ago, JorgeB said:

If there's a RAM issue, it could or not be related to the slowdown, but in any case it's a high priority issue, or you can't trust the data on the server.

thanks. would update the back-ups and then try to do memtests. server is headless, so it involves some of the equipment relocation in the apartment. 

Memtest seems to be ok.

Can you please advice, what should be my next steps on both issues? (Btfrs data corruption, but I didn’t notice any data loss, and the transfer speed).

 

thanks in advance! 

Edited March 9, 20242 yr by mamaraci

memtest is only definitive if it finds errors, reset the btrfs stats and monitor the pool, if more corruptions are found there's still a problem, try with just one stick of RAM, if the same try the other one, that will basically rule out a RAM problem.

Thanks for your replies.

 

So, currently I didn't had memory issues, as per screenshot above. I've followed your link, run the command (btrfs dev stats /mnt/cache), got this:

 

root@LinerTower:~# btrfs dev stats /mnt/cache
[/dev/sdb1].write_io_errs 0
[/dev/sdb1].read_io_errs 0
[/dev/sdb1].flush_io_errs 0
[/dev/sdb1].corruption_errs 313360
[/dev/sdb1].generation_errs 0

Then I've created a script for regular checks, and reset the stats - it was some like 10 days ago.

No notifications during that period. I didn't changed the cable yet, but would do in some close future, or, if any other notifications will arise.

 

Am I right in understanding, that some of the data could be moved to the array with some errors, as per this information, right? I didn't notice any facts of the data corruption yet there, rechecked the latest contributions of the data to the array.

 

Didn't check also RAM modules seprately - too difficult keeping in mind my set-up... 

 

Thanks a lot!

Edited March 23, 20242 yr by mamaraci

24 minutes ago, mamaraci said:

Am I right in understanding, that some of the data could be moved to the array with some errors, as per this information, right?

It's possible, difficult to say for certain.

And, again going back to my initial issue of the slow speed. 

trying to copy to the cache from my desktop ssd - 5MB/s.

Rebooted, restarted the copy - just 7-8MB/s, not higher. 

Diags before reboot is attached

 

Is that a limit? What's the weak point of my system, can you please advice? 

slow_copy_to_cache_6MB_linertower-diagnostics-20240323-1322_.zip

Start by running a single stream iperf test in both directions to confirm the LAN is working well.

Looks like ok:

 

unraid server as server:
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec   533 MBytes   447 Mbits/sec                  sender
[  4]   0.00-10.00  sec   533 MBytes   447 Mbits/sec                  receiver

reverse mode: 
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   728 MBytes   610 Mbits/sec  904             sender
[  4]   0.00-10.00  sec   725 MBytes   608 Mbits/sec                  receiver

 

Not great, but also doesn't look like the main issue, post a screenshot showing a Windows explorer transfer with large files to cache, and the diags saved at that time.

If the SSD(s) are getting hot, they may be slowing down.  Increased ventilation will often improve the performance of sustained data transfer on SSDs.

Sorry for the long silence. But here it is, the same issue.

Copying from external (main desktop, 1gbit lan, ssd nvme) to the unraid server, bu 1gbit lan, to the cache on unraid (ssd 512gb). Transfer speeds are jumping, from 30-60mbytes/ sec, sometimes dropping to below 1MB /s (see the screenshot attached). And the latest diags, taken during the copying process, is attached.

 

Files in copy are photos: raw, jpg files, and XMPs, a mix of everything. I guess, the main drop in speed while copying a lot of smaller XMP files, or touching (viewing) the files on the server with the other parallel process. 

Should I upgrade already? ) Motherboard (+processor since it's soldered), RAM or LAN?
Or smth can be solved in easier / cheaper way, without new hardware..

linertower-diagnostics-20240708-1456.zip

Edited July 12, 20241 yr by mamaraci

With smaller files the speed will be much lower, that is normal, large files appear to max out at around 65MB/s, and that is similar to the iperf tests, so also normal, you can try to retest with iperf after changing some things like NICs, cables, switch, client PC, etc, when you get the expected results with iperf, you should also get them with Unraid.