Unraid Freezes on File Transfer

[EarthYak]

I am running Unraid 6.6.3, when I transfer files, large or small the transfer will sometimes freeze, it is impossible to get through a >10Gb file without this happening. When this happens the GUI is unreachable as are all dockers from any device.  My Unraid server is on an 802.3ad ethernet connection (2 physical links), having checked on the switch the server is attached to, only one link is ever saturated and during the freeze the links go down to only background levels of traffic. I have tested this during a freeze and traffic balanced down either link is affected. The freeze can last for 2-5 minutes, then everything resumes as normal. This was an issue before I aggregated the links as well. 

 

Any help would be appreciated.

hydra-diagnostics-20181126-2042.zip

[EarthYak]

I am using Teracopy to copy/move files to and from a windows 10 machine and the server still responds to pings whilst "frozen".

[trurl]

Take a look at your own syslog. Lots of CPU temperature throttling messages, and lots of connection or other issues with cache2.

[JorgeB]

Yes, lots of issues with a cache device, and they have been happening for some time, since on this boot you already had millions of read/write errors:

 

Nov 23 16:31:39 Hydra kernel: BTRFS info (device sdf1): bdev /dev/sdg1 errs: wr 7631833, rd 7087230, flush 306, corrupt 0, gen 0

 

With SSDs these are most times cable/connection issues, so start by changing the cables on the Kingston SSD, you should then run a scrub and make sure there are no uncorrectable errors.

[EarthYak]

On 11/26/2018 at 11:56 PM, johnnie.black said:

Yes, lots of issues with a cache device, and they have been happening for some time, since on this boot you already had millions of read/write errors:

 

Nov 23 16:31:39 Hydra kernel: BTRFS info (device sdf1): bdev /dev/sdg1 errs: wr 7631833, rd 7087230, flush 306, corrupt 0, gen 0

 

With SSDs these are most times cable/connection issues, so start by changing the cables on the Kingston SSD, you should then run a scrub and make sure there are no uncorrectable errors.

Thanks for the help, I have changed the cable to the back plane the Kingston SSD sits on and moved the other SSD to that back plane. 

 

I have tested twice with a 30Gb file, both times the transfer failed and both times multiple of this error appeared:

 

Nov 28 16:48:35 Hydra kernel: sd 7:0:1:0: attempting task abort! scmd(00000000da5f4e73)
Nov 28 16:48:35 Hydra kernel: sd 7:0:1:0: tag#26 CDB: opcode=0x0 00 00 00 00 00 00
Nov 28 16:48:35 Hydra kernel: scsi target7:0:1: handle(0x0009), sas_address(0x4433221102000000), phy(2)
Nov 28 16:48:35 Hydra kernel: scsi target7:0:1: enclosure logical id(0x500605b00450fb30), slot(1) 
Nov 28 16:48:35 Hydra kernel: sd 7:0:1:0: task abort: SUCCESS scmd(00000000da5f4e73)
Nov 28 16:48:35 Hydra kernel: sd 7:0:1:0: Power-on or device reset occurred
Nov 28 16:48:36 Hydra kernel: sd 7:0:1:0: Power-on or device reset occurred

 

But none of the original errors. I am pretty sure that "sd 7:0:1:0:" refers to the Kingston's location as below. 

 

   *-disk:1
          description: ATA Disk
          product: KINGSTON SA400S3
          physical id: 0.1.0
          bus info: scsi@7:0.1.0
          logical name: /dev/sdc
          version: 71E0
          serial: 50026B777A014273
          size: 223GiB (240GB)
          capacity: 223GiB (240GB)
          capabilities: 15000rpm partitioned partitioned:dos
          configuration: ansiversion=6 logicalsectorsize=512 sectorsize=512

 

Does this make it more likely that I am now looking at a fault with the SSD aswell as/instead of the cable?

[EarthYak]

Also I ran the scrub and no errors were found.

[JorgeB]

2 hours ago, EarthYak said:

 

Does this make it more likely that I am now looking at a fault with the SSD aswell

Possibly, try with a different one to confirm.

[JorgeB]

btw, to keep an eye on future pool errors see here:

 

[EarthYak]

I have replaced the cable to that backplane and the SSD and that appears to have resolved the issue entirely. Thanks for the help.