Hello, 

 

I have an i7-7700 processor and I cannot figure out why transferring 6TB files using Krusader/MC max my CPU to 100%. The disk has just been pre-cleared with no issues so I formatted it to NTFS to begin transfer. But I cannot get it to work because of the constant high usage. Note the issue starts very early and my most recent test using MC I was less than 5 mins of the transfer and my CPU already maxing out. 

 

Any help greatly appreciated

 

 

Post diagnostics. 

23 minutes ago, dlandon said:

Post diagnostics. 

Hi, 

 

Please find attached. I hope this is the file you need

 

tower-diagnostics-20230730-2138.zip

keep in mind, that there is a decent bug in the unraid gui.

For cpu usage it counts in the io waiting time. If you transfer large files it is normal that you outrun the disks and io has to be stopped until the disk is accepting new commands again.

The gui will show this as 100% cpu usage.

To distinguish the reading from a real 100% usage, watch the cpu temp too!. If it stays rather low, its io, if it going up like hell, there is a real usage going on.

(also you can just click some pages on the gui, if they still respond fast, cpu is not at limit yet)

 

5 hours ago, MAM59 said:

keep in mind, that there is a decent bug in the unraid gui.

For cpu usage it counts in the io waiting time. If you transfer large files it is normal that you outrun the disks and io has to be stopped until the disk is accepting new commands again.

The gui will show this as 100% cpu usage.

To distinguish the reading from a real 100% usage, watch the cpu temp too!. If it stays rather low, its io, if it going up like hell, there is a real usage going on.

(also you can just click some pages on the gui, if they still respond fast, cpu is not at limit yet)

 

Thanks for your input. Gui has become very sluggish and only way to get it to work is to somehow be able to pause the transfer by clicking the pause and hope that it will register the command during very slow gui. 
 

any solution to this issue? 

4 hours ago, youngvader said:

any solution to this issue? 

Sorry, lets wait for @dlandon to review your diagnostics.

 

Edited July 31, 20232 yr by MAM59

57 minutes ago, MAM59 said:

Sorry, lets wait for @dlandon to review your diagnostics.

 

Thanks. I'll wait for feedback

This is not necessarily your issue because you said you used MC and not across the network, but be careful with Jumbo frames, especially on a Realtek NIC.  Realtek NICs are historically troublesome on Linix.  Jumbo frames are really not worth the effort.  You have to set up your LAN correctly or you'll have issues and the gains are minimal.

Jul 30 13:22:17 Tower kernel: r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]

 

You appear to be using a port multiplier.

Jul 30 13:22:17 Tower kernel: ata37.15: Port Multiplier 1.2, 0x197b:0x5755 r0, 5 ports, feat 0x5/0xf

I'm not a disk expert, but port multipiers can be a problem.

 

I am also seeing disk/interface issues on the port multiplier.

Jul 30 13:22:17 Tower kernel: ata37.15: Port Multiplier 1.2, 0x197b:0x5755 r0, 5 ports, feat 0x5/0xf
Jul 30 13:22:17 Tower kernel: ata37.00: hard resetting link
Jul 30 13:22:17 Tower kernel: ata24: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata37.00: SATA link down (SStatus 0 SControl 330)
Jul 30 13:22:17 Tower kernel: ata37.01: hard resetting link
Jul 30 13:22:17 Tower kernel: ata25: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata37.01: SATA link down (SStatus 0 SControl 330)
Jul 30 13:22:17 Tower kernel: ata37.02: hard resetting link
Jul 30 13:22:17 Tower kernel: ata26: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata37.02: SATA link down (SStatus 0 SControl 330)
Jul 30 13:22:17 Tower kernel: ata37.03: hard resetting link
Jul 30 13:22:17 Tower kernel: ata27: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata37.03: SATA link down (SStatus 0 SControl 330)
Jul 30 13:22:17 Tower kernel: ata37.04: hard resetting link
Jul 30 13:22:17 Tower kernel: ata28: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata37.04: SATA link down (SStatus 0 SControl 330)
Jul 30 13:22:17 Tower kernel: ata37: EH complete
Jul 30 13:22:17 Tower kernel: ata37: EH complete
Jul 30 13:22:17 Tower kernel: ata29: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata30: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata31: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata32: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata33: SATA link up 6.0 Gbps (SStatus 133 SControl 300)

 

I'd like @JorgeB to have a look.  He is the disk expert.

Were the diags saved during a transfer?

8 hours ago, JorgeB said:

Were the diags saved during a transfer?

sorry not sure how to answer this question. The diagnostics file I already posted in my earlier post. May I know what diags you are referring to?

9 hours ago, dlandon said:

This is not necessarily your issue because you said you used MC and not across the network, but be careful with Jumbo frames, especially on a Realtek NIC.  Realtek NICs are historically troublesome on Linix.  Jumbo frames are really not worth the effort.  You have to set up your LAN correctly or you'll have issues and the gains are minimal.

Jul 30 13:22:17 Tower kernel: r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]

 

You appear to be using a port multiplier.

Jul 30 13:22:17 Tower kernel: ata37.15: Port Multiplier 1.2, 0x197b:0x5755 r0, 5 ports, feat 0x5/0xf

I'm not a disk expert, but port multipiers can be a problem.

 

I am also seeing disk/interface issues on the port multiplier.

Jul 30 13:22:17 Tower kernel: ata37.15: Port Multiplier 1.2, 0x197b:0x5755 r0, 5 ports, feat 0x5/0xf
Jul 30 13:22:17 Tower kernel: ata37.00: hard resetting link
Jul 30 13:22:17 Tower kernel: ata24: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata37.00: SATA link down (SStatus 0 SControl 330)
Jul 30 13:22:17 Tower kernel: ata37.01: hard resetting link
Jul 30 13:22:17 Tower kernel: ata25: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata37.01: SATA link down (SStatus 0 SControl 330)
Jul 30 13:22:17 Tower kernel: ata37.02: hard resetting link
Jul 30 13:22:17 Tower kernel: ata26: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata37.02: SATA link down (SStatus 0 SControl 330)
Jul 30 13:22:17 Tower kernel: ata37.03: hard resetting link
Jul 30 13:22:17 Tower kernel: ata27: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata37.03: SATA link down (SStatus 0 SControl 330)
Jul 30 13:22:17 Tower kernel: ata37.04: hard resetting link
Jul 30 13:22:17 Tower kernel: ata28: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata37.04: SATA link down (SStatus 0 SControl 330)
Jul 30 13:22:17 Tower kernel: ata37: EH complete
Jul 30 13:22:17 Tower kernel: ata37: EH complete
Jul 30 13:22:17 Tower kernel: ata29: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata30: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata31: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata32: SATA link down (SStatus 0 SControl 300)
Jul 30 13:22:17 Tower kernel: ata33: SATA link up 6.0 Gbps (SStatus 133 SControl 300)

 

I'd like @JorgeB to have a look.  He is the disk expert.

Thanks for your reply. I'm very new to Unraid so apologies but not sure what Jumbo frames and port multiplier you are referring to. I will read up on google. thanks

"Jumbo Frames" are very large Ethernet packets. They were invented, when 1Gb/s Ethernet came up and the CPUs of those times had to do a hard job to build and check the headers of in- and outgoing frames.

 

The idea was: "larger frames -> less packets -> fewer calculations needed -> less CPU time spent"

 

It worked, but many older implementations had problems with these frames (you have to set it EVERYWHERE and there are many devices which cannot be controlled, so usually they wont work in your environment).

The more important thing why you can drop them today is that LAN adapters became more and more clever. Today even the cheapest chip allows the CPU to "offload" these tasks freeing the CPU from all calculations. Even error handling is done automatically now. Some very advanced cards even contain the whole TCP/IP protocol and handling everything.

 

So today, there is no need for Jumbo Frames anymore. They are still available for backwards compatibility, but in normal installation they should not be used anymore.

 

 

A "port multiplier" is an addon chip to your SATA controller. These controllers often have only 2 or 4 "real" ports. These are internally connected to the inputs of the multiplier chip. The multiplier has usually 8 or even more output SATA ports. The difference (and the reason why you should not use one if possible) is, that it takes more steps to transfer data. First you need to program the multiplier to use the correct output, then you program the SATA controller to transfer data. This switching takes quite a long time, maximum transfer speed is seriously lowered (by half or even more). The worst case would be that you want to transfer data from disks connected to the same port of the real controller, this would need switching for every packet (and buffering in RAM too).

The bad thing is that you usually cannot know before you buy such a card. There are threads here that give advices which chipsets are "good" and which are "bad"

 

3 hours ago, youngvader said:

Were the diags saved during a transfer?

Jorge wanted to know if the diagnostics were taken during an ongoing disk transfer (with 100% cpu usage). If not, start a transfer, wait for the 100% and take (and upload) new diagnostics...

7 hours ago, MAM59 said:

"Jumbo Frames" are very large Ethernet packets. They were invented, when 1Gb/s Ethernet came up and the CPUs of those times had to do a hard job to build and check the headers of in- and outgoing frames.

 

The idea was: "larger frames -> less packets -> fewer calculations needed -> less CPU time spent"

 

It worked, but many older implementations had problems with these frames (you have to set it EVERYWHERE and there are many devices which cannot be controlled, so usually they wont work in your environment).

The more important thing why you can drop them today is that LAN adapters became more and more clever. Today even the cheapest chip allows the CPU to "offload" these tasks freeing the CPU from all calculations. Even error handling is done automatically now. Some very advanced cards even contain the whole TCP/IP protocol and handling everything.

 

So today, there is no need for Jumbo Frames anymore. They are still available for backwards compatibility, but in normal installation they should not be used anymore.

 

 

A "port multiplier" is an addon chip to your SATA controller. These controllers often have only 2 or 4 "real" ports. These are internally connected to the inputs of the multiplier chip. The multiplier has usually 8 or even more output SATA ports. The difference (and the reason why you should not use one if possible) is, that it takes more steps to transfer data. First you need to program the multiplier to use the correct output, then you program the SATA controller to transfer data. This switching takes quite a long time, maximum transfer speed is seriously lowered (by half or even more). The worst case would be that you want to transfer data from disks connected to the same port of the real controller, this would need switching for every packet (and buffering in RAM too).

The bad thing is that you usually cannot know before you buy such a card. There are threads here that give advices which chipsets are "good" and which are "bad"

 

Thank you for the lengthy explanation. Much appreciated