Jump to content

Server Load Spikes Whenever Writing Large Files. System Becomes Unresponsive.

AnnHashaway

By AnnHashaway
in General Support

Recommended Posts

AnnHashaway Noob

AnnHashaway

Posted
Posted

Since the beginning of running Unraid (about a year) I have had extremely bad performance any time I am writing more than a few gigs to the system. This can be transferring something from an external device, copying from one folder to another on the system through Krusader, and any app that needs to do a lot of reading and/or writing.

 

For example, here is a quick video of NZBGet doing some work. Everything is fine, until the behind-the-scenes transfer starts to take place. You can see the load on the server begin to spike around 1 minute. The load can often peak around 30-40, at which point everything is completely unresponsive.

 

https://streamable.com/a9uyc

 

When this happens, everything grinds to a halt. 

 

Other cases where this happens:

 

  • Using Krusader to copy large files (or many files) from one location to another
  • Transferring files to the system from a separate device

 

System:

 

  • Parity Drive
  • 3 Data Drives
  • 1 Cache Drive

 

The cache can have 200GBs on it, and it still happens. It tends to occur around the 2-5GB transfer point. The first 1-2GBs go very fast, then it practically stops. A 40GB copy from one directory to another can take 45 minutes, even with plenty of room on the cache.

 

Here are my settings:

 

Docker

  • Location: /mnt/cache/docker.img
  • Default Appdata: /mnt/user/appdata

 

Shares:

 

appdata

  • Use Cache: Prefer
  • Excluded disks: none
  • Included disks: all

 

system

  • Use Cache: Prefer
  • Excluded disks: none
  • Included disks: all

 

I was running this all on a Xeon system, and it was the same thing. Now its on an AMD CPU with a new Mobo, and the results are identical.

 

Diagnostics Attached

 

 

diagnostics-20170829-1426.zip

Link to comment
Frank1940 Veteran

Frank1940

Posted
Posted

What do you have connected to your usb ports?  You have thousands of entries in your syslog like this: 

Aug 28 18:55:02 Denali kernel: usb 5-5: new low-speed USB device number 114 using ohci-pci
Aug 28 18:55:02 Denali kernel: usb 5-5: device descriptor read/64, error -32
Aug 28 18:55:02 Denali kernel: usb 5-5: device descriptor read/64, error -32
Aug 28 18:55:02 Denali kernel: usb 5-5: new low-speed USB device number 115 using ohci-pci
Aug 28 18:55:02 Denali kernel: usb 5-5: device descriptor read/64, error -32
Aug 28 18:55:03 Denali kernel: usb 5-5: device descriptor read/64, error -32
Aug 28 18:55:03 Denali kernel: usb 5-5: new low-speed USB device number 116 using ohci-pci
Aug 28 18:55:03 Denali kernel: usb 5-5: device not accepting address 116, error -32
Aug 28 18:55:03 Denali kernel: usb 5-5: new low-speed USB device number 117 using ohci-pci
Aug 28 18:55:04 Denali kernel: usb 5-5: device not accepting address 117, error -32
Aug 28 18:55:04 Denali kernel: usb usb5-port5: unable to enumerate USB device
Aug 28 18:55:04 Denali kernel: usb 5-4: new low-speed USB device number 118 using ohci-pci
Aug 28 18:55:04 Denali kernel: usb 5-4: device descriptor read/64, error -32
Aug 28 18:55:05 Denali kernel: usb 5-4: device descriptor read/64, error -32
Aug 28 18:55:05 Denali kernel: usb 5-4: new low-speed USB device number 119 using ohci-pci
Aug 28 18:55:05 Denali kernel: usb 5-4: device descriptor read/64, error -32
Aug 28 18:55:05 Denali kernel: usb 5-4: device descriptor read/64, error -32
Aug 28 18:55:05 Denali kernel: usb 5-4: new low-speed USB device number 120 using ohci-pci
Aug 28 18:55:06 Denali kernel: usb 5-4: device not accepting address 120, error -32
Aug 28 18:55:06 Denali kernel: usb 5-4: new low-speed USB device number 121 using ohci-pci
Aug 28 18:55:06 Denali kernel: usb 5-4: device not accepting address 121, error -32
Aug 28 18:55:06 Denali kernel: usb usb5-port4: unable to enumerate USB device

 

Link to comment
Frank1940 Veteran

Frank1940

Posted
Posted

Try googling     new low-speed USB device number 114 using ohci-pci     and you will see a number of hits.  See if you can resolve this issue as it puts a lot of noise into your syslog which makes finding any other issues very difficult.  (Once you solve this issue, you should reboot so you can get a clean set of diagnostics.  Before you capture those diagnostics be sure that you run a transfer of data of the type that has the problem and provide the time when you did it.)

Link to comment
AnnHashaway Noob

AnnHashaway

Posted
  • Author
Posted
13 hours ago, Frank1940 said:

Try googling     new low-speed USB device number 114 using ohci-pci     and you will see a number of hits.  See if you can resolve this issue as it puts a lot of noise into your syslog which makes finding any other issues very difficult.  (Once you solve this issue, you should reboot so you can get a clean set of diagnostics.  Before you capture those diagnostics be sure that you run a transfer of data of the type that has the problem and provide the time when you did it.)

 

Thanks. I will look into this and update.

 

In the meantime, what do you think of this? I just tried a 10GB transfer to a share from a computer on the local network. As always, the load went up to about 45 before I had to cancel the transfer. On a whim, I switched the share's cache settings from Yes to No and re-initiated the transfer from scratch. The entire 10GBs transferred at max speed, and the server load didn't spike at all. 

 

So obviously its cache related. When writing to the cache drive, the load spikes to out of control levels. When bypassing the cache, everything works normally. 

Link to comment
AnnHashaway Noob

AnnHashaway

Posted
  • Author
Posted
15 minutes ago, remotevisitor said:

There was a similar case reported a few weeks ago.

 

in that case the cache drive was btrfs and switching it to XFS solved the problem.

 

i assume your cache drive is btrfs as it is the default for the cache drive,.  You could try converting it to XFS and see if the problem goes away.

 

btrfs - Yes, that is the current format. I will give that a shot over the weekend. Thanks for the tip.

 

12 minutes ago, Frank1940 said:

Have you run the trim operation on these two ssd's?  I took a look at the SMART reports and one of your disk has this parameter 


202 Percent_Lifetime_Used   0x0031   251   251   000    Pre-fail  Offline      -       105

I am not an expert on ssd's and I hope someone who is will jump in and give their opinion. 

 

I will give that a look as well this weekend. Thank you.

Link to comment

Archived

This topic is now archived and is closed to further replies.

Go to topic listing
×
×
  • Create New...