Slow Dockers because of downloads

[paultn]

Hi Everyone.

 

I am a New Unraid user and am loving it so far besides one large issue I am having.  First off my Hardware.

I am running a Xeon E5-2670v2 with 16 GB of ram.   I have 5 WD Red 3TB drives and 1 WD Red 4TB drive (Parity) in my Array.   Cache disk is a Samsung 850 Evo 250 GB SSD.

My internet connection is 1Gbps down and I have 3 nics teamed in an 802.11ad configuration to a Cisco Managed switch.

 

I have my docker image as well as appdata stored on my cache drive.   Dockers in use are Sonarr, Radarr, Plex, Crashplan and SABnzbd (or NZBGet as i tried both but settled on SAB as it seems to work a little better).

SABnzbd downloads to the cache drive as well then Sonarr or Radarr moves the completed download.  (Have tried moving to share that uses the cache drive and one that rights direct to the array,  results are basically the same).

 

Everything works fantastic when there are no downloads or only 1 file downloading.  However whenever there is a queue where SABnzbd needs to download, unpack, verify or repair at the same time as well as Sonarr or Radarr potentially movie files.  My cache drive slows right down and drags all the other dockers to a crawl.   It renders plex unusable.  Another thing I have noticed is if I am watching SABnzbd in my web browser it says SABnzbd is not responding every once and awhile then reloads it almost like the web browser loses connection.

 

Normally I would say this is just because the disk is being overworked at this time,  however I basically ran this same setup in Windows Server 2012 (SSD was the C: drive and had plex sonnar and Sabnzbd on it) And this never occurred.   Everything always ran fast and responsive.   This seems to be something to do with Unraid (or Linux or Docker) as the hardware I know is good and should be able to handle this.

 

Sorry for the long post but I wanted to be thorough.

 

Any help would be greatly appreciated as I would really like to get this working because everything else is so much better than Windows.  

[unevent]

Depending on the Docker you are using, you can use 'nice' and 'ionice' settings to tame Sab with regards to CPU and disk priority.  Sab wiki link.  If the Docker has nice and ionice then these settings will work.

Edited October 6, 2017 by unevent

[paultn]

I have tried nice and ionice in SAB and it did not really seem to help.

Just confused why this is such an issue now but it wasn't under Windows.

Edited October 6, 2017 by paultn

[unevent]

Telnet/SSH to unRAID and issue the command below replacing 'container_name' with the name of the container as seen on the Docker tab in unRAID GUI:

 

docker exec -ti container_name /bin/bash

Once in the container, issue these commands to verify nice and ionice are available:

nice --version

ionice --version

 

[tjb_altf4]

I have a 2TB WD Blue as a dedicated download/seeding drive using unnassigned devices plugin.
Media copies to array once completed + processed, virtually no impact to dockers, vms or other cache usage.

 

The performance impact (in my opinion) is due to the high IO nature of download/uploads with parity being thrown in over the top.

[unevent]

@tjb_altf4 brings up a point, is your download/data folder cache-only?  Only the final copy of the finished and processed download should be to the array.

 

[paultn]

28 minutes ago, tjb_altf4 said:

I have a 2TB WD Blue as a dedicated download/seeding drive using unnassigned devices plugin.
Media copies to array once completed + processed, virtually no impact to dockers, vms or other cache usage.

 

The performance impact (in my opinion) is due to the high IO nature of download/uploads with parity being thrown in over the top.

 

I don't believe there should be an impact though.   There is no parity on the cache drive so I cannot see why it would bottleneck so much over Windows.  I may have to look at this alternate drive approach.  Although I feel I may still get an issue as I would like my Media share to use the cache drive so I would be coying from an unassigned device to the cache drive and still cause the performance hit.

 

I am going to look at the nice and ionice a bit more as suggested.

[paultn]

3 minutes ago, unevent said:

@tjb_altf4 brings up a point, is your download/data folder cache-only?  Only the final copy of the finished and processed download should be to the array.

 

My download folder is cache only and the appdata folder and docker image is cache preferred.

[paultn]

9 hours ago, tjb_altf4 said:

I have a 2TB WD Blue as a dedicated download/seeding drive using unnassigned devices plugin.
Media copies to array once completed + processed, virtually no impact to dockers, vms or other cache usage.

 

The performance impact (in my opinion) is due to the high IO nature of download/uploads with parity being thrown in over the top.

So I ended up installing an old OCZ Agility 3 120GB SSD as an unassigned device.   I added this as my download and process location for SAB and while the drive is getting hammered it is not affecting my dockers.

 

I will keep watching but I think this may be the solution.   Still puzzling that Windows never had this issue.

 

Thanks everyone for the help.