Help, Looking on for storage share/guidence for Sabnzbd/NZBGet (Cache) -> Main Array

[randomninjaatk]

So I'm having some difficulty with share configuration for use with NZBGet or Sabnzbd. The goal is to have NZBGet/Sabnzbd to download using a "Download" share that has access to the cache, and then have Sonarr, Radarr, or other programs pickup the finished files and move them to the "Media" share on the Protected Array for long term storage. The purpose of this is to speed up the download/unpacking process because they perform better on the cache. Otherwise I can run into post-processing backups because the files are coming in faster than they can be post-processed.

 

Background, Server setup:

 

CPU: Ryzen 1700x - 8 Core, 16 Thread

1 x 500 GB NVMe SSD - Cache Drive

1 x 12 TB HDD - Parity Drive

1TB, 1TB, 8TB - Array drives  

 

In a typical situation, if the cache was large enough, I could simply let the mover script handle this function by allowing the Media share to use the cache. The problem is I can fill the cache drive in only a few hours and the docker apps that are running will start to crash because there is no space available. The problem I'm running into is that the tools are picking up the files, but when they move them, they are moving them to "Media" share but staying on the cache share instead of writing to the Protected Array. So it's not clearing up the space and causing the cache to fill and then problems start to occur. The "Media" share is configured to not use the cache drive at all and all the apps that are performing the functions are running via docker.

 

Does anyone have any advice or guidance?

 

I was possibly thinking I could replace the cache with a 6TB 7200 RPM standard HDD and let the mover script solve it, but I prefer using SSD/NVMe for VMs, Plex and others for the cache, because it performs better.... I can easily fill more then 1TB in a single day.... #DataHoardingProblems

 

Thanks for any guidance!

 

 

[Squid]

It sounds like you've got /mnt/user mapped to /mnt/user (or /mnt mapped to /mnt) passed through to the containers and are referencing the applicable shares from that mapping, as in that case the system will simply rename the completed downloads (and thereby bypassing the use cache settings) into the media share instead of moving them (which will honor the settings)

 

Post up screen shots of the templates, and the applicable settings from the apps.

[randomninjaatk]

8 minutes ago, Squid said:

It sounds like you've got /mnt/user mapped to /mnt/user (or /mnt mapped to /mnt) passed through to the containers and are referencing the applicable shares from that mapping, as in that case the system will simply rename the completed downloads (and thereby bypassing the use cache settings) into the media share instead of moving them (which will honor the settings)

 

Post up screen shots of the templates, and the applicable settings from the apps.

 

That's exactly how I have it configured, which I figured is the problem... So how do I configure the docker containers to use them as shares instead? 

[Squid]

First off, set up the downloads on both containers to be like this:  https://forums.lime-technology.com/topic/57181-real-docker-faq/?page=2#comment-566086

Then add a mapping of say /movies mapped to /mnt/user/movies and tell radarr to move the completed downloads to /movies

 

[Greygoose]

I was having  the same problem.. 

 

I was setting download schedule in Sabnzb then during off period scheduling mover to activate. Setup like this it would still keep me working. 

 

Squid, thanks for the link. Reading and learning a lot from that thread. 

[randomninjaatk]

1 hour ago, Squid said:

First off, set up the downloads on both containers to be like this:  https://forums.lime-technology.com/topic/57181-real-docker-faq/?page=2#comment-566086

Then add a mapping of say /movies mapped to /mnt/user/movies and tell radarr to move the completed downloads to /movies

 

 

That solved it for me! You rock! I tried searching the forums first but couldn't find it... I really appreciate the help!

[Greygoose]

Sorry to jump thread 

 

Ive attached Sabnzb and sonarr

 

I created a share on cache called downloads. 

 

 

[randomninjaatk]

You need to map directly to the cache, So your for your /data path, change it to:

 

/mnt/cache/Downloads

 

In the share settings, make sure you Downloads share is set to "Cache" only and on your Media share, make sure you have it set to "No" for using the cache.

 

Thats what solved it for me, I was previously trying to accomplish it with the same configuration you have. You also need to make sure you have no "Media" share data on the cache, otherwise that will prevent it from working. So you may need to run the mover first, because if any "Media" share data is on the cache, it will prevent it from moving it to the array...

[Greygoose]

[randomninjaatk]

That looks fine, did you make the change I told you about in my last post?

[Greygoose]

8 minutes ago, randomninjaatk said:

You need to map directly to the cache, So your for your /data path, change it to:

 

/mnt/cache/Downloads

 

In the share settings, make sure you Downloads share is set to "Cache" only and on your Media share, make sure you have it set to "No" for using the cache.

 

Thats what solved it for me, I was previously trying to accomplish it with the same configuration you have. You also need to make sure you have no "Media" share data on the cache, otherwise that will prevent it from working. So you may need to run the mover first, because if any "Media" share data is on the cache, it will prevent it from moving it to the array...

Oh I see. 

 

I understand it now. 

 

Thank you 

[Greygoose]

1 minute ago, randomninjaatk said:

That looks fine, did you make the change I told you about in my last post?

I will change later or tomorrow and let you know. 

 

Thanks

[Greygoose]

Ok i have done the changes.

 

The main issue i had was setting the downloads to cache only, this was giving me errors in common problems plugin saying that files were on cache and parity.

 

I manually deleted each file but the error existed, so i deleted the entire download share and created a new one. then it worked fine

[Greygoose]

It worked

 

thanks