unRAID with SABnzbd

[lionelhutz]

Did you update SickBeard at that time? Did you try to stop SickBeard and only run Sab?

 

But then, I was downloading last night after updating SickBeard and it was working OK. I get a little slowdown every now and then but nothing serious.

 

Peter

[kaiguy]

MrAdams, do you have unmenu installed?  If so, can you go over to myMain and see how many writes have been made to your flash drive?  I'm wondering if perhaps Sab is using your flash drive for its cache, which could seriously slow down your transfers.  As a frame of reference, my server has been up and running for 25 days, and I have about 250 writes to my flash drive.

 

Also, I'd suggest checking your Article Cache Limit under the General Settings of Sab.  I have mine set for 250M, but I'm sure you can get away with a smaller number (say, 100-150M). 

 

Aside from that, I'm at a bit of a loss.  I use Astraweb as well and lately my transfers have been rock solid.  Since you say you don't have the speed problem on your laptop, I'd agree that it is looking more and more like something with the unRAID box.  What else are you running on it?  Maybe you're running out of memory?

[MrAdams]

I am now pretty sure that the problem is somehow caused by Sickbeard. Just shutting down Sickbeard through the web interface makes no difference. But then I tried editing my go file so that Sickbeard wouldn't be loaded at startup, and rebooted the server. That solved the problem and my downloads are back to a smooth and consistent 2.7 MB/s. Now that I think about it, I believe that I updated Sickbeard (through the web interface) just before these problems started. Also around that same time, I had noticed that Sickbeard's Logs and cache folders were in the packages folder on my flash drive for some reason, so I manually editing the Sickbeard config.ini and deleted those folders. Maybe one of these things caused Sickbeard to go haywire?

 

Kaiguy, myMain showed between 900 and 1000 writes to my flash drive in about a 24 hour period. That's obviously really high for some reason. I'm guessing this is related to Sickbeard, too, although this was even after I moved the Logs and cache folders to an array drive.

 

At this point, I should probably move my issue to the Sickbeard thread since it looks like sabnzbd isn't the culprit. Thanks for all the help so far.

[Stokkes]

Anyway we could get the dependency package changed so that files are readable and executable by ALL instead of just "root"?

 

This really messes with SABnzbd on the 5.0 series because python can't be run by anyone other than root. The fallout from this is that permissions get messed on downloaded files (they're owned by root instead of "nobody" which is the convention in the 5.x series).

 

Ideally, one would want to run SAB as the user "nobody" (sudo -u nobody /path/to/SABnzbd.py) which will maintain the file permission consistency.

 

 

[lionelhutz]

Probably about 4 to 6 or 8 pages back someone posted the commands to untar and re-tar the package. Look for my posts about the par2 tests because it was right after that. Just untar it into a temporary directory, chmod the contents of the directory and then tar it again.

 

You could also try PM'ing Romir and see if he could change the package or make a 5.x series package.

 

Peter

 

[sacretagent]

Anyway we could get the dependency package changed so that files are readable and executable by ALL instead of just "root"?

 

This really messes with SABnzbd on the 5.0 series because python can't be run by anyone other than root. The fallout from this is that permissions get messed on downloaded files (they're owned by root instead of "nobody" which is the convention in the 5.x series).

 

Ideally, one would want to run SAB as the user "nobody" (sudo -u nobody /path/to/SABnzbd.py) which will maintain the file permission consistency.

 

 

 

Stokkes

 

how you want to run sudo ?

it is not part of unraid 5...

any other way to adapt the go script so these packages can run as nobody ?

i checked upon BRiT's scripts but that is above my paygrade and he is also running a full slack os with unraid ....

 

maybe unmenu can package the sudo proggie ?

 

 

[sacretagent]

hi

 

I think i figured it out how to run the packages as nobody

 

but BE AWARE you need to change the dependencies pkg

 

mkdir /tmp/depend
cp /boot/custom/packages/SABnzbdDependencies-2.1-i486-unRAID.tgz /tmp/depend
tar -zxvf SABnzbdDependencies-2.1-i486-unRAID.tgz
chown -R nobody install
chown -R nobody usr
tar -zcvf /boot/custom/packages/SABnzbdDependencies-2.2-i486-unRAID.tgz usr install

 

you can delete depend if you want but it will delete automatically next reboot

if want now

rm -r /tmp/depend

 

the go file should look something like this

 

#sleep for 30 seconds
sleep 30

#Sab/sickbeard/couchpotato dependencies
installpkg /boot/custom/packages/SABnzbdDependencies-2.2-i486-unRAID.tgz

#give nobody shell access
usermod -s /bin/bash nobody

#move to nobody
su nobody

#Sabnzbd startup
python /boot/custom/sabnzbd/SABnzbd-0.5.6/SABnzbd.py -d -f /boot/config/SABnzbd.ini -p -s 192.168.1.130:9080 
#Sickbeard startup
python /mnt/cache/.custom/sickbeard/SickBeard.py -d
#couchpotato startup
python /mnt/cache/.custom/couchpotato/couchpotato/CouchPotato.py -d

 

a few remarks though

1. put it at the end of your go script as the shell willl move to nobody

2. nobody needs to be able to access all files in the .custom dirs so better to chown -R nobody /mnt/cache/.custom

3. i am a linux beginner and have no clue how safe this is ...  i tried to access the unraid by using nobody and no passwd and that didn't work so i guess it is pretty safe but don't take my word on it

4. if you need to run other commands after the packages then exit should bring the shell back to root ... but not sure about this ...

5. i put

#Sab/sickbeard/couchpotato dependencies
installpkg /boot/custom/packages/SABnzbdDependencies-2.2-i486-unRAID.tgz

before i go to nobody as installpkg was not a known command when running from nobody

guess it is easily fixed by changing unmenu to run from nobody ... but i didn't want to mess with that

 

It seems to be working as top

 

top - 15:46:54 up 31 min,  2 users,  load average: 0.07, 0.04, 0.00
Tasks:  99 total,   1 running,  98 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.5%us,  0.3%sy,  0.0%ni, 99.0%id,  0.2%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   3049788k total,   403296k used,  2646492k free,     3012k buffers
Swap:        0k total,        0k used,        0k free,   275600k cached

 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
3503 nobody    20   0  294m  27m 3388 S    2  0.9   0:01.33 python
   1 root      20   0   704  332  288 S    0  0.0   0:01.85 init
   2 root      20   0     0    0    0 S    0  0.0   0:00.00 kthreadd
   3 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/0
   4 root      20   0     0    0    0 S    0  0.0   0:00.00 ksoftirqd/0
   5 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/1

shows clearly python is running under nobody

i just downloaded a small file and it is owned by nobody

and i had all rights from windows to open it  

 

will next try to get sickbeard to run... but guess that will be no problem

 

edited for typo's

 

second edit

put sickbead on and put couchpotato on

both run as nobody and sickbeard put nfo's as nobody

now only need to do a reboot to see if everything still works after reboot

 

third edit

 

made a mistake 

it works all from the terminal

but not from GO yet

go runs not in multiuser level

so su nobody he doesn't allow

 

thinking now to put a small script in rc.d or something that executes the last few lines

#move to nobody
su nobody

#Sabnzbd startup
python /boot/custom/sabnzbd/SABnzbd-0.5.6/SABnzbd.py -d -f /boot/config/SABnzbd.ini -p -s 192.168.1.130:9080 
#Sickbeard startup
python /mnt/cache/.custom/sickbeard/SickBeard.py -d
#couchpotato startup
python /mnt/cache/.custom/couchpotato/couchpotato/CouchPotato.py -d

will look at it more tomorrow.. midnight past here and need to work tomorrow....

want this to run

[Bolle]

... bla bla how to get SABnzbd working...

 

Thanks to all, with these instructions and downloads I got it working. Works great using a cache disk (just got an extra 2 Tb Seagate disk in and upgraded to a Plus license)

[sacretagent]

Still looking into this on my free time..

 

Brit

 

i had a look at your scripts and i could integrate them for other people to use but your start-stop daemon is missing in Unraid

 

any chance you can point me to the package ?

 

[stradle]

Love the way this works with sickbeard got all my downloads going to my 'cache' drive.

 

Has anyone managed to create processing scripts for other category types such as music?

Would love a scheduled script that would order music and move from cache - music folder as Artist\Album (year)

Was also looking to schedule the Sickbeard script to only move TV eps weekly or something.

 

If anyone knows of anything like this, let me know.

 

 

 

[speedkills]

I believe you could tweak the mover script if you wanted to move files less often.  Might be a more straightforward method than modifying the sickbeard script.  I hadn't given it much thought but now that you mention it 80% of the tv shows I download get watched and deleted within a couple of days, if the cache script could be modified to only move files at least (n) days old it would save spinning up my array every time I watched a show.

[speedkills]

Just an update for all of you running sabnzbd with sickbeard, sickbeard has recently started writing it's cache and logs to the directory you are in when you launch sickbeard.py, not the directory containing sickbeard.py.  Since my go script has a "cd /boot/packages" in it that means my sickbeard cache and logs were ending up on my flash drive in the packages directory.  My fix was to just add an additional line to my go script right before launching sickbeard to change my working directory to where I am storing sickbeard.  In my case it looks like:

 

#change working directory

cd /mnt/cache/.custom/sickbeard

#launch sickbeard

nohup python /mnt/cache/.custom/sickbeard/SickBeard.py &

 

I suppose I could have shorted that to nohup python SickBeard.py & at this point but it works this way and is no longer writing logs and cache to my flash drive.

 

 

[Tybio]

Just an update for all of you running sabnzbd with sickbeard, sickbeard has recently started writing it's cache and logs to the directory you are in when you launch sickbeard.py, not the directory containing sickbeard.py.  Since my go script has a "cd /boot/packages" in it that means my sickbeard cache and logs were ending up on my flash drive in the packages directory.  My fix was to just add an additional line to my go script right before launching sickbeard to change my working directory to where I am storing sickbeard.  In my case it looks like:

 

#change working directory

cd /mnt/cache/.custom/sickbeard

#launch sickbeard

nohup python /mnt/cache/.custom/sickbeard/SickBeard.py &

 

I suppose I could have shorted that to nohup python SickBeard.py & at this point but it works this way and is no longer writing logs and cache to my flash drive.

 

You can also edit /mnt/cache/.custom/sickbeard/config.ini and define the cache and log directory so they are always absolute.

 

Also, start SickBeard like this now:

 

/mnt/cache/.custom/sickbeard/SickBeard.py --daemon

 

And you don't have to mess with nohup or forking into the background (&) at all.

[speedkills]

Thanks for the tip.  I actually did modify config.ini to use an absolute log directory but couldn't find a place to use an absolute cache directory.  Never did know what the nohup was for and haven't found time to read up on it yet but for some reason it makes me feel better to start it the same way I start sab and couchpotato

 

<Update> Found the cache setting in the config file, it just wasn't in the gui config editor.  Thanks again.

[Tybio]

I prefer to start them all as Daemons so they function to POSIX standards:

 

/mnt/cache/.custom/SABnzbd/SABnzbd.py -d

/mnt/cache/.custom/sickbeard/SickBeard.py --daemon

/mnt/cache/.custom/couchpotato/CouchPotato.py -d

 

Keeps it all nice and clean, and means that they can operate without the baggage of a shell or whatnot .

[Orbi]

I prefer to start them all as Daemons so they function to POSIX standards:

 

/mnt/cache/.custom/SABnzbd/SABnzbd.py -d

/mnt/cache/.custom/sickbeard/SickBeard.py --daemon

/mnt/cache/.custom/couchpotato/CouchPotato.py -d

 

Keeps it all nice and clean, and means that they can operate without the baggage of a shell or whatnot .

 

Yeah that's indeed the preferrable way but, for some unknown reason, SB and CP do not always work for everybody this way.

[lionelhutz]

Just an update for all of you running sabnzbd with sickbeard, sickbeard has recently started writing it's cache and logs to the directory you are in when you launch sickbeard.py, not the directory containing sickbeard.py. 

 

Yeah, it's nothing new and I reported it pages ago. Just edit the config file and put the absolute paths in there. You can actually set the logs path from the interface.

 

Peter

[speedkills]

My apologies, I missed that while reading the 35+ pages of this thread.  I try my best not to double post but you can't win 'em all.

[LVLAaron]

Is there a Wiki entry for setting this up? I have had it up and running for a long time, but it'd be great for whoever it is that packages the dependencies (thanks, btw) could update the link in the wiki so I'm not digging around in a thread to find it....

[lionelhutz]

My apologies, I missed that while reading the 35+ pages of this thread.  I try my best not to double post but you can't win 'em all.

 

It's cool. Another reminder isn't a bad thing. Using the USB stick sure doesn't contribute to it's longevity.

 

Peter

 

[Tybio]

Is there a Wiki entry for setting this up? I have had it up and running for a long time, but it'd be great for whoever it is that packages the dependencies (thanks, btw) could update the link in the wiki so I'm not digging around in a thread to find it....

 

I just tossed one together:  http://lime-technology.com/wiki/index.php?title=Install_Python_based_servers

 

Obviously it needs a lot of work, so get out there and contribute if you can!  I haven't even started on the configuration section, but at least the core of the install is recorded .

[speedkills]

I updated the wiki entry to use a better method of detecting if the cache drive is online.  I believe Joe contributed this code.  It works well because it only starts the apps up if the array actually comes online which is nice for those times when you are modifying something and it doesn't start up automatically, plus it gives you a minute and a half to get things sorted if you need to manually start the array.

[Carpet3]

Sorry if i've missed this in the thread (I imagine I have). I've seen people say to use a cache drive because CoachPotato will stop the drive spinning down, is it the same with just sab and sickbeard?

 

 

[kaiguy]

On my quest to get my cache drive to sleep, I made a few changes to the stock config to Sab and Sickbeard.  For Sab, I changed the article cache limit to 250M (but I think it's unnecessarily high).  Even though I don't use the watched folder option, I changed the interval to 3600 seconds.  For SB, I changed the search interval to 4 hours.  Logs are sent to the volatile file system for both programs. 

 

With this setup, my cache drive spins down as normal.  Personally, I always had my cache drive running when I used CouchPotato, so I moved that app over to my HTPC.  It does appear, however, that even just accessing the web GUI for either one can cause the cache drive to spin up.

[Tybio]

Fantastic updates to the wiki page for those who've been adding.  I think we are almost ready to start sending people that way for initial questions.

 

I've added a header for config options to let the cache drive sleep...I've not done it yet, so hopefully someone will update it .