Jump to content
overbyrn

NZBGet Plugin for UnRaid v5b11+

312 posts in this topic Last Reply

Recommended Posts

Unfortunately I'm out of the plugin game.  Please continue to check the work that Phaze is doing, with luck he'll have an nzbget plugin in the future.   

 

 

Update: 22nd March 2013

 

New plugin and new NZBGet build available.  Big update; plugin development moved to GitHub and now allows for plugin to be updated within itself as well as NZBGet.  This should hopefully allow me to both maintain the plugin and provide fresh builds of NZBGet from SVN source.

 

Thanks to HugBug - the developer of NZBGet - for his help in solving some of the build issues I was having and thanks also to DikkieDirk for testing said builds.  From what I can tell, there are no issues with the plugin or NZBGet at this time. 

 

If you've been running an earlier version, I strongly suggest upgrading when you get a chance.  There have been a ton of new features added to NZBGet over the past few months. 

 

To install:

1. Download the plugin (see below for link).  Subsequent plug-in and NZBGet updates are done from within the plug-in itself.

2. Copy plugin to /boot/config/plugins on your flash drive.

3. Reboot unRAID server or install from command line

4. Go to plugin WEGUI and change initial settings

 

 

Relevant links:

Direct link to the plugin: https://github.com/overbyrn/UnRAID/raw/master/nzbget_overbyrn.plg

 

Full installation instructions on my main GitHub page https://github.com/overbyrn/UnRAID

 

Problems or questions with NZBGet should ideally be directed to the NZBGet forum.  It's great that hugbug (aka nzbget) has taken the time to come here and help, but in fairness to him, I doubt he wants to support his application in two places.

 

Regards,

overbyrn

Share this post


Link to post

Nice!

 

In case it's of use, here are the current lines from my script -- I did look at making a plugin a few weeks back, but could not work out what the hell was going on in the SABnzbd plugin code...  ???

 

I was too lazy to include all the dependencies that were already installed on my machine for SABnzbd, so didn't include unrar etc.

 

Note that there were some specific issues with certain library versions and compatibility with v9.x but you may have already seen that in the nzbget forum.

 

#################################################################
# install packages for nzbget                                   #
#################################################################
cd /boot/packages
installpkg kernel-headers-2.6.33.4_smp-x86-1.txz
installpkg pkg-config-0.23-i486-2.txz
#1 installpkg libxml2-2.7.6-i486-1.txz
#2 installpkg libxml2-2.6.31-i486-2.tgz
installpkg libxml2-2.7.8-i486-3.txz
installpkg make-3.81-i486-1.txz
installpkg libsigc++-2.2.8-i486-2sl.txz
installpkg binutils-2.20.51.0.8-i486-1.txz
installpkg glibc-2.11.1-i486-3.txz
installpkg perl-5.10.1-i486-1.txz
installpkg gcc-4.2.4-i486-1.tgz
installpkg gcc-g++-4.2.4-i486-1.tgz
installpkg ncurses-5.7-i486-1.txz
installpkg bind-9.4.3_P4-i486-1.txz
installpkg cxxlibs-6.0.13-i486-2.txz
installpkg patch-2.5.4-i486-1.txz
installpkg zlib-1.2.5-i486-4.txz
#################################################################



#################################################################
# Compile par2lib                                               #
#################################################################
cd /tmp
cp /boot/packages/nzbget-9.0-testing-r477.tar.gz .
tar -zxf nzbget-9.0-testing-r477.tar.gz
cp /boot/packages/libpar2-0.2.tar.gz .
tar -zxf  libpar2-0.2.tar.gz
cd libpar2-0.2
cp /tmp/nzbget-9.0-testing/libpar2-0.2-*.patch .
patch < libpar2-0.2-bugfixes.patch
patch < libpar2-0.2-cancel.patch
./configure --prefix=/usr
make
make install
#################################################################



#################################################################
# Compile nzbget                                                #
#################################################################
cd /tmp/nzbget-9.0-testing
./configure --with-tlslib=OpenSSL
make
make install
# /tmp/nzbget-9.0-testing/nzbget -c /mnt/cache/unraid_apps/nzbget/nzbget.conf -D
#################################################################

Share this post


Link to post

Hi Neil, thanks again for your initial write up.  Was very handy.  Most of the packages you've listed are only needed to compile nzbget and libpar and can be removed once the binary has been built.

 

libsig, libxml, nurses, openssl, unrar and zlib seem to be the only dependencies needed to run it thereafter. 

 

I've not found any library compatibilities yet, although I initially built it on a Slackware 13.37 system which gave me problems as it was using a point revision higher of libstdc++, so I had to start fresh on a 13.1 system.  I'm no expert compiling stuff but one of the things I've learnt through doing this is how to build packages where the resultant application is in the directories I want it in.  Originally nzbget spreads itself over /usr/local/bin, usr/local/share, where I wanted it all in one /usr/local/nzbget location which was quite a challenge.  My eventual configure line was...

 

./configure --prefix=/usr/local --bindir=/usr/local/nzbget --sbindir=/usr/local/nzbget --sharedstatedir=/usr/local/nzbget --localstatedir=/usr/local/nzbget --docdir=/usr/local/share/doc --datarootdir=/usr/local/nzbget/webui --datadir=/usr/local --with-tlslib=OpenSSL

 

Quite a line!

 

 

 

Share this post


Link to post

Well done in getting it working. I really don't think I'd have been able to make a plugin, it's all Greek to me.

 

Hopefully a few people will try this out and can share  their experiences compared to SABnzbd.

Share this post


Link to post

Neil, thanks for the heads up.

 

I've built the new version r497 and updated the plugin.  Please see original post for new version 0.2 of plugin.  I've not done any testing other than to install the plugin onto a virgin dev system sufficiently to see that NZBGet installed and the web page was accessible.

 

 

Regards,

overbyrn

Share this post


Link to post

Neil, thanks for the heads up.

 

I've built the new version r497 and updated the plugin.  Please see original post for new version 0.2 of plugin.  I've not done any testing other than to install the plugin onto a virgin dev system sufficiently to see that NZBGet installed and the web page was accessible.

 

 

Regards,

overbyrn

 

Nice, thanks. Up until now, I had been using my own build, but compiling r497 came up with missing stuff that I could not be arsed to work out, so I just installed your version.

 

I installed it "over the top" of my existing install directory. It did wipe out my original nzbget.conf though!

 

Fortunately, I had a backup, so I copied that back over and it all seems to be working well so far.

 

Cheers,

 

Neil.

Share this post


Link to post

Neil, sorry the plugin nuked your config. Glad you had a backup though. Afraid I didn't do much to consider folks using nzbget prior to the plugin. 

 

Not sure why you had compilation issues.  I'm using a Slackware 13.1 VM.  Although I'm not an active user of nzbget, feel free to let me know when a new version is out as it doesn't take much to create a new build.

 

 

Rgds,

overbyrn

Share this post


Link to post

Neil, sorry the plugin nuked your config. Glad you had a backup though. Afraid I didn't do much to consider folks using nzbget prior to the plugin. 

 

Not sure why you had compilation issues.  I'm using a Slackware 13.1 VM.  Although I'm not an active user of nzbget, feel free to let me know when a new version is out as it doesn't take much to create a new build.

 

 

Rgds,

overbyrn

 

I was doing the compilation using my unRAID server. I think hugbug has changed nzbget somewhat in that new version -- maybe it has do with "now using minified versions of libraries for better performance and smaller size;"

 

When compiling, it complained about not being able to find svnversion, and possibly some other issue.

 

I was hoping you were an active nzbget user, as that would help. If you do "fall off the edge of the earth"  :-\ , I'm sure hugbug can help out with compiling issues in future, he's been helpful in the past.

Share this post


Link to post

Neil, in case I fall off the edge of the earth I've documented my build process below.  I've just done this on a vanilla UnRAID 5.0b12 system, so I know it's good for building on UnRAID as opposed to needing a keep full slackware build handy.

 

It pretty much mirrors your original config in terms of packages etc, but I've listed them all here just in case something is different.  The only extra part of the process are the steps to create a package of the compiled source.

 

Obtain and install the following packages;

bind-9.4.3_P4-i486-1.txz
binutils-2.21.51.0.6-i486-1.txz
cxxlibs-6.0.14-i486-1.txz
gcc-4.4.4-i486-1.txz
gcc-g++-4.4.4-i486-1.txz
glibc-2.13-i486-4.txz
kernel-headers-2.6.37.6_smp-x86-2.txz
libpar2-0.2-patched-rj.tgz
libsigc++-2.2.7-i486-1gsb.txz
libxml2-2.7.6-i486-1.txz
make-3.82-i486-2.txz
ncurses-5.7-i486-1.txz
openssl-0.9.8r-i486-3.txz
patch-2.5.4-i486-1.txz
pkg-config-0.25-i486-1.txz
zlib-1.2.5-i486-4.txz

NB: libpar2-0.2-patched-rj.tgz is my compiled libpar package.  My nzbget plugin downloads this from my dropbox account and you should have a copy in /boot/config/plugins/nzbget

 

Download and extract a copy of the nzb source;

cd /tmp
wget http://sourceforge.net/projects/nzbget/files/nzbget-9.0-testing-r497.tar.gz
tar zxvf nzbget-9.0-testing-r497.tar.gz

 

Enter the directory of the extracted source and create a sub-directory called pkg.  We are going to install to this directory instead of the usual operating system location so as to make creating a package easy.

cd /tmp/nzbget-9.0-testing
mkdir pkg

 

Run the pre-compilation configure phase.  The parameters ensure all the nzbget file locations are created where we want them.

./configure --prefix=/usr/local --bindir=/usr/local/nzbget --sbindir=/usr/local/nzbget --sharedstatedir=/usr/local/nzbget --localstatedir=/usr/local/nzbget --docdir=/usr/local/share/doc --datarootdir=/usr/local/nzbget/webui --datadir=/usr/local --with-tlslib=OpenSSL

 

Now we're going to make (compile) the source specifying our pkg dir as the destination.

make DESTDIR=`pwd`/pkg install

 

Create a slackware package from the compiled and 'installed' source;

cd pkg
makepkg /boot/nzbget-testing-r497-1overbyrn.tgz

During package creation, you will be asked the following:

Would you like to reset all directory permissions to 755 (drwxr-xr-x) and

directory ownerships to root.root ([y]es, [n]o)?

Answer y for yes.  Package creation will continue and you'll end up with a slackware package in the location you provided.

Slackware package /boot/nzbget-testing-r497-1overbyrn.tgz created.

 

Your next step depends on whether you intend to continue to use my plugin.  If you did, then you could replace the section of the plugin which deals with downloading the package from my dropbox with a location of your chosing.  Locate the following lines in the plugin;

<FILE Name="/boot/config/plugins/nzbget/nzbget-9.0-testing-r497.tgz" Run="upgradepkg --install-new">

<URL>http://dl.dropbox.com/u/572553/UnRAID/nzbget/nzbget-9.0-testing-r497.tgz</URL>

</FILE>

Replace the url entry for another location which points to the file also making sure to amend the previous "FILE Name" line to reflect the same file name as your telling it to get on the URL line.  If you don't want to specify a URL, you could replace the URL line entirely for something like;

<LOCAL>/boot/config/plugins/nzbget/nzbget-9.0-testing-r497.tgz</LOCAL>

This will tell the plugin to always expect a copy of the file to be at this location and if found, install it.  Then you can just drop a later version into the nzbget plugin folder on your flash drive and all will be good.

 

 

Regards,

overbyrn

Share this post


Link to post

Thanks for the documentation. I shall keep that in a safe place. Next to my cat photos.  ;)

Share this post


Link to post

just an FYI - 9.0 final is out

Thanks for the heads-up.  New version plugin (0.3) for nzbget 9.0 stable attached to first post.  Please test and confirm working.

 

 

Regards,

overbyrn

Share this post


Link to post

Running the new version. So far so good. Had some issues before, but since then I've upgraded to RC8 (from beta 12) and also upgraded the plugin so I'm not sure what the cause was. It works now that's what matters!

 

Two questions:

Is there a way to get nzbget to postprocess and move files to different folders? For example /media/tv/ or /media/music/ much like the sabnzbd category function.

Is there also a way to clean up the downloads, for example remove .sfv or .txt?

 

And of course, a big thank you for the plugin!

Share this post


Link to post

Two questions:

Is there a way to get nzbget to postprocess and move files to different folders? For example /media/tv/ or /media/music/ much like the sabnzbd category function.

Is there also a way to clean up the downloads, for example remove .sfv or .txt?

 

And of course, a big thank you for the plugin!

It does support categories:

 

iah1e.png

 

I find it easiest to put NZBs in subfolders of the blackhole and then nzbget will create and place the folders appropriately.

 

in my case, my blackhole is \\MICRO\cache\unraid_apps\nzbget\nzb\

 

If I place a TV nzb in \\MICRO\cache\unraid_apps\nzbget\nzb\TV\

 

then it gets downloaded and unpacked in to \dest\TV\nzbname\ which in my case in \MICRO\cache\TV\nzbname\

 

The only downside is unpacking separate categories to specific custom drives (like /mnt/disk4/ ) is not in the default script. I did write a script to do it, but it was a bit messy as I am not really a coder.

Share this post


Link to post

Thanks! I'll give the new version a try when I get back home next week.

Share this post


Link to post

I'm willing to try this one out. Using SABNZBD now.

 

What are advantages/drawbacks comparing SAB and NZBGet?

 

Is setup/configuring easy? Just like a quick try-out without hassle ;D

 

Can both live together?

 

 

Share this post


Link to post

I'm willing to try this one out. Using SABNZBD now.

 

What are advantages/drawbacks comparing SAB and NZBGet?

 

Is setup/configuring easy? Just like a quick try-out without hassle ;D

 

Can both live together?

See my sig for info. Yes, they co-exist. You can even run both simultaneously.

Share this post


Link to post

I installed it, and it looks great.

 

It is in /mnt/cache/.nzbget

 

The maindir also is at /mnt/cache/.nzbget

 

Is it possible to use absolute folders for the other options like can I set the destination directory to /mnt/disk/user0/Movies ?

I hope that way the downloading, verifying, repairing take place on the cache disk but the downloaded file is unpacked/moved to the array directly.

I already answered this question on this page. You can also ask questions of the developer on his forum: http://sourceforge.net/apps/phpbb/nzbget/

 

I might also use a separately mounted disk like /mnt/disk/sdk1 as "scratch" disk for nzbget to use for the messy stuff, I would set perhaps maindir to /mnt/disk/sdk1.

 

 

Can I also use absolute folders in categories?

 

If my destination path is like /mnt/user0/Movies, how would I create a categorie that puts the download in /mnt/user0/video/Bluray?

Can I set a categorie like ../video/Bluray to accomplish that?

I already answered this question on this page. You can also ask questions of the developer on his forum: http://sourceforge.net/apps/phpbb/nzbget/

 

If a download is password protected is it then possible to add the password like in SABNZB: filename / password so it will be passed on the upacking procedure?

Click on the name of the NZB in the download list, then click "PP Parameters" and you'll see:

 

oj45ch.jpg

 

Note that DestDir on this page just means it will unpack on the cache drive and move after unpacking. It's a real pain. I did ask hugbug about that, and he suggested rewriting the postprocess.sh script. I tried that, but I don't really know what I'm doing. If you (or other Linux experts) can do it, please post your script here.

 

I have been able to rewrite the script unpack to (for example) /mnt/disk4 for certain filenames, but it was a bit flaky, so I abandoned it. It is possible to do, with a custom script.

Share this post


Link to post

If you want to unpack to another directory, you have to edit the existing script or write your own.

 

I haven't seen that other error, but I'm not running the current version.

Share this post


Link to post

Well, for me it isn't working then, gonna stay with SABNZBD for a while, which is working for me.

 

I might return if the bugs are ironed out and features are expanded a bit more.

What bugs? SABnzbd crashes unRAID on a regular basis. It's a huge RAM hog. nbzget has never crashed on me.

Share this post


Link to post

http://sourceforge.net/apps/phpbb/nzbget/viewtopic.php?f=4&t=523

 

nzbget 9.1-testing-r538

 

Changes since nzbget 9.0

added full par-scan feature needed to par-check/repair files which were renamed after creation of par-files:

new option <ParScan> to activate full par-scan (always or automatic);

the automatic full par-scan activates if missing files are detected during par-check, this avoids unnecessary full scan for normal (not renamed) par sets;

replaced a browser error message when trying to add local files in IE9 with a better message dialog.

 

After upgrading to a faster ISP, I did some tests with SABnzbd and nzbget (and Newsbin on a W7 PC).

The tests were perforned on my HP Microserver, which has a dual core 1.3GHz CPU, so it's very low power and not the fastest in the world, but perfectly adequate for unRAID.

 

The best throughput I can get is 111mbps (13.9MB/sec), that's with Newsbin on my Win7 laptop, which is in constant development and the developer puts an emphasis on speed.

 

I set up SABnzbd and nzbget with the same settings -- 8 connections to the Super(Giga)News USA servers and 8 to the EU servers (16 total). All connections are SSL.

 

With SABnzbd, the best I could get was 8.5 to 9MB/sec using about 70% CPU.

 

With nzbget, I got 13.07MB/sec, also using about 65 to 70% CPU.

 

 

Share this post


Link to post

Plugin updated to version 0.4.  Original post amended, new plugin version attached.

 

This updates nzbget to the latest nightly (v10.0 r541) from svn.  It contains a useful enhancement to par post-processing. 

 

From the svn log:

added full par-scan feature needed to par-check/repair files which were renamed after creation of par-files; new option <ParScan> to activate full par-scan (always or automatic); the automatic full par-scan activates if missing files are detected during par-check, this avoids unnecessary full scan for normal (not renamed) par sets

 

Please note: this version was compiled from svn and the latest commit was less than 24hrs ago.  As with any bleeding edge version, there could be issues the developer of nzbget has introduced.  I don't use nzbget as my usenet download application so I'll leave it to others to give this verson a try.  Suggest you remain on latest stable via v0.3 of plugin if unsure or revert back to this in the event of issues with latest nightly.

 

 

Regards,

overbyrn

Share this post


Link to post

Thanks for the v10.0 build, overbyrn!

 

It's working very nicely, I just tested out the full par-repair thingy and it works very well.

Share this post


Link to post
Guest
This topic is now closed to further replies.