newznab + unRAID integration?

[shat]

@jake:

 

Newznab+ is the version you receive from donating to the developers. It includes a database (current release 2513) of submitted and tested regular expressions that transform a headers post title into a usable release.

 

Most users are not successful in their attempts at setting up newznab without having either (a) the stock regex library provided from donation or without prior experience with regular expressions and writing your own.

 

As a developer, I must say, personally regular expressions are one of my least favorite aspects of development and a topic that takes time to master.

 

I would suggest possibly looking into an already established index to use and save yourself the time, trouble, resources and bandwidth associated with operating an even modest newznab index. Yes, I know I am plugging my own index I offer up to fellow unraid users, but it had its definite pros.

 

See topic, nzb index, private in lounge thread for info.

 

If you do opt to operate your own index I am always around to help out.

[jakebake]

@jake:

 

Newznab+ is the version you receive from donating to the developers. It includes a database (current release 2513) of submitted and tested regular expressions that transform a headers post title into a usable release.

 

Most users are not successful in their attempts at setting up newznab without having either (a) the stock regex library provided from donation or without prior experience with regular expressions and writing your own.

 

As a developer, I must say, personally regular expressions are one of my least favorite aspects of development and a topic that takes time to master.

 

I would suggest possibly looking into an already established index to use and save yourself the time, trouble, resources and bandwidth associated with operating an even modest newznab index. Yes, I know I am plugging my own index I offer up to fellow unraid users, but it had its definite pros.

 

See topic, nzb index, private in lounge thread for info.

 

If you do opt to operate your own index I am always around to help out.

 

THank you! Yeah I did see your other thread. How do I get an invite or donate? I definitively don't want to take the time to set my own up (nor do I know how) so just using an already established one would be much better haha. I am still a little unclear on if I need newznab at all though if I decide to go with yours. If I donate do I get newznab+ that downloads from your index or does it just work like something like nzbmatrix?

[aoaaron]

i've been scammed twice already by donating to sites which have not existed...

 

newzab doesn't seem possible for me due to bandwith issues etc and my unraid box is pretty weak.

 

any reccomendations for sites please to join?

[zack84a]

i've been scammed twice already by donating to sites which have not existed...

 

newzab doesn't seem possible for me due to bandwith issues etc and my unraid box is pretty weak.

 

any reccomendations for sites please to join?

 

Check the lounge.  There is an unraid member (Shat) that is setting up a private site, and is inviting unraid members to join.  Considering he is a member here, and people have been joining without problem, I don't forsee any shady stuff.  He has been a great help to people trying to get their own server up and running as well. 

[shat]

i've been scammed twice already by donating to sites which have not existed...

 

newzab doesn't seem possible for me due to bandwith issues etc and my unraid box is pretty weak.

 

any reccomendations for sites please to join?

 

he joined. He knows there is nothing shady over here.

[shat]

i've been scammed twice already by donating to sites which have not existed...

 

newzab doesn't seem possible for me due to bandwith issues etc and my unraid box is pretty weak.

 

any reccomendations for sites please to join?

 

Check the lounge.  There is an unraid member (Shat) that is setting up a private site, and is inviting unraid members to join.  Considering he is a member here, and people have been joining without problem, I don't forsee any shady stuff.  He has been a great help to people trying to get their own server up and running as well.

 

Appreciate the referral. There have been a lot of Unraid users request invites and all is running well as everyone seems to be quite pleased.

 

It's a continuing effort. Just talked a buddy into closing his index (just him) to help me with mine as I needed another set of eyes to look after things when I can't.

[darbronnoco]

I'm pretty close to getting this up and running but I don't have something set correctly.  I think it probably has to do with the rewrite file. 

 

My Newznam ServerRoot setting in the plugin is /mnt/cache/TowerApps/WebserverRoot/

and the web server root is /mnt/cache/TowerApps/WebserverRoot/

 

Mod Rewrite

# User customizations
# Add your own settings here. These won't get overwritten upon reboot
#
url.rewrite-once = (
"^/.*\.(css|jpg|jpeg|gif|png|js|ico)" => "/tmp/checkconfig",
  "^/(admin|install).*$" => "/tmp/checkconfig",
  "^/([^/\.]+)/?(?:\?(.*))$" => "index.php?page=&",
  "^/([^/\.]+)/?$" => "index.php?page=",
  "^/([^/\.]+)/([^/]+)/?(?:\?(.*))$" => "index.php?page=&id=&",
  "^/([^/\.]+)/([^/]+)/?$" => "index.php?page=&id=",
  "^/([^/\.]+)/([^/]+)/([^/]+)/?$" => "index.php?page=&id=&subpage=" )

 

At one point if I navigated to ip:81/www/install  the install page showed up.  I think I need to either adjust a path or the rewrite since the install seems to work otherwise.  Any suggestions?  Thanks.

[Helmonder]

i've been scammed twice already by donating to sites which have not existed...

 

newzab doesn't seem possible for me due to bandwith issues etc and my unraid box is pretty weak.

 

any reccomendations for sites please to join?

 

Consider using spotweb... It is much lighter on the system and used a fraction of the bandwidth newznab does.. It is LIKE newznab in the way that it gives you your own nzb files, but they are not created by the tool itself but by the total community, they scan the newsgroups and "tag" releases with the correct names.. works a charm !

[AlexNathan345]

I wonder if anyone could help me: My Newznab+ installation on unRAID was working fine for about 3 weeks now until yesterday. When I go into the web GUI there aren't any groups anymore, and therefore also no releases. When I try to add a new group it also doesn't show that group - the list is empty (both in front end and admin area). When I bulk add a list of groups it outputs that it added some and updated others but still doesn't show any. So the weird thing is they must be there somewhere (database) but they don't show/work. Any idea how I can fix this?

 

Before this stopped working I've done a few updates, one of them might have contributed to the problem: I updated unRAID from v5.0rc8a to v5.0rc9a (and re-booted the server and ran the 'permissions' script as advised in the update thread). I also updated the Influencer plugin for Newznab and (as I do regularly) updated Newznab using 'svn update' on the installation directory.

 

Any help would be very much appreciated, as the installation is useless in its current state. Thanks a lot!

[lainie]

I would try going to your newznab database directory (/path/to/mysql/newznab) & run "myisamchk *.MYI" to see if it reports any errors. More details on how to repair any errors found can be found here: http://dev.mysql.com/doc/refman/5.0/en/myisam-repair.html.

 

I run a mysqldump daily to keep recent backups available, but running the repairs listed in the link above has worked for me in the past so that I did not have to restore from a backup.

 

There may be a different method if you have switched to using Innob instead of the default Myisam tables.

[AlexNathan345]

Thanks so much for your quick reply. I'm running the standard setup so this should work.

 

I ran "myisamchk *.MYI" which said that most/all tables are marked as crashed and that the last repair failed (I never ran one).

 

I then tried to use the first repair step from your linked instructions (stopped MySQL, ran "myisamchk -r" on one of the tables but get the error "myisamchk: error: Can't create new tempfile: 'userseries.TMD' - MyISAM-table 'userseries.MYI' is not fixed because of errors".

 

Unfortunately, I'm not at all familiar with MySQL so don't quite know what's going on here. But I'm very willing to learn. Again, thanks for helping me out.

[AlexNathan345]

LAINIE, you saved my day. I went through the repairs list and repaired all files. Now Newznab is back up and running again - THANKS SO MUCH!!!

[lainie]

Glad to hear that worked for you. 

 

Now, start running regular backups. Those repair instructions will fix many issues, but sometimes you will get a corruption that just does not want to be fixed.

 

I added this line to cron. It creates a backup (& gzips it for a smaller file size) every morning at 2:15am.

15 2 * * * mysqldump -u root -pmysqlpassword newznab | gzip > /mnt/user/backups/newznab_`date +%Y%m%d`.sql.gz

Notice that there is no space between -p and your mysqlpassword.

Change the path to a good location for your backup.

The `date +%Y%m%d` for today's date is 20130111, this can be adjusted to whatever format you prefer.

 

You can manually remove old backups or script something to your needs.

 

Keep in mind that cron is stored in volatile memory, so any changes made will not be kept on a reboot. I have edited my go file to recreate a few of my custom cron jobs.

[AlexNathan345]

Backups of the database sound like a very sensible idea, thanks for the tip. Am I right in assuming that this needs adding to the cron.d file found in /etc?

 

And what would I need to add to the go script to have this re-installed after a reboot? Sorry for all these pretty basic questions, but my Linux/SQL knowledge is pretty basic...

[lainie]

Backups of the database sound like a very sensible idea, thanks for the tip. Am I right in assuming that this needs adding to the cron.d file found in /etc?

 

And what would I need to add to the go script to have this re-installed after a reboot? Sorry for all these pretty basic questions, but my Linux/SQL knowledge is pretty basic...

 

Mine is stored in /var/spool/cron/crontabs/root.

 

There are many ways to set it in the go file... I have multiple custom lines that I added to a file I created & named "/boot/custom/mycron".

 

I added this to my go file (one step to add my custom lines to the existing cron):

# Add custom jobs to crontab
cat /boot/custom/mycron >> /var/spool/cron/crontabs/root 

 

If this is the only line you are adding, you may prefer this:

echo "15 2 * * * mysqldump -u root -pmysqlpassword newznab | gzip > /mnt/user/backups/newznab_`date +%Y%m%d`.sql.gz">> /var/spool/cron/crontabs/root 

Think I used echo in the past with the date & it actually interpreted the date instead of keeping the `date +%Y%m%d` format... which defeats using a date stamp. This is fine if you aren't using a data stamp or if you are only interested in keeping one backup & trust yourself to catch any corruption before it is overwritten.

 

Here is another method; creates a backup of the existing cron, then adds the custom cron lines:

# Add custom jobs to crontab
crontab -l > /boot/custom/systemcron
cat /boot/custom/mycron >> /boot/custom/systemcron
cat systemcron | crontab -

 

[AlexNathan345]

After updating newznab+ last night (svn update and database patches) I ran into another problem - it doesn't add any releases anymore. When investigating the problem I found this in the newznab chat:

 

"newznab has moved to using the mysqli php mysql library. you will need to include it in your php.ini if its not present, on windows this would look like extension=php_mysqli.dll, or extension=mysqli.so in arch"

 

Do I have to do anything here? Where's the php.ini file on unRAID?

[lionelhutz]

Odd, I updated 2 nights ago and it's still updating just fine.

 

How did you do the update?

 

Do you still have the registration code in the admin setup page like the email instructs?

[AlexNathan345]

Yes, reg code is still in there.

 

Updated using 'svn update' and then running the 'update_database_version.php' script. Updated with svn before but never ran the update_db script before...

 

Weird thing is: whenever I run 'optimise_db.php' it seems to 'break' the database - the above suggested 'myisamchk' finds and corrects lots of errors with the files every time I 'optimise' so something can't be quite right.

 

Any ideas?

 

PS: When running 'update_binaries.php' I get lots of 'repair' messages - might this be part of the problem? Might the problem be my newshosting provider? Any way I can check this to narrow the problem down?

[zack84a]

After updating newznab+ last night (svn update and database patches) I ran into another problem - it doesn't add any releases anymore. When investigating the problem I found this in the newznab chat:

 

"newznab has moved to using the mysqli php mysql library. you will need to include it in your php.ini if its not present, on windows this would look like extension=php_mysqli.dll, or extension=mysqli.so in arch"

 

Do I have to do anything here? Where's the php.ini file on unRAID?

 

In the same place you are in that releases aren't getting added after this last update.

 

Your php.ini is location on your flash drive in the plugins folder (specifically the simplefeatures).. However we have that line already there, so I am guessing thats not it..

 

I watching for noticeable errors in my scripts but currently aren't seeing them.

 

[AlexNathan345]

Are you using Newshosting as your provider, too?

 

Just read this on their site: http://www.newshosting.com/en/newshosting-usenet-faq-new-headers.php

 

Might be (part of) the problem? Don't quite know how to address this if it is...

[zack84a]

Are you using Newshosting as your provider, too?

 

Just read this on their site: http://www.newshosting.com/en/newshosting-usenet-faq-new-headers.php

 

Might be (part of) the problem? Don't quite know how to address this if it is...

 

I am using newshosting.. to you have the article that explains whats happening to headers, little confused by FAQ.

 

Next question is who to switch to :\

[zack84a]

Alright I did some repairs to my database using HeidiSQL (check,repair,optimize) and ran update_binaries and update_releases and its creating the releases again..  So it appears like I am back to 100%.

[AlexNathan345]

Installed HeidiSQL on my Windows machine, connected to database, verify is ok. Then started repair but only get error messages (error when fixing table / can't change ownership of xxx file / repair status: operation failed) - any idea where I'm going wrong here?

[zack84a]

Installed HeidiSQL on my Windows machine, connected to database, verify is ok. Then started repair but only get error messages (error when fixing table / can't change ownership of xxx file / repair status: operation failed) - any idea where I'm going wrong here?

 

Try running heidisql with root sql account?  those commands could also be ran via sql on the machine..

[AlexNathan345]

Thanks, but I was running them as root. How would I ran the commands directly on the machine (from telnet)? I used various myisamchk commands before - is this essentially the same thing?