May 19, 201214 yr Some other people have requested it in the past so I figured Id ask one last time. I know its not strictly necessary but I know some of the news groups I like aren't indexed so Id like to do it myself. Also having a backup when sites go down is also a nice thing to have.
December 9, 201213 yr I would like to second this, with the events of the last few days I think this is more useful than ever
December 9, 201213 yr funny idea... running your own usenet indexer locally... maybe i'll look into it (not saying, that nobody else should...) without taking a closer look at newznab, i'd guess it will need a lot of space for its data...
December 9, 201213 yr funny idea... running your own usenet indexer locally... maybe i'll look into it (not saying, that nobody else should...) without taking a closer look at newznab, i'd guess it will need a lot of space for its data... Not necessarily, it's only headers -- space used will be determined by the number of groups, the size of those groups and how many days retention you want to keep. keeping 1,000 days is probably not the best idea.
December 9, 201213 yr I third this. All my index sites are gone and I was just looking at building a NewzNAB server today.
December 10, 201213 yr I was able to get it working with the following: SimpleFeatures web server, installed along with the suite of SimpleFeatures plugins here: http://code.google.com/p/unraid-simplefeatures/ This is enough to get started, since it fills all the requirements on this page: http://newznab.readthedocs.org/en/latest/install MySQL server plugin from the Wiki: http://lime-technology.com/wiki/index.php/UnRAID_Plugins Installed/configured MySQL (had to run "chmod 1777 /tmp" before it'd install and set the password correctly), then followed this guide to install Newznab in Ubuntu: http://newznab.readthedocs.org/en/latest/guides/install_ubuntu-11.10/ Obviously this is for Ubuntu, but it's enough to get you started. Since SimpleFeatures comes with lighttpd, you'll need to use that instead. I was able to configure it via the web interface and point the webroot at the newznab root. I created a separate share for all newznab files, since it can grow very large if you have a lot of newsgroups indexing. I should note that I'm not using Newznab Plus, since I want to test everything first. So far the site's running fine, and I'm indexing a few newsgroups without any real issues.
December 11, 201213 yr I installed newznab on a VM for fun...lots of requirements for it. Anyone figured out how to disable email from within newznab? I'd rather deal with just invite links and the like and not have to worry about getting my unraid box setup with a valid domain etc.
December 11, 201213 yr I'll try to take a look soon as well, no promises though. VERY busy time of year. Really the "plugin" itself wouldn't be much, I would hate to write in another web-server for the plug-in since there is already one available, and with it installed that gives you most of the dependencies you need, so the plug-in would really only have to pull down newznab and unzip it in the "data directory". Everything else is setup from within newznab.
December 11, 201213 yr Well, there are a few things (Now that I've done the install and have it working) that I wanted to share. 1> Sphinx: After doing some tests with, and without Sphinx...on low power unraid servers it is going to be very important. My i7 desktop saw a huge speed bump from having searchd running. 2> Extras: Lots of people might not care about thumbnailing, but having the following makes the index much more complete FFmpeg v0.8.7 MediaInfo v0.7.50 Lame v3.98.4 unrar v4.00-beta3 3> Needs screen to run in default method. There is a init.d method, but I haven't looked into it yet to see how easy it is to run in the unraid setup. 4> Running in lighttpd seems to work perfectly fine as steeef noted 5> After 12 hours with no backlog search...so only indexing 12 hours of posts these are my disk stats: newznab directory: 120M mysql directory: 708M I'm not sure how fast they will grow, I think a large chunk of that is rolling indexes...so it should grow much slower over the coming weeks...but we should collect some data from people already running it over the next little while and have a recombination on setup. Otherwise people will point at their 20G cache drive and get into all sorts of trouble .
December 11, 201213 yr Well, there are a few things (Now that I've done the install and have it working) that I wanted to share. 1> Sphinx: After doing some tests with, and without Sphinx...on low power unraid servers it is going to be very important. My i7 desktop saw a huge speed bump from having searchd running. 2> Extras: Lots of people might not care about thumbnailing, but having the following makes the index much more complete FFmpeg v0.8.7 MediaInfo v0.7.50 Lame v3.98.4 unrar v4.00-beta3 3> Needs screen to run in default method. There is a init.d method, but I haven't looked into it yet to see how easy it is to run in the unraid setup. 4> Running in lighttpd seems to work perfectly fine as steeef noted 5> After 12 hours with no backlog search...so only indexing 12 hours of posts these are my disk stats: newznab directory: 120M mysql directory: 708M I'm not sure how fast they will grow, I think a large chunk of that is rolling indexes...so it should grow much slower over the coming weeks...but we should collect some data from people already running it over the next little while and have a recombination on setup. Otherwise people will point at their 20G cache drive and get into all sorts of trouble . How many groups are you indexing? Just movie and tv groups or more then that? Do you think it would be much more processing power for sphinx and some of the other addons? Probably a stupid question but what do mean by default method? I assume you are talking about it running the update scripts automatically and a way for us not to have to worry about consoling a screen on? Currently have backfilled 7-14 days on most of the TV and Movie groups, and my msql directory is little less then yours, need to check my nzb folder.
December 11, 201213 yr How many groups are you indexing? Just movie and tv groups or more then that? Do you think it would be much more processing power for sphinx and some of the other addons? Probably a stupid question but what do mean by default method? I assume you are talking about it running the update scripts automatically and a way for us not to have to worry about consoling a screen on? Currently have backfilled 7-14 days on most of the TV and Movie groups, and my msql directory is little less then yours, need to check my nzb folder. I'm indexing a bunch of groups, which is why I think a large part of my usage is cached headers that should roll off soon...so the size should be more stable than growing at this rate forever . Yep, the "easy" way running: sudo screen sh <NEWZNAB_PATH> newznab_screen_local.sh [code] Yes, I need to fix the sudo...I ran it from root the first time and got all the permissions jacked up...whopse! I'm actually thinking that this is best done from Cron...I found an example init.d script but it's just a loop: http://pastebin.com/T7stkSPx Might be far better to create a tiny wrapper shell script to call from Cron every 5 minutes. Just create a PID file in it so they don't over-run each other. I'll work on the cron system and post here tonight if I make progress.
December 11, 201213 yr How many groups are you indexing? Just movie and tv groups or more then that? Do you think it would be much more processing power for sphinx and some of the other addons? Probably a stupid question but what do mean by default method? I assume you are talking about it running the update scripts automatically and a way for us not to have to worry about consoling a screen on? Currently have backfilled 7-14 days on most of the TV and Movie groups, and my msql directory is little less then yours, need to check my nzb folder. I'm indexing a bunch of groups, which is why I think a large part of my usage is cached headers that should roll off soon...so the size should be more stable than growing at this rate forever . Yep, the "easy" way running: sudo screen sh <NEWZNAB_PATH> newznab_screen_local.sh [code] Yes, I need to fix the sudo...I ran it from root the first time and got all the permissions jacked up...whopse! I'm actually thinking that this is best done from Cron...I found an example init.d script but it's just a loop: http://pastebin.com/T7stkSPx Might be far better to create a tiny wrapper shell script to call from Cron every 5 minutes. Just create a PID file in it so they don't over-run each other. I'll work on the cron system and post here tonight if I make progress. Yah in all reality.. If someone did make a plugin for this, I would think they just layout steps to install simplefeatures web server, and then the plugin just be the install of newznab and then have a config setting up of the chron / init.d and setting a frequency of when to grab..
December 11, 201213 yr Following information in this thread I've managed to get newznab plus up and running. Pretty much just a case of install simple features web server and then install newsnab. Question - what groups are recommended for tv shows and movies? I've added a few likely looking ones to start with but it doesn't seem to be picking up many tv shows (not showing the latest Dexter for example). If I understand rightly, the default settings for new groups is to scan back for 3 days or 50000 posts, so I would have thought that Dexter would have been picked up (binsearch shows it in a.b.teevee in that period and that is one of the groups I'm experimenting with). I'm currently running a backfill for 7 days of a.b.teevee to see if that picks it up.
December 11, 201213 yr Following information in this thread I've managed to get newznab plus up and running. Pretty much just a case of install simple features web server and then install newsnab. Question - what groups are recommended for tv shows and movies? I've added a few likely looking ones to start with but it doesn't seem to be picking up many tv shows (not showing the latest Dexter for example). If I understand rightly, the default settings for new groups is to scan back for 3 days or 50000 posts, so I would have thought that Dexter would have been picked up (binsearch shows it in a.b.teevee in that period and that is one of the groups I'm experimenting with). I'm currently running a backfill for 7 days of a.b.teevee to see if that picks it up. I think it also has to do with the regex set up for newzbin. I added couple groups to test, and it only picked up one episode of How I met your mother. I'm using the free version. I wonder if it's worth it to go with the donated version of newzbin
December 11, 201213 yr I think it also has to do with the regex set up for newzbin. I added couple groups to test, and it only picked up one episode of How I met your mother. I'm using the free version. I wonder if it's worth it to go with the donated version of newzbin I am running with the donated version of newznab.
December 11, 201213 yr I think it also has to do with the regex set up for newzbin. I added couple groups to test, and it only picked up one episode of How I met your mother. I'm using the free version. I wonder if it's worth it to go with the donated version of newzbin I am running with the donated version of newznab. Yea. just saw that. sorry lol
December 11, 201213 yr Would it make sense to a vpn together with Newznab? Sent from my GT-I9100 using Tapatalk 2
December 11, 201213 yr I was able to get it working with the following: SimpleFeatures web server, installed along with the suite of SimpleFeatures plugins here: http://code.google.com/p/unraid-simplefeatures/ This is enough to get started, since it fills all the requirements on this page: http://newznab.readthedocs.org/en/latest/install MySQL server plugin from the Wiki: http://lime-technology.com/wiki/index.php/UnRAID_Plugins Installed/configured MySQL (had to run "chmod 1777 /tmp" before it'd install and set the password correctly), then followed this guide to install Newznab in Ubuntu: http://newznab.readthedocs.org/en/latest/guides/install_ubuntu-11.10/ Obviously this is for Ubuntu, but it's enough to get you started. Since SimpleFeatures comes with lighttpd, you'll need to use that instead. I was able to configure it via the web interface and point the webroot at the newznab root. I created a separate share for all newznab files, since it can grow very large if you have a lot of newsgroups indexing. I should note that I'm not using Newznab Plus, since I want to test everything first. So far the site's running fine, and I'm indexing a few newsgroups without any real issues. I must be seriously missing something, but where is the actual code located to download ?
December 11, 201213 yr I must be seriously missing something, but where is the actual code located to download ? from here.
December 11, 201213 yr I sent a user step by step instructions last night. He said he would post a good guide on it. I would but haven't had time to sit and do it.
December 11, 201213 yr I'll be standing mine up in a VM under ESX, a tutorial in this would be awesome. I'll use the purchased version, the Regex stuff is a concern of mine. I hadn't realized all these sites I'd been using for indexing were running something I could setup myself!
December 11, 201213 yr Check out the first few comments here for a regex workaround (at least enough of one to test).http://www.reddit.com/r/usenet/comments/14krxf/newznab_gotta_pay_to_play/
Archived
This topic is now archived and is closed to further replies.