December 29, 201213 yr Did you restart mysql from the web interface after making the change? I'm sure I did but I went and reinstalled everything again and tried making the change without deleting the mysql-bin files and again it seems to have corrupted my database somehow.
December 29, 201213 yr WARNING: The mysql5 plugin can REALLY mess with disk-space when using the default options. By default the plugin enables bin-logging, this records every SQL statement to a mysql-bin file in the mysql root directory. With all the importing I'd been doing, I had over 300G of log files. To solve this I copied /etc/mysql/my-medium.cfg to /boot/config/plugins/mysql and renamed it my.cfg Then edited this section: # Replication Master Server (default) # binary logging is required for replication log-bin=mysql-bin to comment out the line: # Replication Master Server (default) # binary logging is required for replication #log-bin=mysql-bin Reinstalled the plg and removed the mysql-bin files Thought others might like to know Can you elaborate more on what you did to resolve this? I seen the files growing and did exactly as you said but now my newznab isn't picking up any releases Edit: Seen this in the script. Group 1 of 17 Processing alt.binaries.tv Group alt.binaries.tv has 3,843 new parts. First: 209770075 Last: 536466388 Local last: 536462545 Getting 3,843 parts (536462546 to 536466388) - 0 in queue Received 3843 articles of 3843 requested, 0 blacklisted, 0 not binaries WARNING: 3843 Parts failed to insert 76 new, 0 updated, 3,843 parts. 0.59 headers, 1.55 update, 2.14 range. Group processed in 2.24 seconds Also in the MySQL error log file is. 121228 18:12:33 [ERROR] /usr/sbin/mysqld: Table './newznab/parts' is marked as crashed and last (automatic?) repair failed 121228 18:12:33 [ERROR] /usr/sbin/mysqld: Table './newznab/parts' is marked as crashed and last (automatic?) repair failed 121228 18:12:33 [ERROR] /usr/sbin/mysqld: Table './newznab/binaries' is marked as crashed and last (automatic?) repair failed 121228 18:12:33 [ERROR] /usr/sbin/mysqld: Table './newznab/binaries' is marked as crashed and last (automatic?) repair failed 121228 18:12:33 [ERROR] /usr/sbin/mysqld: Table './newznab/parts' is marked as crashed and last (automatic?) repair failed 121228 18:12:33 [ERROR] /usr/sbin/mysqld: Table './newznab/parts' is marked as crashed and last (automatic?) repair failed 121228 18:12:33 [ERROR] /usr/sbin/mysqld: Table './newznab/parts' is marked as crashed and last (automatic?) repair failed 121228 18:12:33 [ERROR] /usr/sbin/mysqld: Table './newznab/partrepair' is marked as crashed and last (automatic?) repair failed Not sure what I did but it seems they're not getting added to the database now that I made the change? i had my newznab running for a day, tryin to learn how to pull the releases and such, and was able to pull a whole 1 nzb lol from the tv group after the initial test. so i added few more groups and run the update_scripts.php and it crashed and the entire newznab is no longer accessible on the web. i'm also seing the same mysql php errors as above. just tried the copy cfg command and i cant seem to get it working. still a linux noob here, so a quick google search tells me copy=cp and so on, but this is what i get with it... i know i'm probaly doing something wrong ... root@unRaid:~# cp /etc/mysql/my-medium.cfg /boot/config/plugins/mysql/my.cfg cp: cannot stat `/etc/mysql/my-medium.cfg': No such file or directory another question i have is the newznab plugin page ever supposed to change fron "NOT CONFIGURED" to anything else? mine is still stuck on the red "NOT CONFIGURED" even though it was working. also now that everything crashed is there a way to restart it all without rebooting the server?? i tried disabling webserver/mysql but no luck...
December 29, 201213 yr I'm not sure what is going on...it worked perfectly for me. Let me edit the post to pull the recommendation for now...I'm really sorry you all are having issues with it. I looked over your posts and it sounds like you are doing what I did, I really don't know what is going on. can one of you put your my.cfg on pastebin?
December 30, 201213 yr Ok, updated cron script is ready. Changes: Totally reworked logging, if you want it to log to console so you can test without having to tail a log file then add an option DEBUG=1 to the variables section. By default all text but errors will go to the log file defined by LOG=. Logging now has date/time stamps. I've also tested out post-processing and it seems to work for me. To test yourself make sure that -p is being called, and remove /tmp/nn-opt-last.txt. Might also remove the logfile at that point so the next run will create a new one...makes it more easy to see what is happening. Lastly, I will be releasing a plugin soon that will include logrotate, as most systems have this by default I don't want to bloat the script with this functionality. The plugin is amazingly complex, so it's going to take me a bit to get it tested out...but for now you can add a crontab to delete the log file: * 1 * * * rm /var/log/newznab_cron.log That will have no impact on the script as it doesn't maintain state with the log file. Hope this is becoming useful to people! It might be checked into newznab shortly, working with their dev team on a lot of this stuff . Tybio Link: https://github.com/Tybio/mux/raw/master/newznab_cron.sh
December 30, 201213 yr While it is not an issue, I thought you might want to know that I noticed an inconsistency in the code. The following two commands include echo as part of the text that is written to the log. log echo "INFO: Creating lock file" log echo "INFO: Setting Variables" This results in a log file entry like this. 2178: [12/30/12 04:20:01] INFO: Updating binaries 2177: [12/30/12 04:20:01] INFO: Skipped Optimization, timer expires in 640 minutes 2176: [12/30/12 04:20:01] echo INFO: Setting Variables 2175: [12/30/12 04:20:01] echo INFO: Creating lock file Update: Thanks for updating the script. It is very easy to follow what is going on in the script now. This is my first run after adding the "p" flag back in. It processed exactly as I expected. 567: [12/30/12 07:20:01] echo INFO: Creating lock file 568: [12/30/12 07:20:01] echo INFO: Setting Variables 569: [12/30/12 07:20:01] INFO: /tmp/nn-opt-last.txt not found, creating. 570: [12/30/12 07:20:01] INFO: Updating binaries 791: [12/30/12 07:22:37] INFO: Updating releases 835: [12/30/12 07:23:03] INFO: Starting optimization... 836: [12/30/12 07:23:03] INFO: Optimizing DB 838: [12/30/12 07:24:00] INFO: Getting TV Schedule 840: [12/30/12 07:24:14] INFO: Getting Movie Times 841: [12/30/12 07:24:15] INFO: Updating Release Parsing 943: [12/30/12 07:24:16] INFO: Setting timestamp 944: [12/30/12 07:24:16] INFO: Finished at Sun Dec 30 07:24:16 CST 2012, taking 4 minutes, exiting. 945: [12/30/12 07:30:01] echo INFO: Creating lock file 946: [12/30/12 07:30:01] echo INFO: Setting Variables 947: [12/30/12 07:30:01] INFO: Skipped Optimization, timer expires in 710 minutes 948: [12/30/12 07:30:01] INFO: Updating binaries 1175: [12/30/12 07:32:56] INFO: Updating releases 1210: [12/30/12 07:33:17] INFO: Finished at Sun Dec 30 07:33:17 CST 2012, taking 3 minutes, exiting.
December 30, 201213 yr Ok, updated cron script is ready.... Tybio Link: https://github.com/Tybio/mux/raw/master/newznab_cron.sh i've been searching the forum and google for the last hour on how to run this script and im still lost here do i "cd" into the update_scripts/cron_scripts ? and the main question is how do i start this newznab_cron.sh script?? i've edited it to my correct newznab path, but im lost on how to start it... every command i type in i get no file and so on. sorry for the noob questions, but im sure theres a few of us following this thread edit: i think i got it running , i had the old install directory left behind so i was trying to run the cron.sh from the old directory...dumbass... i can see the log now under simplefeatures moving along... final question, once it is running i just closed the putty session but i notice the log seems to have stopped... how do i disconnect without stopping the cron.sh job? 2nd edit: i messed it up after closing putty. now the log says this when i try running the script again 113: [12/30/12 19:01:53] ERROR: /tmp/nn-cron.lock found, exiting
December 30, 201213 yr Author Hey guys & gals, been out of town all weekend. Just getting home, I'll get settled in and try to catch up tonight!
December 31, 201213 yr Has anyone managed to get the audio preview option working? I've installed LAME and under audio releases I see a Listen button but when I click on it it appears to be playing the preview but I don't hear anything playing. I'm using Google Chrome as my main browser in OS X.
December 31, 201213 yr Tybio, found an error in your script, when I ran the latest version on github I got this: ./cron_newznab_process.sh: line 193: syntax error near unexpected token `fi' ./cron_newznab_process.sh: line 193: ` fi' To fix it I changed this: if [ $CLEAN ] { log "INFO: Cleaning up useless releases" $PHPBIN ${NEWZNAB_BASE}/misc/testing/update_cleanup.php | log } To this: if [ $CLEAN ]; log "INFO: Cleaning up useless releases" $PHPBIN ${NEWZNAB_BASE}/misc/testing/update_cleanup.php | log fi Not sure if this is 100% correct though. Thanks for the good work!
December 31, 201213 yr Also note that update cleanup does nothing if you don't comment out the two lines referenced in the file. It just echos output.
December 31, 201213 yr Or maybe that is update_parsing, it's one of the two. Double check. It's like line 32/33 or 21/22. Something like that. It is documented in the header of the file.
December 31, 201213 yr Anyone set the enable deep password scanning in the newznab setting? what would be the path to unrar on unraid? Thanks
December 31, 201213 yr Anyone set the enable deep password scanning in the newznab setting? what would be the path to unrar on unraid? Thanks Yes, I have recently turned on deep password scanning. It helps with running update_parsing.php. Most likely your unrar path is /usr/bin... however you can use the following command to check root@tower:/mnt/cache# which unrar /usr/bin/unrar
January 1, 201313 yr Fix checked in, I slipped into perl mode for some reason. sorry about that. Tybio Tybio, found an error in your script, when I ran the latest version on github I got this: ./cron_newznab_process.sh: line 193: syntax error near unexpected token `fi' ./cron_newznab_process.sh: line 193: ` fi' To fix it I changed this: if [ $CLEAN ] { log "INFO: Cleaning up useless releases" $PHPBIN ${NEWZNAB_BASE}/misc/testing/update_cleanup.php | log } To this: if [ $CLEAN ]; log "INFO: Cleaning up useless releases" $PHPBIN ${NEWZNAB_BASE}/misc/testing/update_cleanup.php | log fi Not sure if this is 100% correct though. Thanks for the good work!
January 1, 201313 yr Tybio, Still running the previous version I commented on yesterday. In my version and a previous version, I have noticed that updates can crash/lock etc. I see that you have code in it to check for this condition. # If the lockfile exists, and the process is still running then exit if [ -e ${LOCKFILE} ]; then LOCKFILE_AGE=$(find ${LOCKFILE} -mmin +119) if [ -n $LOCKFILE_AGE ];then log "ERROR: $LOCKFILE found, exiting" exit else log"ERROR: $LOCKFILE is $LOCKFILE_AGE, removing it and continuing" kill -TERM -`cat ${LOCKFILE}` echo $$ > ${LOCKFILE} fi else log echo "INFO: Creating lock file" echo $$ > ${LOCKFILE} fi For some reason, it does not appear to catch that the file is old and should be deleted. I issued the following command and the script starts running again. kill -TERM -`cat /tmp/nn-cron.lock`
January 1, 201313 yr I am trying to get Newznab+ installed with the plugin. But I am not getting anywhere, can someone provide some steps that I should do? I am not a linux guru by any strech, but got all other SB, Sab working fine with the plugins. I have the feeling the webserver is not working. I update the config in the settings tab and press apply. But soon as I go back in there its back to the original settings. Also I cant start the newznab plugin because there is no start option.
January 1, 201313 yr Right, they are not removing the lock when the stupid perl style if statement is found. I've just duplicated the situation here and it looks like that -mmin if statement is broken. Let me dig into this a bit more, I'll get something checked in soon and let you know. Tybio, Still running the previous version I commented on yesterday. In my version and a previous version, I have noticed that updates can crash/lock etc. I see that you have code in it to check for this condition. # If the lockfile exists, and the process is still running then exit if [ -e ${LOCKFILE} ]; then LOCKFILE_AGE=$(find ${LOCKFILE} -mmin +119) if [ -n $LOCKFILE_AGE ];then log "ERROR: $LOCKFILE found, exiting" exit else log"ERROR: $LOCKFILE is $LOCKFILE_AGE, removing it and continuing" kill -TERM -`cat ${LOCKFILE}` echo $$ > ${LOCKFILE} fi else log echo "INFO: Creating lock file" echo $$ > ${LOCKFILE} fi For some reason, it does not appear to catch that the file is old and should be deleted. I issued the following command and the script starts running again. kill -TERM -`cat /tmp/nn-cron.lock`
January 1, 201313 yr Ok, fix for the stale lock file checked in. I've tested it the best I can, but lots of variables at play here. Give it a spin.
January 1, 201313 yr Ok, fix for the stale lock file checked in. I've tested it the best I can, but lots of variables at play here. Give it a spin. Worked like a charm. It was hung and on the expected run, it cleared the file and started executing the appropriate scripts. I did not notice at first that you removed the -q flag. Was 59 minutes just an arbitrary number or was there a specific reason to select it. The link below is for people that may not no where you have the files. I know I had to go looking for them as I have just been using the versions you have been attaching to the thread. https://github.com/Tybio/mux
January 1, 201313 yr Just an arbitrary number, I know that if I get much more than an hour behind things get backed up...and my normal run is about 4 minutes, so an hour seemed fair.
January 1, 201313 yr By the way, people going to the link for the cron script...be wary of the plugin...It will not /break/ anything, but I'm not quite finished with it yet. I need to clean up a few items before formal release, like setting it up to optionally install mediainfo and then test startup/shutdown. If you want to mess with it...give it a go however...but the rist is yours Tybio
January 2, 201313 yr Tybio, I gotta say thanks for the CRON script. I don't know anything about well, any of this stuff! But between Influencer's plugin and your scripts and help on the forums I managed to get Newznab up and running on my unraid server and, ultimately, decide to move it over to a small separate webserver I built from spare parts around the house cause I just didn't want to risk too much stuff I don't know about on my unRaid server. I'm still using your Cron script there but, the lock file has me wondering a bit. I checked and it ended up starting a second run before the first run finished so I killed everything and increased the time in the script for the lock file check. This had me wondering, and I apologize since I don't know anything about this stuff but, since my log file shows an entry like this: [01/01/13 16:29:44] Received 20001 articles of 20001 requested, 0 blacklisted, 0 not binaries [01/01/13 16:30:01] ERROR: /tmp/newznab_cron.pid is stale, removing it and continuing ... would it be better, or even possible, to check the log file for the last action to determine if things have frozen or crashed. Say check if the last entry was more than 60 minutes ago then we assume things crashed and it is safe to start a new run, or just check the modified date/time of the log file and assume that's the last action time? Like I said, I don't know about this stuff so sorry if I'm way off base here. Regardless I appreciate the hell out of your work on this, it has helped me a ton!!!
January 2, 201313 yr I just changed the value of mine to 120 minutes. I have seen it take up about an hour on my system when I was about a day behind. I believe two hours is more than adequate for any unforeseen issues.
Archived
This topic is now archived and is closed to further replies.