New Application: MovieGrabber - unMenu pkg (without Python 2.7) - Updated 3/11


Recommended Posts

First things first, "Hi all, I'm new here, this is my first post, great to be on board" and all that. Just setup my unraid system today/yesterday (haven't gone to bed yet), and I decided to make a package. It was easier than I thought it was going to be, so I might have missed something. Bought a pro license before I even had everything for the server, so I am anticipating that people have a cache drive. Not required, but generally a good idea with this one.

 

Originally set out to start using python 2.7 for a handful of things, and ended up realizing I can get away with 2.6. Moviegrabber is neat though, so I decided to mod it (via the installer) so that it runs under python 2.6.

 

This requires the argparse library, conf for that is here too. Grab both files, drop them into your package directory, and install via unmenu.

 

Let me know if there are issues, I'd love to know about them, being my first package. Tested whatever I could though, dropping files, different locations, clean installs, upgrades, etc. Version number is pulled from the python file, and the only thing that is a bit kludgy (IMHO) yet is the way it daemonizes. Basically just spawns a background job, and then stop command kills it. Works well enough, but I'd love to have a "daemon" command in unraid, although it's definately overkill.

 

Thanks to prostuff1 and mrmachine, based the scripts off of some of their work. Thanks to Joe L for help with daemonization so you can use the start moviegrabber user script.

 

CHANGE LOG

2/26/12 - Initial upload, tested out a number of ways, seems to work nicely other than userscript to start MovieGrabber.

2/26/12 - Modified to handle user script to start (or stop) MovieGrabber properly. No known bugs, but that doesn't mean there aren't any.

3/11/12 - Updated version to 2.1.7, which seems to work well. Tested moviegrabber out, where it was giving an Sqlite error before as it was sharing a connection between processes, it's not failing anymore. So long as your search criteria is set correctly, looks like it functions fine now. No more requirement to modify the decimal.Decimal call in the code anymore either.

3/11/12 - Updated rc.d script to properly detect started/stopped state. (Always saw it as started)

 

If you have any issues with the INSTALL please post the following:

unRAID version, other addon's, and the output of all these commands

ps -ef | grep MovieGrabber
cat PACKAGE_DIRECTORY/moviegrabber-src-2.1.7.zip.auto_install
cat PACKAGE_DIRECTORY/moviegrabber-src-2.1.7.zip.manual_install
ls ***OF THE INSTALL DIRECTORY OF moviegrabber***
cat *** moviegrabber install directory***/configs/webconfig.ini
cd ***moviegrabber install directory***
cat /var/log/syslog | grep -i MovieGrabber
python MovieGrabber.py

 

Let that last one run for about 30 seconds, and Ctrl+C to shut down the server. Give me the output from all those, and I should be able to work out most issues.

 

This requires python (2.6), openssl, sqlite, python-argparse (included here)

 

Home page: http://forums.sabnzbd.org/viewtopic.php?f=6&t=8569

Some screenshots (from homepage):

avn5lu.jpg

2mi0s2v.jpg

ayafif.jpg

argparse-unmenu-package.conf

moviegrabber-unmenu-package.conf

Link to comment
  • Replies 63
  • Created
  • Last Reply

Top Posters In This Topic

Nice work. 

Clearly you are not a beginner at programming. 

 

As far as demonizing..

 

Techniques I've used are

nohup whatever &

or

echo "whatever" | at now

or

have the daemon script detach as a child process (disown it) like this:

 

Invoke daemon process, putting it in tht background, sending output from stdout and stderr somewhere.  Then disown it.

process_to_daemonize >/var/log/logfile 2>&1 &

disown %%

 

Glad to have you helping.  I'm sure people will be interested.  You certainly learned a lot looking at the existing examples.  The unMENU package file format was designed to be easy to understand if you knew shell scripting.

 

Link to comment

Hey, that's pretty cool. I like that: "echo 'whatever' | at now" Would have never thought of that myself. I ALWAYS forget about 'at'.

 

I've got some code in flexget, among a number of other things in the open source world. I've been around for a little while, but tend to just poke at other people's stuff and make it better, unless there's a compelling reason to write my own (sometimes I poke to much and rewrite large portions of the original...). Ideally I'll be throwing up a maraschino package in a few minutes here too, whetting my pallete before jumping into the larger stuff. In the meantime, I'll fix the daemonization with your suggestions. I learn something new every day, didn't even know about disown...

Link to comment

Glad you managed to take some inspiration from my unMenu packages!

 

I will take a look through them and see if I find any suggestions to improve them.

 

 

EDIT:  Looks good to me.  It is almost like deja vu looking at the code!!  I knew separating all those packages related to python and the SAB, SickBeard, and CouchPotato packages would be the correct move.

Link to comment

I learn something new every day, didn't even know about disown...

I learn something about "bash" every day too... and I've been working on UNIX/linux since about 1980.  (about 32 years.... yikes)  I've had one version or another of UNIX at home since 1985.  Today, between xbmc clients, android tablets & phones, and unRAID servers, I have about 8 or 9 Linux OS operating at home.

 

I did not know of "disown" until I looked for a way to set a new process group. (effectively disassociating a program from a controlling terminal)

I think I used it in my cache_dirs script first.

 

A couple of days ago I learned of the &> operator in it.

&>filename

      # Redirect both stdout and stderr to file "filename."

      # This operator is now functional, as of Bash 4, final release.

Another unRAID user used it in a script.

 

I grew up with the Borne shell, and then the Korn shell, and before those two, the Mashey shell  (It still had labels and gotos).

 

I've often been forced to use utilities in creative ways.  That is how I ended up writing unMENU's web-server and plug-in system in "awk"

 

Glad to have you helping automate installation of some useful utilities.    It looks like MovieGrabber will be very interesting for some.

 

Joe L.

Link to comment

Awesome guys, thanks for the encouragement. Once I get all this stuff integrated together, is there interest in a v5 plugin? I'm integrating sickbeard, sabnzbd, moviegrabber, couchpotato, flexget (for the stuff outside the realm of the others), pyload, mysql DB for XBMC, and maraschino (just pointing it to the right place really), so it might take a bit.

 

I'm kind of liking things as I'm looking at them for v5 plugins, I'd like to have auto-start/stop on array start/stop, since right now my stuff isn't to hot with an unmounted array (if you store stuff on cache directory rather than SNAP).

Link to comment

After installing it, while trying to set it up Im getting this error:

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/CherryPy-3.2.2-py2.6.egg/cherrypy/_cprequest.py", line 656, in respond
    response.body = self.handler()
  File "/usr/lib/python2.6/site-packages/CherryPy-3.2.2-py2.6.egg/cherrypy/lib/encoding.py", line 188, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/CherryPy-3.2.2-py2.6.egg/cherrypy/_cpdispatch.py", line 34, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/mnt/cache/.moviegrabber/MovieGrabber.py", line 3819, in save_config_imdb
    if decimal.Decimal(kwargs["good_rating2"]) != decimal.Decimal(0.0) and decimal.Decimal(kwargs["preferred_rating2"]) >= decimal.Decimal(kwargs["good_rating2"]):
  File "/usr/lib/python2.6/decimal.py", line 649, in __new__
    "First convert the float to a string")
TypeError: Cannot convert float to Decimal.  First convert the float to a string

 

From looking at sabnzbd forum, it looks like this was already solved, just wanted to know if anyone else is getting this error.

Link to comment

Yep, there were two lines without the quotes.  Unfortunately now I coming into another issue, but that one will be addressed in the next version.  Can you add the stop and start buttons to the user scripts area like prostuff1 adds to his.  Just for consistancy  :D

 

Those should already be in there... They are there on mine.

 

I assume you're talking about this issue? http://forums.sabnzbd.org/viewtopic.php?f=6&t=8569&start=300#p61875

 

That's supposed to be fixed in new version due out in a few days. I spent a while trying to get that far, ended up finding out that my search criteria wasn't perfect (still don't know what was wrong with it), and the background service wasn't running. He'll be putting in better logging later, so ideally will be able to work everything out shortly.

Link to comment

what a great port !...its so good to have guys like you here doing stuff like this, bringing UNraid to the masses, the more user friendly it becomes the more people will look to it for everyday use !!, i know just enough to get things running, in an ideal world it will all be install packages at a click of a button..thank you !

Link to comment

what a great port !...its so good to have guys like you here doing stuff like this, bringing UNraid to the masses, the more user friendly it becomes the more people will look to it for everyday use !!, i know just enough to get things running, in an ideal world it will all be install packages at a click of a button..thank you !

 

agreed...

 

With at least 2 new unMenu packages and post count in the low teens. Most of the new stuff goes to the new package model which is worthless to the 4.7 crowd.

 

Dragonfyre13's approach is very appreciated.

Link to comment

what a great port !...its so good to have guys like you here doing stuff like this, bringing UNraid to the masses, the more user friendly it becomes the more people will look to it for everyday use !!, i know just enough to get things running, in an ideal world it will all be install packages at a click of a button..thank you !

 

agreed...

 

With at least 2 new unMenu packages and post count in the low teens. Most of the new stuff goes to the new package model which is worthless to the 4.7 crowd.

 

Dragonfyre13's approach is very appreciated.

The good news is that once somebody creates an unMENU package, the conversion to a "plugin" model for the advanced graphics is fairly simple.  (the 5.0 "plugin" model was designed based on what was learned in the evolution of the unMENU package..conf file structure.)
Link to comment

Hello.  I tried to install this package, and was successful.  Thanks for the easy setup.  I tried this on windows 7, and the default host is 127.0.0.1.  Problem is, once I turned of windows 7, no one's grabbing the movies. 

 

With add-on on unraid, it will stay up all the time. 

 

My question is, what should be my host?  I left it at 0.0.0.0, but no activity in history.  I have all folders set up properly to the best of my knowledge.  Exactly as how I did it in Windows.  except, this time, I didn't use a map drive, instead I used something like "/mnt/user/Media/nzbs," etc. 

 

I just don't understand why even after restart, run, waiting for a day, nothing happens? 

Link to comment

Sorry but not working for me either.

 

grep can't find the process

Source files are in the package directory

 

The installer copied the contents of the package dir. Guess I need something to unzip the file. I fixed that by manually unzipping and adding to my app directory on the cache drive.  I first configured that directory in unMenu. Now when I "ls" the right files are there. I also chmod 775 MovieGrabber.py since the error log gave permission errors "chmod: cannot access `/mnt/cache/.app/moviegrabber/MovieGrabber.py'"

 

Also got this error "Tower unmenu[3044]: ImportError: No module named MovieGrabber"

 

config directory is empty so no webconfig.ini file

 

when I try to run from command line I get "You need Python 2.7.x installed to run MovieGrabber". Version "python-2.6.4-i486-1.txz" is installed.

 

I'm running unRaid 4.7 with a few addons - apcupsd, python-argparse, jre, Subsonic, Maraschino, Monthly Parity Check, Clean Powerdown and python

 

I also get "Installed, but version is different. Current version=''expected '2.1.5'" from unMenu.

 

Thanks,

DB

Link to comment

I just don't understand why even after restart, run, waiting for a day, nothing happens?

 

Honestly, I'm waiting for the next version to come out before going hot and heavy at this. His logging in current version is abysmal, which leads to things like this (it silently fails...). Best bet, check to make sure your "search" parameters are set properly (try deleting them, see if it loads OK).

 

Also, make sure your logging directory is set OK (try setting it to "/tmp", though that will get blown away when you reboot)

 

Can you explain what your process is again, a bit more step by step? I'm not sure I understand what you mean between windows 7, turning windows 7 off, etc. Here's what I got, lemme know what I have wrong:

  • Installed moviegrabber manually on windows 7, set the host to 127.0.0.1 (localhost)
  • Set folders to mapped drives sitting on unraid server
  • downloads properly

Then:

  • installed unraid package attached to post
  • set everything the same as on the windows 7 box, but instead of mapped drives, pointed to local unraid drives (/mnt/user/Media/*something here*)
  • set host to 0.0.0.0 (that's correct by the way, means "listen on all network interfaces on this box")
  • restarted unraid or moviegrabber on unraid or windows 7?
  • shut down windows 7 box? Perhaps shut down unraid? Not sure where this comes into play, or what's actually happening here. I could just be dense though.
  • Moviegrabber doesn't pull movies properly

 

If that's the case, and I understand correctly, it sounds like you're running into our issue which should be fixed in next version. Basically, even after you have the search parameters right, it's not handling the sqlite connection properly and is throwing an error because of it. Supposed to be fixed this week at some point with the newest version of moviegrabber (2.16), as is the whole "not logging anything out on failure". If there's nothing (including an sqlite error) appearing in your logs, try removing everything from your search parameter field, see if that starts showing stuff in the logs.

Link to comment

Sorry but not working for me either.

Let's see if we can work it out. There's certainly still some kinks, especially since I'm trying to get it working on python 2.6 when it's originally built for python 2.7. I do some awk fun in the install script to get around this, which is also the reason for the argparse requirement.

 

grep can't find the process

Source files are in the package directory

Grep not finding the process I'm assuming is post install, guessing that's because of the second part. if the manual_installation or auto_installation scripts aren't run it won't start out of rc.d, thus the process won't be running. If the source files are sitting in the package directory that means they were unzipped there, so it's getting that far (I'm not sure, but I thought that's how it was supposed to be done. Correct me if I'm wrong, that's what the packages I looked at did)

 

The installer copied the contents of the package dir. Guess I need something to unzip the file. I fixed that by manually unzipping and adding to my app directory on the cache drive.  I first configured that directory in unMenu. Now when I "ls" the right files are there. I also chmod 775 MovieGrabber.py since the error log gave permission errors "chmod: cannot access `/mnt/cache/.app/moviegrabber/MovieGrabber.py'"

The installer actually copies the contents after it modifies them a bit. In addition to adding some things to webconfig.ini (setting to the host and port you specify in the unmenu interface), and after changing minimum version to python 2.6 instead of 2.7, since argparse is available via package.

 

Also got this error "Tower unmenu[3044]: ImportError: No module named MovieGrabber"

This would be the version number stuff. Basically, to get the version number of Moviegrabber, it goes to the installation directory and runs python, importing MovieGrabber (the main moviegrabber python script), and pulls the version number out of there.

 

config directory is empty so no webconfig.ini file

 

when I try to run from command line I get "You need Python 2.7.x installed to run MovieGrabber". Version "python-2.6.4-i486-1.txz" is installed.

This all looks to be because it's not running the auto_installation or manual_installation files, which do some modification there (webconfig.ini and changing min python version to 2.6)

 

I'm running unRaid 4.7 with a few addons - apcupsd, python-argparse, jre, Subsonic, Maraschino, Monthly Parity Check, Clean Powerdown and python

I don't have a 4.7 system to test on, and never used it, but from what I hear it should work fine with your setup. You've got python-argparse in there, which is the main thing (and you're using maraschino, the other pkg I made! :D)

 

I also get "Installed, but version is different. Current version=''expected '2.1.5'" from unMenu.

 

Thanks,

DB

That would be associated with the ImportError message above. It can't import MovieGrabber in the directory you have specified as your install directory.

 

So, to move forward for a solution, can you send:

  • what you have in the package configuration (install directory, port, host)
  • output of: find /boot -maxdepth 3 -iregex ".*moviegrabber.*" -printf "%t - %p\n"
  • run from your installation directory, the output of: find . -maxdepth 1 -iregex "moviegrabber\.py\|.*\.ini" -printf "%t - %p\n"
  • if there is a manual_installation file in the packages directory, the output of running that as well (or run the install via unmenu, and paste the output of the installation from the top of the page)

Link to comment

agreed...

 

With at least 2 new unMenu packages and post count in the low teens. Most of the new stuff goes to the new package model which is worthless to the 4.7 crowd.

 

Dragonfyre13's approach is very appreciated.

 

Thanks! Suppose now is a bad time to mention that I abhor XML, and would rather just write a script? :D

 

Seriously though, glad to be giving back. Already have a handful of packages from others, worth it to give something back if I can. Now to figure out how to build a package dependent upon PHP, a webserver, php-mysql, etc. for these: http://forum.xbmc.org/showthread.php?t=83304 http://forum.xbmc.org/showthread.php?t=106980

Link to comment

Thanks! Suppose now is a bad time to mention that I abhor XML, and would rather just write a script? :D

 

Seriously though, glad to be giving back. Already have a handful of packages from others, worth it to give something back if I can. Now to figure out how to build a package dependent upon PHP, a webserver, php-mysql, etc. for these: http://forum.xbmc.org/showthread.php?t=83304 http://forum.xbmc.org/showthread.php?t=106980

 

Nice! Looking forward to your future scripts.

 

With respect to my failed install..what you posted makes sense. Since the customizations were not applied, the app failed to start even after I manually copied  the installer files. I'll try to run the installer from the shell and see it that fixes things.

 

Since the package is a zip don't I need some utility to decompress? Could that be what's missing?

 

Thanks

Link to comment

ah, yeah it uses unzip. Not sure if that's in the default unraid v5 setup, or something added. For your setup you will need unzip though.

just added unzip and re-ran install directly from unMenu. Worked right away.

 

Thanks

 

Forgot to say

I'm running 4.7. Guess that utility is not part of the default install.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.