FileBot containers


Recommended Posts

Or why is it sendig a signal to spin up the drive?  Is there a method or command to throw in the script to disable this or is there something in one of the parameters thats doing this and I don't know it?

 

There is no "signal" to spin up a drive. A drive is spun up based on a read or write to the drive that cannot be satisfied by cached data in memory.

 

The way the "spin up" drive works is that it reads a random sector on the drive.

 

So whatever you are doing is causing a disk access.

Link to comment

Ok, so I'm at the point where this has spiraled out of control.

Just so we're all on the same page: I upgraded to 6.0 a few days ago, that is literally all I've changed in terms of settings for unRAID.  Cache dirs I applied a new user defined setting (-U 0) and thats it.  Dockers have changed - filebot on and off.

 

So right now, literally anything I do in terms of using sonarr, or CP, will spin up the drive it's downloading to.

I havent' changed any parameters or volume mappings in months, or probably since I got my cache drive even longer than that.

I realize this isn't really the thread to be posting this in, but I have absolutely no clue what's causing this or why it's all of a sudden started.

Clearly it's not filebot anymore.

 

 

These are my mappings:

CouchPotato

/downloads /mnt/cache/appdata/Downloads/dst/Movies/

/movies /mnt/user/Movies/

/config /mnt/cache/appdata/couchpotato

 

NZBGet

/data /mnt/user

/config /mnt/cache/appdata/nzbget/config

/downloads /mnt/cache/appdata/Downloads/

 

PlexMediaServer

/tv /mnt/user/Television/

/config /mnt/cache/appdata/plex

/movies /mnt/user/Movies/

 

Sonarr /config /mnt/cache/appdata/Sonarr

/downloads /mnt/cache/appdata/Downloads/dst/TV

/tv /mnt/user/Television/

 

All are up to date and updated dockers.

 

My cache_dirs settings are in the image attached.

I've had inotifywait -mr -e access -e modify /mnt/disk2  monitoring the drive as things light up, bu tobviously because cache_dirs is enabled, it's an enormous amount of info getting thrown at it.  Also, with it enabled, it seems like ANY change sent to the drive (being downloaded, etc) will spin the drive.

 

Also, any time I try to access even just general /Movies list in shell or over windows samba, the drives will spin.  I have no depth limit used.  Everything should have been cached, so why is it acting like it's not?

 

With cache_dirs off, this doesn't occur.  I can browse shares, etc. and drives don't spin. 

Doesn't this sound like the exact opposite of what should be happening?

Yes I should probably post this in the cache_dirs thread, and I plan to, but since this back and forth with you guys helping has been going on, I thought I'd update here as well.... this is racking my brain way too much.

CacheDirs.jpg.6fce72d83aed1ab2cb9aeed426dcf35c.jpg

Link to comment

I can't remove/edit any files or folders that filebot has created since it runs as root:root, is it possible to change this?

 

Doh. Yeah, I can fix that. I set up a cron job to run newperms on my cache drive, because transmission does this too. So I didn't notice that these containers have this problem.

Link to comment

I can't remove/edit any files or folders that filebot has created since it runs as root:root, is it possible to change this?

 

Okay, it's fixed. But you need to do three things:

  • Click on your container, then the "Advanced" button in the top right, and add a new environment variable called "UGID" with value "99:100"
  • Run "chown nobody:users /path/to/config/*"
  • When you first run it, it will exit and complain in the logs about a filebot.sh version change. You'll need to edit your filebot.sh, bumping the version number to 2 and changing /root to /files

 

People who create new FileBot containers from my template repository won't have any problems, but existing containers will need a manual update.

Link to comment

Hm.. what user do you have that share mounted as? What were you trying to do, rename the file or something? Can you telnet/ssh into your server, and do:

 

ls -l /path/to/that/file

id -ng mount_username

 

For example, I mounted the share using user "privacy" which has group "users". FileBot creates the file for me with owner "nobody", but group "users". The file permissions are -rw-rw-rw, where the second rw means that group "users" can read and write the file. Since "privacy" is in that group, I can rename the file just fine.

 

If that all looks good, maybe try checking the permissions on the parent dir:

 

ls -ld /path/to/that

Link to comment

Thinking about this some more, the umask might have been setting the group permissions so that the file wasn't writable. (And my newperms automated runner was "fixing" it, which is why I didn't see this.)

 

I've updated the container again with umask support. You'll need to update the environment variables again, setting 3 of them instead of the one UGID one before: "USER_ID=99", "GROUP_ID=100", and "UMASK=0000".

 

Hopefully that fixes it for you. It looks to me that FileBot creates the files with 0664 permissions.

Link to comment

Alright, I updated with the 3 new UGID values. Now I can move/rename/delete the file, even though it shows root:root. I'm guessing this is because of 0000?

 

root@Tower:/mnt/cache/tv/Hannibal/Season 03# ls -l
total 960489
-rw-rw-rw- 1 root root 982579115 Jun 26 06:40 Hannibal\ -\ 3x04\ 720p\ -\ Aperitivo.mkv

 

root@Tower:~# id -u nobody
99
root@Tower:~# id -g nobody
100

 

7161144592d6a62cb049806b457683b6.png

 

Now I only have one more problem. I run uTorrent 2.2.1 on my windows machine and it downloads to /cache/.apps/utorrent/temp and moves completed files to /utorrent/finished. Filebot doesn't pick up any folders that get dropped off into /finished, only files.

So if it's /finished/hannibal.3x04.mkv, then it gets processed. But if it's /finished/FOLDER/hannibal.3x04.mkv, then nothing happens and watcher just stays on "Waiting for changes...". Doesn't even work if I drag and drop a folder into /finished.

 

However, if I run ruTorrent docker and point Filebot to /rutorrent/finished, then everything that ruTorrent moves to that directory gets processed completely fine.

 

I know I could just move to ruTorrent, but I like uT's queueing and RSS system better. Any idea why this is happening?

Link to comment

Edit: Fixed link

 

Alright, I updated with the 3 new UGID values.

 

Whoops! Is should look like this: https://www.dropbox.com/s/agued77qhqhdaii/cap.png?dl=0

 

Now I can move/rename/delete the file, even though it shows root:root. I'm guessing this is because of 0000?

 

Actually, all your settings were ignored. The container has a default umask of 0000, which is why it happened to work even though the user/group was still root/root (again the default).

 

Filebot doesn't pick up any folders that get dropped off into /finished, only files.

 

I didn't have it searching for changes recursively. So one way the above could happen is:

 

1) Subdir gets created

2) FileBot sees it, and processes it

3) File gets put in subdir

4) FileBot misses it.

 

I've enabled recursive monitoring, so you should be good now.

Link to comment

Your dropbox link doesn't work.

 

It still doesn't detect folders that uTorrent moves or folders that I drag and drop into /finished. And it does no processing to make it miss it, just sits on "Waiting for changes".

 

The only thing that the recursive update has changed is that now you get a double process within the same folder after unraring, because the unraring process creates a new folder. And Filebot has already moved the unrared file from the new folder, so the second processing ends with "No files selected for processing Failure (°_°)" . It worked fine as it was before. (I tested this out with ruTorrent)

 

ruTorrent docker still works fine and triggers the watcher with either files or folders.

 

I've tried:

 

Make uTorrent move completed folders to the ruTorrent /finished directory, the one that works with everything if ruTorrent moves the folder. Didn't trigger processing.

Drag and drop folders into watched directory through Windows Explorer. Didn't trigger processing.

Use MC to move a create a folder and move it into the watched directory. Didn't trigger processing.

 

Use MC to create a folder in the watched directory. That did trigger processing.

Use Windows Explorer to create a folder in the watch directory. That did trigger processing.

 

So somehow the only process that moves folders and triggers it is the ruTorrent "AutoMove". Nothing else can move a folder and trigger it.

 

Maybe the simplest solution is just to move to ruTorrent  ;D

Link to comment

Used the filebot RDP container last night and it's great, it's a program I only use from time to time but having a docker to use it is very convenient.

 

Only issue is, and this might be a filebot limitation rather than your docker, is there any way to set data persistence, so that my preferences are saved

 

ie: format, input folder, output folder..

 

It's no biggy but would be a little more convenient for the end luser.  :D

Link to comment

Your dropbox link doesn't work.

 

Fixed. Link is https://www.dropbox.com/s/agued77qhqhdaii/cap.png?dl=0

 

It still doesn't detect folders that uTorrent moves or folders that I drag and drop into /finished. And it does no processing to make it miss it, just sits on "Waiting for changes".

 

I don't know how much time you want to spend debugging this, but I've updated the container to drop a filebot.conf file into the config directory when it starts. You can set DEBUG=1 in there to have it dump the raw events that it gets from inotifywatch. They're very confusing (and sometimes very surprising and enlightening), but might help you understand what's going on. Perhaps I need to add a new event to my list of "things that should trigger a FileBot run".

 

The only thing that the recursive update has changed is that now you get a double process within the same folder after unraring, because the unraring process creates a new folder. And Filebot has already moved the unrared file from the new folder, so the second processing ends with "No files selected for processing Failure (°_°)" . It worked fine as it was before. (I tested this out with ruTorrent)

 

This can happen depending on how long things take. In the config file you'll see a few different settings for controlling how eagerly we run FileBot. For example, I doubled the "settle time" from 5 seconds to 10 seconds. That means that the directory needs to be idle for more than 10 seconds before we'll run FileBot.

Link to comment

Only issue is, and this might be a filebot limitation rather than your docker, is there any way to set data persistence, so that my preferences are saved

 

I think it might be a FileBot problem with the Linux version. Even if I simply stop and start the FileBot app without restarting the container, the app doesn't remember the input directory. IFAICT, the only thing it remembers is the main panel selection. :(

Link to comment

I think it might be a FileBot problem with the Linux version. Even if I simply stop and start the FileBot app without restarting the container, the app doesn't remember the input directory. IFAICT, the only thing it remembers is the main panel selection. :(

 

But do let me know if you see any important setting that survives a stop/start of the container but not a delete/recreate. I can fix those.

 

Will do mate, thanks for letting me know, like I said it's not a big deal at all, it's just I can never find the options setting when I need to do it.  :D

Link to comment
  • 4 weeks later...

I'm pretty new to using filebot, love how much functionality it has, but am having some trouble getting things to work how I want.

 

My folders are set up like this:

 

input: /mnt/user/Downloads (folder is cache only)

output: /mnt/cache/

 

Filebot simply renames and moves everything to the root of the cache drive so that mover can work its magic. Everything is moved within a single drive, so I would expect the renames to happen instantly, but for some reason the files get rewritten to the disk, taking a fair amount of time. Does anyone know what I can do?

Link to comment

I'm pretty new to using filebot, love how much functionality it has, but am having some trouble getting things to work how I want.

 

My folders are set up like this:

 

input: /mnt/user/Downloads (folder is cache only)

output: /mnt/cache/

 

Filebot simply renames and moves everything to the root of the cache drive so that mover can work its magic. Everything is moved within a single drive, so I would expect the renames to happen instantly, but for some reason the files get rewritten to the disk, taking a fair amount of time. Does anyone know what I can do?

Regardless of where the files are physically stored, any operating system (Linux, windows, OS X, etc) looks at where it *thinks* they are and then decides to read / write the file to a new location, or to merely rename it to the location.

 

In this case, the mount points are different for the input and the output.  Linux (and windows / OS X) doesn't realize that the source and destination are actually on the same physical hard drive.  Hence why it does the read / write instead of a rename.  Its just how all operating systems operate.  I'm not familiar with filebot, so I can't tell you if you set the same mount point for your input and your output if it will work (ie: use different folders within the mount points)

 

Link to comment

I'm pretty new to using filebot, love how much functionality it has, but am having some trouble getting things to work how I want.

 

My folders are set up like this:

 

input: /mnt/user/Downloads (folder is cache only)

output: /mnt/cache/

 

Filebot simply renames and moves everything to the root of the cache drive so that mover can work its magic. Everything is moved within a single drive, so I would expect the renames to happen instantly, but for some reason the files get rewritten to the disk, taking a fair amount of time. Does anyone know what I can do?

Regardless of where the files are physically stored, any operating system (Linux, windows, OS X, etc) looks at where it *thinks* they are and then decides to read / write the file to a new location, or to merely rename it to the location.

 

In this case, the mount points are different for the input and the output.  Linux (and windows / OS X) doesn't realize that the source and destination are actually on the same physical hard drive.  Hence why it does the read / write instead of a rename.  Its just how all operating systems operate.  I'm not familiar with filebot, so I can't tell you if you set the same mount point for your input and your output if it will work (ie: use different folders within the mount points)

 

Thanks, that helped me solve my issue. I just stopped using the input folder, grabbed files from output and renamed to output. No more waiting.

Link to comment
  • 1 month later...
  • 2 weeks later...

Thanks a lot coppit - maybe someone else has any hint or advise.

 

Another thing that is filling my log:

touch: cannot touch ‘/var/lib/tomcat7/logs/catalina.out’: No such file or directory
./run: line 8: /usr/lib/jvm/java-7-openjdk-amd64/bin/java: No such file or directory
touch: cannot touch ‘/var/lib/tomcat7/logs/catalina.out’: No such file or directory
./run: line 8: /usr/lib/jvm/java-7-openjdk-amd64/bin/java: No such file or directory
touch: cannot touch ‘/var/lib/tomcat7/logs/catalina.out’: No such file or directory
./run: line 8: /usr/lib/jvm/java-7-openjdk-amd64/bin/java: No such file or directory

Link to comment
  • 2 weeks later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.