FileBot containers

[coppit]

THIS CONTAINER IS NO LONGER SUPPORTED.

PLEASE USE DJOSS' CONTAINER INSTEAD.

(Or Sonarr, which is a more comprehensive solution.)

 

FileBot is an automated file renamer and metadata fetcher.

 

Note: I used to have separate UI and non-UI containers. They have now been merged into one container that supports both methods of use.

 

THE UI IS DISABLED BY DEFAULT. To enable it, edit your filebot.conf, setting RUN_UI=true. Then restart the container.

You can use this container in automated mode, where you configure FileBot, and then drop files into a "watch dir" that will automatically be converted.

 

Or you can use it in UI mode, where you can either use a remote desktop client or a web browser to use the UI.

 

For more information on usage: https://hub.docker.com/r/coppit/filebot/

For source code: https://github.com/coppit/docker-filebot

This container is part of my template repository: https://github.com/coppit/docker-templates

 

Breaking change on June 26, 2016

 

In order to fix the automated FileBot so that it creates files with nobody:users ownership instead of root:root, I had to change a few things. Unfortunately existing users will need to do a few things to migrate their config files and containers to the new image:

 

• Click on your container, then the "Advanced" button in the top right, and add new environment variables: "USER_ID=99", "GROUP_ID=100", and "UMASK=0000"
• Run "chown nobody:users /path/to/config/*"
• When you first run it, it will exit and complain in the logs about a filebot.sh version change. You'll need to edit your filebot.sh, bumping the version number to 2 and changing /root to /files

Edited November 24, 2019 by coppit

[sacretagent]

NO GUI ?

just a task running in the background ?

how you gonna compare things ?

[coppit]

Yeah, I guess it should be filebot-cli or something. It's meant to be no-touch. You could compare the input and output directories, I suppose.

 

I've found UIs in docker containers to be really clunky (web UIs being the exception, of course). If I have some time I'll take a look at a UI, but I wouldn't count on it.

[snoopy86]

Is this going to check in the background the input directory for new files? What happens if torrent isn't completed in the input directory and filebot will want to copy it to new location?

[coppit]

I keep my incomplete files in a separate directory.

 

The change detection only triggers for new directories, files that were open for writing and have been closed, and for files moved into the directory. Opening or writing to a file doesn't trigger it, nor does reading a file. However, some other file in the directory could get written, causing FileBot to run on the whole thing. I don't know if filebot is smart enough to detect when files have opened file handles to them.

 

The safest thing would be to go from incomplete -> complete -> sorted, where complete is FileBot's input, and sorted is its output.

[snoopy86]

Yes, i thought so. Will do the same. Tnx!

[coppit]

I've created a container with a UI. I'm looking for feedback.

 

I've updated the top post with info. It's also available via the "community applications" plugin UI.

[magiin83]

Has anyone been successful with using more intricate rename lines other than the one originally put into the filebot.sh script here?

The original line for movie renaming is:  n.replaceAll(/[\`\u00b4\u2018\u2019\u02bb]/, "'"'"'").replaceAll(/[\u201c\u201d]/, '"'"'""'"'"')'

I'm having an enormously hard time (which may or may not be due to the fact that I'm not completely well versed on linux cli) trying to do anything other than:

 

{n}.{y}/{n.}.{y}.{vf}... etc etc, standard basic renaming.... however, what I'm looking to do is:

 

{n.replaceAll(/[]/, " - ").upperInitial().space('.')}.({y})/{n.replaceAll(/[]/, " - ").upperInitial().space('.')}.{y}.{vf}.{vc}.{ac}{'-CD'+pi}"

 

Essentially all the info after the name of the movie is easily done, but the naming of the movie to make sure no odd characters and anything that woudl throw it off or stop the process is removed.  So something like Naked.Gun.-.33.1/3.(Year)/Naked.Gun.-.33.1/3.(Year).720p.BRRip.x264.MP3.mkv  would be created.

 

If anyone has a moment to help me out with something like this, I'd greatly appreciate it. 

 

[snoopy86]

You should ask this in the filebot forum. There are also plenty of scripts available on that site.

[magiin83]

You should ask this in the filebot forum. There are also plenty of scripts available on that site.

 

I did, and got my answer.  It was an issue with double quotes.

 

 

On the other hand though, I am having an issue that I imagine most other people using the docker are having too... All drives are spun down, file gets downloaded, then passed to filebot... which then spins up the drive it's writing to, essentially bypassing the cache drive.... why is this occurring?  Or why is it sendig a signal to spin up the drive?  Is there a method or command to throw in the script to disable this or is there something in one of the parameters thats doing this and I don't know it?

 

Here's the log on the filebot side:

 

[Jun 18 10:49:05] Detected new directory: Chef.2014.1080p.BluRay.x264.YIFY-NZBgeek

[Jun 18 10:49:11] Input directory stabilized for 5 seconds. Running FileBot.

Locking /root/amc.log

Run script [fn:amc] at [Thu Jun 18 10:49:16 EDT 2015]

Parameter: ut_dir = /input

Parameter: ut_kind = multi

Parameter: music = n

Parameter: artwork = y

Parameter: deleteAfterExtract = y

Parameter: clean = y

Parameter: xbmc = 192.168.1.25

Parameter: excludeList = /config/amc-exclude-list.txt

Parameter: movieFormat = {n.replaceAll(/[?!]/,".-").upperInitial().space(".")}.{y}/\{n.replaceAll(/[?!]/,".-").upperInitial().space(".")}.{y}.{vf}.{vc}.{ac}{'-CD'+pi}

Using excludes: /config/amc-exclude-list.txt (14)

Input: /input/Chef.2014.1080p.BluRay.x264.YIFY-NZBgeek/4cfd5a4f39cce4cef8acbef868856c413770b665.mp4

4cfd5a4f39cce4cef8acbef868856c413770b665.mp4 [series: Chef, movie: Chef (2014)]

Exclude Series: Chef

Group: [tvs:null, mov:chef 2014, anime:null] => [4cfd5a4f39cce4cef8acbef868856c413770b665.mp4]

Rename movies using [TheMovieDB]

Auto-detect movie from context: [/input/Chef.2014.1080p.BluRay.x264.YIFY-NZBgeek/4cfd5a4f39cce4cef8acbef868856c413770b665.mp4]

Stripping invalid characters from new path: Chef.2014/\Chef.2014.1080p.x264.AAC

[M O V E] Rename [/input/Chef.2014.1080p.BluRay.x264.YIFY-NZBgeek/4cfd5a4f39cce4cef8acbef868856c413770b665.mp4] to [/output/Chef.2014/Chef.2014.1080p.x264.AAC.mp4]

Processed 1 files

Fetching movie artwork for [Chef (2014)] to [/output/Chef.2014]

Fetching /output/Chef.2014/poster.jpg => {category: posters, width: 1400, height: 2100, language: en, url: http://image.tmdb.org/t/p/original/jc2BCqYtoyUM6rWdtW6KvJjL2Dq.jpg}

Fetching /output/Chef.2014/fanart.jpg => {category: backdrops, width: 1920, height: 1080, language: null, url: http://image.tmdb.org/t/p/original/sbH9aJV7fqhRaazCSWuOwNDIf1m.jpg}

Fanart not found: /output/Chef.2014/clearart.png / movieart

Fanart not found: /output/Chef.2014/logo.png / movielogo

Fetching /output/Chef.2014/disc.png => {type=moviedisc, id=82643, url=http://assets.fanart.tv/fanart/movies/212778/moviedisc/chef-5412e100d0e94.png, lang=en, likes=4, disc_type=bluray}

Notify XBMC: 192.168.1.25

Clean clutter files and empty folders

Delete /input/Chef.2014.1080p.BluRay.x264.YIFY-NZBgeek

Delete /input

Done ?(??)?

 

Command shown on unraid log that displays:

Jun 18 10:29:04 MediaNAS kernel: mdcmd (53): spindown 2

Jun 18 10:49:53 MediaNAS emhttp: /usr/bin/docker logs --tail=350 -f FileBot 2>&1

Jun 18 10:50:38 MediaNAS emhttp: /usr/bin/tail -n 42 -f /var/log/syslog 2>&1

Jun 18 10:50:55 MediaNAS emhttp: /usr/bin/docker logs --tail=350 -f FileBot 2>&1

[trurl]

On the other hand though, I am having an issue that I imagine most other people using the docker are having too... All drives are spun down, file gets downloaded, then passed to filebot... which then spins up the drive it's writing to, essentially bypassing the cache drive.... why is this occurring?  Or why is it sendig a signal to spin up the drive?  Is there a method or command to throw in the script to disable this or is there something in one of the parameters thats doing this and I don't know it?

Whether the cache drive gets used or when a drive is spun up has nothing to do with filebot or any docker for that matter.

 

If you are writing directly to one of the disks, then cache doesn't get used. Only user shares are cached. unRAID spins up drives when they are read or written.

[magiin83]

On the other hand though, I am having an issue that I imagine most other people using the docker are having too... All drives are spun down, file gets downloaded, then passed to filebot... which then spins up the drive it's writing to, essentially bypassing the cache drive.... why is this occurring?  Or why is it sendig a signal to spin up the drive?  Is there a method or command to throw in the script to disable this or is there something in one of the parameters thats doing this and I don't know it?

Whether the cache drive gets used or when a drive is spun up has nothing to do with filebot or any docker for that matter.

 

If you are writing directly to one of the disks, then cache doesn't get used. Only user shares are cached. unRAID spins up drives when they are read or written.

 

That much I understand, however, if for example: I have nzbget download something, process it, then pass it to the array, using cache drive, with settings so all downloads are technically moved later, why does it not send the signal to spin up the drive, where as with filebot, this is happening? Is it because it's download the metadata separately and is causing this?

[trurl]

On the other hand though, I am having an issue that I imagine most other people using the docker are having too... All drives are spun down, file gets downloaded, then passed to filebot... which then spins up the drive it's writing to, essentially bypassing the cache drive.... why is this occurring?  Or why is it sendig a signal to spin up the drive?  Is there a method or command to throw in the script to disable this or is there something in one of the parameters thats doing this and I don't know it?

Whether the cache drive gets used or when a drive is spun up has nothing to do with filebot or any docker for that matter.

 

If you are writing directly to one of the disks, then cache doesn't get used. Only user shares are cached. unRAID spins up drives when they are read or written.

 

That much I understand, however, if for example: I have nzbget download something, process it, then pass it to the array, using cache drive, with settings so all downloads are technically moved later, why does it not send the signal to spin up the drive, where as with filebot, this is happening? Is it because it's download the metadata separately and is causing this?

What are your volume mappings?

[magiin83]

On the other hand though, I am having an issue that I imagine most other people using the docker are having too... All drives are spun down, file gets downloaded, then passed to filebot... which then spins up the drive it's writing to, essentially bypassing the cache drive.... why is this occurring?  Or why is it sendig a signal to spin up the drive?  Is there a method or command to throw in the script to disable this or is there something in one of the parameters thats doing this and I don't know it?

Whether the cache drive gets used or when a drive is spun up has nothing to do with filebot or any docker for that matter.

 

If you are writing directly to one of the disks, then cache doesn't get used. Only user shares are cached. unRAID spins up drives when they are read or written.

 

That much I understand, however, if for example: I have nzbget download something, process it, then pass it to the array, using cache drive, with settings so all downloads are technically moved later, why does it not send the signal to spin up the drive, where as with filebot, this is happening? Is it because it's download the metadata separately and is causing this?

What are your volume mappings?

 

See attached, or:

 

Filebot:

/input /mnt/cache/appdata/Downloads/dst/Movies/

/output /mnt/user/Movies

/config /mnt/cache/appdata/filebot

 

NzbGet:

/data /mnt/user

/config /mnt/cache/appdata/nzbget/config

/downloads /mnt/cache/appdata/Downloads/

 

So with Nzbget and videosort script - downloads, pases to script, renames, finishes, then sends to /data... drive does not spin.

With Nzbget and filebot - downloads to /downloads, filebot picks up from there in /input, then /outputs with downloading metadata

[trurl]

...

Filebot:

/input /mnt/cache/appdata/Downloads/dst/Movies/

/output /mnt/user/Movies

/config /mnt/cache/appdata/filebot

 

NzbGet:

/data /mnt/user

/config /mnt/cache/appdata/nzbget/config

/downloads /mnt/cache/appdata/Downloads/

 

So with Nzbget and videosort script - downloads, pases to script, renames, finishes, then sends to /data... drive does not spin.

With Nzbget and filebot - downloads to /downloads, filebot picks up from there in /input, then /outputs with downloading metadata

I use NZBGet but not any of these other things so excuse me if I miss this obvious.

 

You say videosort script sends to /data, which is mapped to /mnt/user. I assume you mean it is sending it to a subfolder of /data.

 

Filebot's /output is the Movies user share. Is that share configured to use the cache disk? Maybe something about that share's configuration is making it bypass cache. Post a screenshot of the Movies share page.

[magiin83]

...

Filebot:

/input /mnt/cache/appdata/Downloads/dst/Movies/

/output /mnt/user/Movies

/config /mnt/cache/appdata/filebot

 

NzbGet:

/data /mnt/user

/config /mnt/cache/appdata/nzbget/config

/downloads /mnt/cache/appdata/Downloads/

 

So with Nzbget and videosort script - downloads, pases to script, renames, finishes, then sends to /data... drive does not spin.

With Nzbget and filebot - downloads to /downloads, filebot picks up from there in /input, then /outputs with downloading metadata

I use NZBGet but not any of these other things so excuse me if I miss this obvious.

 

You say videosort script sends to /data, which is mapped to /mnt/user. I assume you mean it is sending it to a subfolder of /data.

 

Filebot's /output is the Movies user share. Is that share configured to use the cache disk? Maybe something about that share's configuration is making it bypass cache. Post a screenshot of the Movies share page.

 

Correct, videosort is mapped in a way that it sends to a subfolder (Movies or Television)... so if I were to ever stop using sonarr, it would rename XXXXX, if it's a movie category or tv category, then sends to the appropriate directory on the array in /data (/mnt/user/________)

 

The movies user share as far as I know, is setting correctly, but you may have a better answer about that then I do.  It was the next place I was going to check.

 

See attached.  I do really appreciate the help!  It's driving me crazy what's causing this.

[trurl]

Share page look OK. How much free space is on the cache drive?

[magiin83]

Share page look OK. How much free space is on the cache drive?

 

120gigs... lol.  It's not an issue of space.  I just don't know what could be causing this, and the only thing I can notice (at least based off the log) that would be causing it is FileBot, which I know, also makes no sense.  But it doesn't occur with just nzbget.

 

 

To summarize in case ayone else is looking at this page and gets confused what this post is about (or has any other input which I'd greatly appreciate any help to resolve):

 

-------------

 

I am having an issue that I imagine most other people using the docker are having too... All drives are spun down, file gets downloaded, then passed to filebot... which then spins up the drive it's writing to, essentially bypassing the cache drive.... why is this occurring?  Or why is it sendig a signal to spin up the drive?  Is there a method or command to throw in the script to disable this or is there something in one of the parameters thats doing this and I don't know it?

 

Here's the log on the filebot side:

 

[Jun 18 10:49:05] Detected new directory: Chef.2014.1080p.BluRay.x264.YIFY-NZBgeek

[Jun 18 10:49:11] Input directory stabilized for 5 seconds. Running FileBot.

Locking /root/amc.log

Run script [fn:amc] at [Thu Jun 18 10:49:16 EDT 2015]

Parameter: ut_dir = /input

Parameter: ut_kind = multi

Parameter: music = n

Parameter: artwork = y

Parameter: deleteAfterExtract = y

Parameter: clean = y

Parameter: xbmc = 192.168.1.25

Parameter: excludeList = /config/amc-exclude-list.txt

Parameter: movieFormat = {n.replaceAll(/[?!]/,".-").upperInitial().space(".")}.{y}/\{n.replaceAll(/[?!]/,".-").upperInitial().space(".")}.{y}.{vf}.{vc}.{ac}{'-CD'+pi}

Using excludes: /config/amc-exclude-list.txt (14)

Input: /input/Chef.2014.1080p.BluRay.x264.YIFY-NZBgeek/4cfd5a4f39cce4cef8acbef868856c413770b665.mp4

4cfd5a4f39cce4cef8acbef868856c413770b665.mp4 [series: Chef, movie: Chef (2014)]

Exclude Series: Chef

Group: [tvs:null, mov:chef 2014, anime:null] => [4cfd5a4f39cce4cef8acbef868856c413770b665.mp4]

Rename movies using [TheMovieDB]

Auto-detect movie from context: [/input/Chef.2014.1080p.BluRay.x264.YIFY-NZBgeek/4cfd5a4f39cce4cef8acbef868856c413770b665.mp4]

Stripping invalid characters from new path: Chef.2014/\Chef.2014.1080p.x264.AAC

[M O V E] Rename [/input/Chef.2014.1080p.BluRay.x264.YIFY-NZBgeek/4cfd5a4f39cce4cef8acbef868856c413770b665.mp4] to [/output/Chef.2014/Chef.2014.1080p.x264.AAC.mp4]

Processed 1 files

Fetching movie artwork for [Chef (2014)] to [/output/Chef.2014]

Fetching /output/Chef.2014/poster.jpg => {category: posters, width: 1400, height: 2100, language: en, url: http://image.tmdb.org/t/p/original/jc2BCqYtoyUM6rWdtW6KvJjL2Dq.jpg}

Fetching /output/Chef.2014/fanart.jpg => {category: backdrops, width: 1920, height: 1080, language: null, url: http://image.tmdb.org/t/p/original/sbH9aJV7fqhRaazCSWuOwNDIf1m.jpg}

Fanart not found: /output/Chef.2014/clearart.png / movieart

Fanart not found: /output/Chef.2014/logo.png / movielogo

Fetching /output/Chef.2014/disc.png => {type=moviedisc, id=82643, url=http://assets.fanart.tv/fanart/movies/212778/moviedisc/chef-5412e100d0e94.png, lang=en, likes=4, disc_type=bluray}

Notify XBMC: 192.168.1.25

Clean clutter files and empty folders

Delete /input/Chef.2014.1080p.BluRay.x264.YIFY-NZBgeek

Delete /input

Done ?(??)?

 

Command shown on unraid log that displays:

Jun 18 10:29:04 MediaNAS kernel: mdcmd (53): spindown 2

Jun 18 10:49:53 MediaNAS emhttp: /usr/bin/docker logs --tail=350 -f FileBot 2>&1

Jun 18 10:50:38 MediaNAS emhttp: /usr/bin/tail -n 42 -f /var/log/syslog 2>&1

Jun 18 10:50:55 MediaNAS emhttp: /usr/bin/docker logs --tail=350 -f FileBot 2>&1

[trurl]

Something funny with your user shares? From console or telnet, what do you get with these?

ls -lah /mnt/user

ls -lah /mnt/user0

ls -lah /mnt/cache

[magiin83]

Something funny with your user shares? From console or telnet, what do you get with these?

ls -lah /mnt/user

 

root@MediaNAS:~# ls -lah /mnt/user

total 21K

drwxrwxrwx 1 nobody users  26 Jun 18 10:49 ./

drwxr-xr-x 7 root  root  140 Jun 17 08:40 ../

drwxrwxrwx 1 nobody users  112 Feb 21 11:39 Backup\ Storage/

drwxrwxrwx 1 nobody users 1.2K May 10 03:40 Exercise/

drwxrwxrwx 1 nobody users  18 Jun 18 11:33 Movies/

drwxrwxrwx 1 nobody users 3.9K Jun 18 10:13 Television/

drwxrwxrwx 1 nobody users  76 Jun 17 14:56 appdata/

 

ls -lah /mnt/user0

 

root@MediaNAS:~# ls -lah /mnt/user0

total 26K

drwxrwxrwx 1 nobody users  192 Jun 18 10:12 ./

drwxr-xr-x 7 root  root  140 Jun 17 08:40 ../

drwxrwxrwx 1 nobody users  112 Feb 21 11:39 Backup\ Storage/

drwxrwxrwx 1 nobody users 1.2K May 10 03:40 Exercise/

drwxrwxrwx 1 nobody users  22K Jun 18 10:12 Movies/

drwxrwxrwx 1 nobody users 3.9K Jun 18 10:13 Television/

 

ls -lah /mnt/cache

 

root@MediaNAS:~# ls -lah /mnt/cache

total 16K

drwxrwxrwx 1 nobody users  26 Jun 18 10:49 ./

drwxr-xr-x 7 root  root  140 Jun 17 08:40 ../

drwxrwxrwx 1 nobody users  18 Jun 18 11:33 Movies/

drwxrwxrwx 1 nobody users  76 Jun 17 14:56 appdata/

 

[magiin83]

Ok, so after some extensive testing... there is something within filebot absolutely sending a signal to spin up the drive it's about to write the files downloaded and processed by filebot to.

It's not xbmc notifications, it's not the moving of the files to the array.... it's the metadata being written when grabbing artwork and so on.

 

As you can see in this image take from command line, permissions are a problem, since the download is done by nzbget (nobody) and filebot writes as root, and also seems to be writing DIRECTLY to the array, rather than using the cache drive.

 

[magiin83]

See above for recent issues with this.

Coppit, or anyone else, do you experience the same issues?  Drive spins exactly as metadata starts to grab, unlike how it should be doing like with the movie, creating a link on the cache drive

[coppit]

It seems like the workaround is simple: Specify the output dir within your cache drive, and let mover handle moving it later.

 

Of course FileBot is going to write to the output dir, but from the docker container's perspective it can't control what unraid does with that write, or even that unraid exists. This seems like a general unraid<->docker issue that's not related to filebot. To confirm, you could modify your other containers to also use /mnt/user/... and I bet you'll see the same spin-up behavior.

 

I wonder for example if Docker's volume mapping somehow bypasses or interferes with unraid's cache drive implementation?

[magiin83]

It seems like the workaround is simple: Specify the output dir within your cache drive, and let mover handle moving it later.

 

Of course FileBot is going to write to the output dir, but from the docker container's perspective it can't control what unraid does with that write, or even that unraid exists. This seems like a general unraid<->docker issue that's not related to filebot. To confirm, you could modify your other containers to also use /mnt/user/... and I bet you'll see the same spin-up behavior.

 

I wonder for example if Docker's volume mapping somehow bypasses or interferes with unraid's cache drive implementation?

 

Every other docker I have that points to the array (ie Nzbget or couchpotato or sonarr) all don't have the issue of spinning up the drive.

I did also try pointing filebot to /mnt/cache/Movies rather than to the array of the same folder, but the same thing actually occurred, drive spun up. 

I'm not blaming filebot or even this docker for that, a random spin up could even mean that the drive itself on the spindown delay needs to be reinforced again.... just saying, there's something odd about it when I don't have the issue with any other docker handling files that go to the array.

[coppit]

Does NzbGet write to its /data directory (which you mapped to /mnt/user)? And if so, are the files/directories that it writes to on some non-cache drive? What happens if you configure FileBot's output dir to be /mnt/user?

 

When I get a chance I'll do some experiments, but I don't know how filebot could be somehow telling unraid to spin up the drives, or what I could do to prevent that. The only thing I could imagine doing is filing a bug against unraid.