[SUPPORT] reven - FileFlows


reven

Recommended Posts

The app won't start for me.  Looking through the logs, the exception below is the only error being generated when trying to start.  I didn't setup the app to use my gpu as I am intersted in the moving functions.

 

Unhandled exception. code = Error (1), message = System.Data.SQLite.SQLiteException (0x800007BF): SQL logic error
no such table: DbObject
at System.Data.SQLite.SQLite3.Prepare(SQLiteConnection cnn, String strSql, SQLiteStatement previous, UInt32 timeoutMS, String& strRemain)

 

I wish I could provide my perspective, but still learning and don't have the knowledge to speak with confidence.

 

 

Link to comment

try again now.

 

There was an issue with the database create script I missed in testing.  I was experiementing with using mysql for better perfomance vs sqlite, but went back to sqlite but forgot to update the create script.  

 

but long story short, update the docker, should be good to go now.

 

I'm going to work on a youtube video showing how to use it.  But I covered most things on the wiki on github. 

https://github.com/revenz/FileFlows/wiki

Edited by reven
Link to comment

I originally built it as an alternative to unmaniac and tdarr.  But realised it could do much more, just need the plugins to be created for it.  Eg fixing id3 tags on audio files.  sending an email when a file is found, or when something happens to a file.   Sending a message to a alexa to play something when something with a file happens eg "New episode of Doctor Who ready to play".   Calling into node-red and using those plugins to do even more.

 

But what I'm primarily using it for now is to just monitor a downloads directory and ensure all videos downloaded are in h265 with AC3 audio, english as main language, and cropping black bars if detected, then moving the file into a converted folder that sonarr/radarr will pickup.   So I'm using it as a middle man, 
sonarr/radarr sends request to sabnzbd -> sabnzdb downloads file -> fileflows converts files moves file to folder -> sonarr/radarr see new file in folder and renames it/moves it -> they tell plex

 

 

Link to comment
7 hours ago, reven said:

do a force update on the docker.   this may have been a bad build..
also what browser are you using?   if not chrome, try chrome.

and can you post the version number of the app.  shown in the bottom left corner of the menu bar.

Did a force update, and that seems to have fixed it.

 

Thanks!

  • Like 1
Link to comment

Version: 0.0.1.68

- added variables to inputfile

- fixed issue wwith filebrowser using wasm directory instead of server

- added support for backspace to remove node from flow page

- styled scrollbar in firefox

- standardized file/folder variables

- updated library worker to ignore resulting files after being processed

- filebrowser now sorts items by name

- filebrowser will default to /media if not on windows and if that directory will exist if no path is set


 

Version: 0.0.1.63

- Added "Renamer" node

- Added "Pattern Match" node, this is a regular expression match node

- Added "Variables" to flows.  Can be used in renamer with { character and in the Function node with the Variable object

- Added "Movie Lookup" node to a new MetaNode plugin

- Fixed issue with output connections were not being deleted

- added validation to editor

- increased hot zone of the input/output nodes to make them easier to connect

- change cursor of input/outputs so easier to tell when the connection will be succesful

- made some files use no cache to make upgrades easier

- made input-array add the value (if valid) to the list on blur.  eg "Extensions" if you dont press enter after entering one, it will now auto add it to list

 

New version: 0.0.1.56

- fixed issue with file extension node not working

- fixed issue with Code not working

- added "Pattern" node which lets you test against a regular expression

- started adding unit tests to plugins

Link to comment
On 11/24/2021 at 1:37 PM, remati said:

Ideas for a plugins:

1) Rename a processed file if "x264" or "h264" is detected in the filename and change it to "h265" after conversion.

2) Add video resolution to the filename.

 

Nice container by the way! Looking forward to the updates.

 

Version: 0.0.1.70

- Fixed issue with new Flows not having default "File Input"

- Added InputKeyValue type to editor

- BasicNodes: Added Pattern Replacer node which lets you do string/regular expression replacements in the filename



This adds the renamer you requested in 1)

can be regular expressions, or just simple string replacements.  Executed in the order given.   so this example replace Something.s03e05.mkv to Something.03x05.mkv then Something.3x05.mkv then to Something 3x05.mkv then Something - 3x05.mkv
image.thumb.png.24e06cc93d8586509ba6e0468eeeaccb.png

  • Thanks 1
Link to comment

Nice work on the renamer plugin!

 

I see in your dockerfile you are using the jrottenberg/ffmpeg:4.1-nvidia container for your base. I think it's about 1GB in size. For those of us that do not use an Nvidia GPU for encoding but instead use Intel's QuickSync would it be possible to use jrottenberg/ffmpeg:4.1-vaapi instead?

Link to comment
7 minutes ago, remati said:

Nice work on the renamer plugin!

 

I see in your dockerfile you are using the jrottenberg/ffmpeg:4.1-nvidia container for your base. I think it's about 1GB in size. For those of us that do not use an Nvidia GPU for encoding but instead use Intel's QuickSync would it be possible to use jrottenberg/ffmpeg:4.1-vaapi instead?

This is the first docker app I've ever done, so I'm really no expert at it.  I used the first thing that I got working which was that jrottenberg base.  I don't think the vaapi has nvidia hw support (correct me if I'm wrong).

So I'll have to figure out how to make a docker container that has nvidia/intel/amd? support.

  • Thanks 1
Link to comment

Version: 0.0.1.87

- Added file size monitor to show file size before/after processing

 

Version: 0.0.1.85

- Fixed issue with newly created directories having wrong permissions on them

- Fixed issue with Audio Normalization in Video_H265_AC3 node not working

- Fixed issue with library files showing as successful when the flow failed

 

Version: 0.0.1.81

- Attempting to fix an issue when a file is still being moved/copied to location and its picked up by the library worker.

- Attempting to fix new directory permissions issue

 

Version: 0.0.1.79

- Made it work in a "Demo" mode.   Which is now available on fileflows.com website.

- Fixed Safari browser issue (hopefully, havent had time to test this, was a minor bug)

 

Version: 0.0.1.70

- Fixed issue with new Flows not having default "File Input"

- Added InputKeyValue type to editor

- BasicNodes: Added Pattern Replacer node which lets you do string/regular expression replacements in the filename


image.thumb.png.d753d2f1ac8802d70f6b8087c3aa88dc.png

Link to comment

Version: 0.0.1.105

- Fixed pages to work better on mobile devices.

 

Version: 0.0.1.104

- Added "Priority" to Libraries, Lowest, Low, Normal, High, Highest.   So you can process your entire library at a lower priority and keep the newer stuff automatically processed first.

- Fixed blazor issue with portlets and styled the blazor error at the bottom of the screen (with link to how to report this issue)

- Fixed styling of menu but on mobile

 

Version: 0.0.1.102

- BIG refactor targeting performance improvements

- MUCH FASTER NOW with large lists of files (processing hasnt changed)

- the datalist filter is now near instant, vs before it was very laggy

- data is now stored in memory as well as the database, meaning any get calls can use the in memory objects vs accessing the database which causes slowness issues

- added "File Detection Interval" to libraries which determines how long to wait to recheck the file size to see if the file is being written to.  0 means don't recheck.   This should help with PVR applications that write in chunks

- added "Last Scanned" to library datalist to show when the library was last scanned

- still pulling out the old 3rd party controls, reason for using this library was the datalist which has been replaced.

 

Version: 0.0.1.97

- replace datalist with custom one, this is a lot faster for large lists of files in the library files page

- removed some controls and replaced with custom.  they were from the same library as the original datalist and since im not using that im removing all components from that library

- added cache store to help with load times

- added ability to set scan intervals per library.  this is in seconds.

- misc speed improvements

 

Version: 0.0.1.89

- Added Help link to main menu that opens to the wiki showing how to do a basic video node configuration

 

Version: 0.0.1.88

- Small change to movie lookup to ignore anything after a year if found in the file/folder name

image.thumb.png.566a8fa50c342302b094ccf3cf572e0e.pngimage.thumb.png.73fbdc79a78e36b5f4f9611b1fa67f7d.pngimage.thumb.png.67b9b260bfd6bc0e8883ba525b266853.png 

 

Edited by reven
Link to comment

Haven't downloaded this yet, but I might try it out to do something for me:

 

I am going to have family throw some of their photos into a directory.  I'd like that to be monitored and then on intervals, have the picture (or video) files be picked up and moved to new folders and organized into sub folders based on file creation year/month/day.  Not when the file was copied to the directory, but based off when the image/video was originally taken. 

 

Can this do that out of the box?

Link to comment
1 hour ago, birdsofprey02 said:

Haven't downloaded this yet, but I might try it out to do something for me:

 

I am going to have family throw some of their photos into a directory.  I'd like that to be monitored and then on intervals, have the picture (or video) files be picked up and moved to new folders and organized into sub folders based on file creation year/month/day.  Not when the file was copied to the directory, but based off when the image/video was originally taken. 

 

Can this do that out of the box?

Yes thats very easy to do

Just an Input File, and a Move action.   Inside the move action set the destination path to where you want to move them to.  this example im moving to D:\videos\pics as the bsae, then using the filemodifiedyear\month\day to create subfolders.   Pressing the "{" key will bring up a variable list you can use.  You can use create date or modified date.  In windows at least, windows tends to leave Last Write Time as the date the image was taken, and creation date whenever the file was copied to a location, so here im using the modified date.   Just check  you files to see what date to use.

image.thumb.png.93fef133a621a2e6b7b4e7c2041bbb52.pngimage.png.d72f4c33285ad1ffc82571135658bf2e.png

Link to comment

Version: 0.0.1.131

- Added "Schedule" to libraries

- Simplified "Libary" editor and put more advanced features on separate tabs

- Improvements to make the app more responsive and work better on mobile

- Added "Out of Schedule" to Libary Files, which shows files that the system knows about but will not be processed currently as it is outside their schedule.  These will automatically be processed once in their schedule

- Added "Time Zone" to settings, needed of schedules to work.  The system will set this based on the user if unset, so should be right most of the time, but if incorrect you can manually change the syetem Time Zone here.   This is only needed due to the docker being in UTC/GMT0 time and it needs to adjust the schedules to the correct timezone.

- Added "Executor" node.   This lets you run any process from a flow.

- Flow Templates can now prompt for settings at the add screen to simplify making a flow from a template.

 

Version: 0.0.1.120

- added Guide/Help when FileFlows has no flows/libraries configured.

- This changes the dashboard to explain the steps needed

 

Version: 0.0.1.118

- added unsupported banner shown on safari, that doesnt play nice with the flow page, I dont have a mac thats easy to dev on.  Until safari gets big user numbers, this will remain unsupported

- improved menu layout and added pages into groups

- added "Help" button to most pages to open help for that page

- unprocessed files from disabled libraries no longer appear in the queues

- added telemetry data for used templates (so I know which ones people like/use)

 

Version: 0.0.1.112

- Added Templates to Libraries.

   - Video Library

   - Music Library

   - Picture Library

   - SABnzbd Videos

   - Plex DVR

 

Version 0.0.1.111

- Added Descriptions to templates

- Fixed issue with creating folders in linux, where the permissions were incorrectly set

 

Version: 0.0.1.109

- Added fileCreateDay, fileCreateMonth, fileCreateDay, fileModifiedDay, fileModifiedMonth, fileModifiedYear to variables

 

Version: 0.0.1.108

- Added "Rescan" to libraries.  This will clear the last scan time and cause the library worker to rescan this library next time it runs

- Added "Templates" to "New Flow".  With a few templates (as seen in picture).   I'll extend on these as time goes on.    Happy to take suggestions for templates

image.png.6acd66585fe5f68c499ea4affcda0949.pngimage.png.32690aede5716ca30629fdc9789cf388.png

Link to comment

After some suggestions for features/plugins now.   So if anyone has any ideas or whats me to add anything let me know.

It's getting pretty stable.  I've just converted my entire library of TV/Movies using it.  Handy removing black bars from things so Plex can "zoom in" (personally I dont mind losing a little on the side to have a video full height on my TV).  Also everything being H265/AC3 I can guarantee everything in my house can playback everything without needing to transcode and file sizes are good (Saved about 1/3rd the storage I was using)
image.png.57da36eed189ad4c340962c466136411.png

Theres things still left to do
1. A plugin system to search/download new plugins from a repo.  This will save a new docker update when a node is added/fixed.
2. Add support for multiple workers at a time.  
3. Add support for external worker nodes, so can process files on mutliple systems.

But those things are probably more edge cases than main cases, so interested to see if anyone has a different use that may be a better thing to tackle next.

Link to comment
  • 2 weeks later...

Big Update.

 

Can now have additional processing nodes, so you can offload some processing (video conversion/whatever) to more systems.  

 

I've added windows installers to https://fileflows.com for the server app and the node.

 

The server/main app has an internal node, so if you dont want to have 2 installs, you don't need to.

 

unRAID currently just have the single server docker image.   Not sure if people would want a node docker for this?  Makes more sense making unRAID the server IMO.  But if someone has a use case for unRAID as a node, let me know.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.