February 4, 20233 yr Support for Video Duplicate Finder docker container Application Name: Video Duplicate Finder Application Site: https://github.com/0x90d/videoduplicatefinder Docker Hub: https://hub.docker.com/r/jlesage/video-duplicate-finder Github: https://github.com/jlesage/docker-video-duplicate-finder Make sure to look at the complete documentation, available on Github ! Post any questions or issues relating to this docker in this thread.
February 6, 20233 yr Holy ... This thing works surprisingly well. It’s blazing fast and accurate. It’s not transcoding anything but plays all media very well. If I need to nitpick the videoplayer has no controls but you can skip with a right click. All this in a browser I am impressed. How does this work? Thank you for adding this.
February 6, 20233 yr Author 1 hour ago, NoobSpy said: If I need to nitpick the videoplayer has no controls but you can skip with a right click. I was not sure if a video player would be useful or not, so I integrated a basic one. Note that you can also use left/right arrow keys. 1 hour ago, NoobSpy said: All this in a browser I am impressed. How does this work? The browser is basically running an HTML VNC client that connects to the application running in the container.
February 6, 20233 yr 40 minutes ago, Djoss said: I was not sure if a video player would be useful or not, so I integrated a basic one. Note that you can also use left/right arrow keys. It’s a great adition!
February 12, 20233 yr Great tool for my way too full media server, highly appreciated. There seems to be an issue with folder names including a "#" tho, not sure if this is an issue with the tool and if it can be fixed by you or if its an alpine linux thing? If I select a folder for the "Search directorys" that includes a hashtag at the front (first character) it selects the next directory above it that does not start with a hashtag and there seems to be no way around it... would be great if that could be fixed or I'll just have to change my directory structure. ^^
February 12, 20233 yr Author 3 hours ago, MellowB said: Great tool for my way too full media server, highly appreciated. There seems to be an issue with folder names including a "#" tho, not sure if this is an issue with the tool and if it can be fixed by you or if its an alpine linux thing? If I select a folder for the "Search directorys" that includes a hashtag at the front (first character) it selects the next directory above it that does not start with a hashtag and there seems to be no way around it... would be great if that could be fixed or I'll just have to change my directory structure. ^^ I was able to reproduce. This is an issue with the app itself. I created a bug report for this: https://github.com/0x90d/videoduplicatefinder/issues/398
February 13, 20233 yr This isn't working for me, it runs for a while but eventually I check and the container has stopped running. Maybe I have too many files, but don't we all? I have been using this on windows for years and it's great now, except for some groups of 50+ false duplicates. In it's early days I'd have to run it a bunch of times to get anything, because each run made a little more progress before crashing. Maybe that's what's happening here? I'd much rather run this in a docker container on the system that the files are on, than my Windows PC.
February 13, 20233 yr 4 minutes ago, Djoss said: Do you have anything useful in the container log when it crashes ? "...\appdata\VideoDuplicateFinder\log\nginx\error.log" is empty right now. I just starting running a scan, I'll check again if/when it crashes. I think I checked previously and it didn't look useful, but I may not know what to look for. EDIT: I had the warning "Fix Common Problems: Warning: Docker application VideoDuplicateFinder has volumes being passed that are mounted by Unassigned Devices, but they are not mounted with the slave option". I fixed that, don't know if it could be the cause. Edited February 13, 20233 yr by bobobeastie
February 13, 20233 yr Author You can get the container log by running "docker logs VideoDuplicateFinder". You can run the command after the crash.
February 14, 20233 yr 10 hours ago, Djoss said: You can get the container log by running "docker logs VideoDuplicateFinder". You can run the command after the crash. I have no clue what any of it means, see attached. Thanks. VDF.log
February 17, 20233 yr Author So is it only the amount of files to scan that creates the crash ? How many do you need to have ?
February 17, 20233 yr 3 hours ago, Djoss said: So is it only the amount of files to scan that creates the crash ? How many do you need to have ? I don't know how I would know that the number of files is the cause for sure. I would/do choose to scan all of them preferably, rough estimate for me is 230K files. I scan the same directories successfully with the Windows version. It takes about a day to finish, and I generally start it at night and close everything so it has enough memory. I noticed that this container only uses a bit more than a gig or RAM, which seems low in comparison. I guess I'll run a test on a single small directory...
February 17, 20233 yr Author 14 minutes ago, bobobeastie said: rough estimate for me is 230K files Ouch! 15 minutes ago, bobobeastie said: I guess I'll run a test on a single small directory... Yes, if you can find a way to (more easily) reproduce that would be useful.
February 18, 20233 yr On 2/17/2023 at 11:28 AM, Djoss said: Yes, if you can find a way to (more easily) reproduce that would be useful. It was successful on a smaller directory. I just realized I had the Scan Images or whatever checked and the setting that says 1 is most stable was at 2. I think the 2 was my fault. I'm trying the whole thing with those two issues fixed. EDIT: It just finished on the whole thing! Thanks. Edited February 19, 20233 yr by bobobeastie new details
August 13, 20232 yr This looks like just what I need to clean up old versions of family videos. Trying it out and I get the error "cannot find shared FFmpeg libraries". Want to make sure its not a problem with my install - was ffmpeg/ffprobe bundled into the container or do I need to install it separately. If I need to install separately, should I be putting it in the "/mnt/user/appdata/VideoDuplicateFinder" directory? Thanks!
August 14, 20232 yr Author 9 hours ago, Jonwork88 said: This looks like just what I need to clean up old versions of family videos. Trying it out and I get the error "cannot find shared FFmpeg libraries". Want to make sure its not a problem with my install - was ffmpeg/ffprobe bundled into the container or do I need to install it separately. If I need to install separately, should I be putting it in the "/mnt/user/appdata/VideoDuplicateFinder" directory? Thanks! Thanks for reporting. This should be fixed with the latest version of the image.
December 4, 20232 yr Seeing a problem where the files don't appear to be deleting. I choose "Select Lowest Duration/Quality" and then after all the boxes are selected for the files I want to delete, I choose "Delete From Disk", and the selected items still show up in the list. In fact at the window at the bottom of the page, it shows me how many duplicates there are and "Total Size Removed:0"
December 4, 20232 yr Oh there was a log message, but I'm not sure what it means. Other docker apps have no problems accessing this folder/files... 22:39:29 => Failed to delete file '/storage/X/Cyber-DL/Sorted Downloads/Videos/good_movie.mp4', reason: Access to the path '/storage/X/Cyber-DL/Sorted Downloads/Videos/good_movie.mp4' is denied., Stacktrace at System.IO.FileSystem.DeleteFile(String fullPath) at VDF.GUI.ViewModels.MainWindowVM.DeleteInternal(Boolean fromDisk, Boolean blackList, Boolean createSymbolLinksInstead, Boolean permanently) in /tmp/vdf/VDF.GUI/ViewModels/MainWindowVM.cs:line 911
December 4, 20232 yr I think I corrected the issue. For some reason, this one particular download folder is owned by an incorrect entity. After changing it to be like the others, the files were deleted fine with no error. Ignore me. Leaving this post here in case others run into this, as other unraid apps don't have issues, so maybe this app is running with different perms?
December 4, 20232 yr Author 8 hours ago, yoleska said: this one particular download folder is owned by an incorrect entity What was this entity ? 8 hours ago, yoleska said: so maybe this app is running with different perms By default the app is running under the correct user (nobody/users). My guess is that the files that didn't have the correct ownership were create by another app that was not running under this user ?
December 6, 20232 yr The other application was running on a linux VM that also is hosted by Unraid, but I think the app set it's own permissions. It was Cyber-DL, and I'm thinking now the permission issue wasn't directly related to that app, but something funky I did. In either case, it's all resolved now after changing permissions on the video directories.
March 11, 20251 yr Would someone point me to a cheat sheet on what the variables in the Expression Builder mean? I'm guessing these are obvious to a C# programmer, but, for example, I have no idea what item.IsBestSize means and my web searches for "item.IsBestSize","Dynamic Expesso item.IsBestSize", etc have not helped at all.
March 12, 20251 yr First of all: great docker! But to make the best even better, I would like for a small addition: The exclusion list currently only allows full paths, this is less helpful for me. Here every movie has its own subdirectory and within it is another subdir called ".actors". You might already guessed it, it contains the pictures of the players in the movie. And because often some actors play within more than one movie, there are always some duplicates for these pics to be found. In my case over 156996! It can be easily seen that this gives you an overwhelming list to check, with most of the dupes to be correct. I would like to check images, but also like to exclude "*/.actors" from the scan.
March 12, 20251 yr Can the files tab do what you need for .actors? I just started fooling with it and there seems to be no documentation, so I'm having to figure it out by trail and error.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.