[Support] Djoss - Video Duplicate Finder


Djoss

Recommended Posts

Support for Video Duplicate Finder docker container

Application Name: Video Duplicate Finder
Application Site: https://github.com/0x90d/videoduplicatefinder

Docker Hub: https://hub.docker.com/r/jlesage/video-duplicate-finder
Github: https://github.com/jlesage/docker-video-duplicate-finder

 

Make sure to look at the complete documentation, available on Github !


Post any questions or issues relating to this docker in this thread.

  • Like 2
Link to comment

Holy ... This thing works surprisingly well. It’s blazing fast and accurate. It’s not transcoding anything but plays all media very well. If I need to nitpick the videoplayer has no controls but you can skip with a right click. All this in a browser I am impressed. How does this work? Thank you for adding this.

Link to comment
1 hour ago, NoobSpy said:

If I need to nitpick the videoplayer has no controls but you can skip with a right click.

I was not sure if a video player would be useful or not, so I integrated a basic one.  Note that you can also use left/right arrow keys.

 

1 hour ago, NoobSpy said:

All this in a browser I am impressed. How does this work?

 

The browser is basically running an HTML VNC client that connects to the application running in the container.

  • Like 1
Link to comment

Great tool for my way too full media server, highly appreciated.

There seems to be an issue with folder names including a "#" tho, not sure if this is an issue with the tool and if it can be fixed by you or if its an alpine linux thing?

If I select a folder for the "Search directorys" that includes a hashtag at the front (first character) it selects the next directory above it that does not start with a hashtag and there seems to be no way around it... would be great if that could be fixed or I'll just have to change my directory structure. ^^

Link to comment
3 hours ago, MellowB said:

Great tool for my way too full media server, highly appreciated.

There seems to be an issue with folder names including a "#" tho, not sure if this is an issue with the tool and if it can be fixed by you or if its an alpine linux thing?

If I select a folder for the "Search directorys" that includes a hashtag at the front (first character) it selects the next directory above it that does not start with a hashtag and there seems to be no way around it... would be great if that could be fixed or I'll just have to change my directory structure. ^^

 

I was able to reproduce.  This is an issue with the app itself.  I created a bug report for this: https://github.com/0x90d/videoduplicatefinder/issues/398

  • Like 1
Link to comment

This isn't working for me, it runs for a while but eventually I check and the container has stopped running. Maybe I have too many files, but don't we all? I have been using this on windows for years and it's great now, except for some groups of 50+ false duplicates.  In it's early days I'd have to run it a bunch of times to get anything, because each run made a little more progress before crashing.  Maybe that's what's happening here?  I'd much rather run this in a docker container on the system that the files are on, than my Windows PC.

Link to comment
4 minutes ago, Djoss said:

Do you have anything useful in the container log when it crashes ?

"...\appdata\VideoDuplicateFinder\log\nginx\error.log" is empty right now.  I just starting running a scan, I'll check again if/when it crashes.  I think I checked previously and it didn't look useful, but I may not know what to look for.

 

EDIT: I had the warning "Fix Common Problems: Warning: Docker application VideoDuplicateFinder has volumes being passed that are mounted by Unassigned Devices, but they are not mounted with the slave option". I fixed that, don't know if it could be the cause.

Edited by bobobeastie
Link to comment
3 hours ago, Djoss said:

So is it only the amount of files to scan that creates the crash ?  How many do you need to have ?

I don't know how I would know that the number of files is the cause for sure.  I would/do choose to scan all of them preferably, rough estimate for me is 230K files.  I scan the same directories successfully with the Windows version.  It takes about a day to finish, and I generally start it at night and close everything so it has enough memory. I noticed that this container only uses a bit more than a gig or RAM, which seems low in comparison.

 

I guess I'll run a test on a single small directory...

Link to comment
On 2/17/2023 at 11:28 AM, Djoss said:

Yes, if you can find a way to (more easily) reproduce that would be useful.

It was successful on a smaller directory.  I just realized I had the Scan Images or whatever checked and the setting that says 1 is most stable was at 2.  I think the 2 was my fault. I'm trying the whole thing with those two issues fixed.

 

EDIT: It just finished on the whole thing! Thanks.

Edited by bobobeastie
new details
Link to comment
  • 5 months later...

This looks like just what I need to clean up old versions of family videos.  Trying it out and I get the error "cannot find shared FFmpeg libraries".  Want to make sure its not a problem with my install - was ffmpeg/ffprobe bundled into the container or do I need to install it separately.  If I need to install separately, should I be putting it in the "/mnt/user/appdata/VideoDuplicateFinder" directory?
Thanks!

Link to comment
9 hours ago, Jonwork88 said:

This looks like just what I need to clean up old versions of family videos.  Trying it out and I get the error "cannot find shared FFmpeg libraries".  Want to make sure its not a problem with my install - was ffmpeg/ffprobe bundled into the container or do I need to install it separately.  If I need to install separately, should I be putting it in the "/mnt/user/appdata/VideoDuplicateFinder" directory?
Thanks!

Thanks for reporting.  This should be fixed with the latest version of the image.

Link to comment
  • 3 months later...

Seeing a problem where the files don't appear to be deleting.  I choose "Select Lowest Duration/Quality" and then after all the boxes are selected for the files I want to delete, I choose  "Delete From Disk", and the selected items still show up in the list. In fact at the window at the bottom of the page, it shows me how  many duplicates there are and  "Total Size Removed:0" 

 

image.png.053e4392f07d0a64d17dca65792d353b.png

 

 

Link to comment

Oh there was a log message, but I'm not sure what it means.  Other docker apps have no problems accessing this folder/files...

 

22:39:29 => Failed to delete file '/storage/X/Cyber-DL/Sorted Downloads/Videos/good_movie.mp4', reason: Access to the path '/storage/X/Cyber-DL/Sorted Downloads/Videos/good_movie.mp4' is denied., Stacktrace    at System.IO.FileSystem.DeleteFile(String fullPath)
   at VDF.GUI.ViewModels.MainWindowVM.DeleteInternal(Boolean fromDisk, Boolean blackList, Boolean createSymbolLinksInstead, Boolean permanently) in /tmp/vdf/VDF.GUI/ViewModels/MainWindowVM.cs:line 911

 

Link to comment

I think I corrected the issue. For some reason, this one particular download folder is owned by an incorrect entity.  After changing it to be like the others, the files were deleted fine with no error.  Ignore me.  Leaving this post here in case others run into this, as other unraid apps don't have issues, so maybe this app is running with different perms? 

Link to comment
8 hours ago, yoleska said:

this one particular download folder is owned by an incorrect entity

 

What was this entity ?

 

8 hours ago, yoleska said:

so maybe this app is running with different perms

 

By default the app is running under the correct user (nobody/users).

 

My guess is that the files that didn't have the correct ownership were create by another app that was not running under this user ?

Link to comment

The other application was running on a linux VM that also is hosted by Unraid, but I think the app set it's own permissions.   It was Cyber-DL, and I'm thinking now the permission issue wasn't directly related to that app, but something funky I did.  In either case, it's all resolved now after changing permissions on the video directories. 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.