Jump to content
Djoss

[Support] Djoss - dupeGuru

85 posts in this topic Last Reply

Recommended Posts

3 hours ago, JustinChase said:

I really like your docker, in general, but in use, it causes me much frustration.

 

PLEASE allow one to perform all actions on selected or marked.  In practice, there is no good way to get a list selected and delete that selection.

 

inverting the marking doesn't allow you to pick the reference files, so when one tries to actually do something, the options available just aren't adequate to allow me to do what I need.

Unfortunately I'm not the author of dupeGuru, so I cannot really change functionality of the application itself.

However, from what I understand, you want to perform actions on reference and non-reference files.  I guess you know that you can change which file is the reference?  Once all files the proper status, you can perform the action on them.

 

Share this post


Link to post
1 hour ago, Djoss said:

Unfortunately I'm not the author of dupeGuru, so I cannot really change functionality of the application itself.

However, from what I understand, you want to perform actions on reference and non-reference files.  I guess you know that you can change which file is the reference?  Once all files the proper status, you can perform the action on them.

 

the problem is that you can only delete selected, not marked files.  however, you can't invert the marked files to the reference files, so you can't always delete the files you want.  you can't mark the selected files as another example.

 

The combinations available just don't let you "pick" what you want to delete.  The point (for me) for finding duplicates is to delete the duplicate files, but the methods offered don't let me mark or select (pick) the right files easily.

 

If you could delete selected instead of marked, that would help, but if you could just act on either marked or selected the same way, I could get done what I want.

Share this post


Link to post
On 11/13/2018 at 4:20 PM, JustinChase said:

however, you can't invert the marked files to the reference files, so you can't always delete the files you want.

Yes this is possible: Actions -> Mark Selected into Reference

Share this post


Link to post
1 hour ago, Djoss said:

Yes this is possible: Actions -> Mark Selected into Reference

You've made my point, convert SELECTED into reference.  there's no way to convert MARKED into reference.

 

Also, you can only send one of them to the recycle bin (marked or selected; not both).

 

As I said you cannot perform all actions on marked AND selected, which limits what one can do.

Edited by JustinChase

Share this post


Link to post

Djoss maintains the container, not the application. You may have better luck making the request on the application Github site: https://github.com/hsoft/dupeguru/


Edit: Although this statement on Github means it will likely not be addressed (at least he is upfront about it):



Current status: Unmaintained

I haven't worked on dupeGuru for a while and frankly, I don't want to. I never had any duplicate problems so I don't even care about the raison d'être of this thing.

I don't want to answer incoming issues and I don't want to let them pile off unanswered either, that feels rude. So here I am, being straightforward about it.

If you're considering using dupeGuru, you might want to give it a try but if it doesn't meet your needs I suggest that you use another program because it's unlikely to ever be improved again.

If you're a developer wanting to pick it up, by all means, do so! Fork it off and release something. I will be more than happy to "officially" point to any fork that remotely looks like a serious effort. I will also be happy to assist if you have questions about the code.

Edited by planetix

Share this post


Link to post
On 5/5/2017 at 8:45 PM, jonathanm said:

Very definitely possible, and can be quite problematic for someone who doesn't have a good handle on what a user share is.

Consider the following.

/mnt/disk1/myshare/test.txt

/mnt/disk2/myshare/test.txt

/mnt/user/myshare/test.txt

In this example, the file accessed at /mnt/user/myshare/test.txt will be the one on disk1, for all intents and purposes /mnt/disk2/myshare/test.txt doesn't exist in the user share system. However... if you delete /mnt/user/myshare/test.txt, then the one on disk2 will show up the next time the user share system is reloaded. It's quite possible to have 2 or more identically named files this way, but the contents not be equal. You could theoretically hide files from the user shares this way, and cause all sorts of confusion.

 

Typically this kind of shenanigans occurs when someone with a poor grasp of how unraid works starts playing around with disk shares. That is one of the main reasons disk shares are disabled by default.

 

In my case, this happened when using unBalance to scatter files from one disk to multiple others.

As I see this docker could not help with looking for duplicates on the disk-level, so we know of any other docker og app that can help us with finding and deleting these duplicate files? :)

Share this post


Link to post
3 minutes ago, Squazz said:

so we know of any other docker og app that can help us with finding and deleting these duplicate files?

Fix Common Problems' Extended Tests looks for duplicated files. (/mnt/disk1/share/test.txt and /mnt/disk2/share/test.txt)

Share this post


Link to post
Just now, Squid said:

Fix Common Problems' Extended Tests looks for duplicated files. (/mnt/disk1/share/test.txt and /mnt/disk2/share/test.txt)

I did hear that it could do something like that, of couse I havn't tried "extended tests" yet :/

Share this post


Link to post
15 minutes ago, Squid said:

Fix Common Problems' Extended Tests looks for duplicated files. (/mnt/disk1/share/test.txt and /mnt/disk2/share/test.txt)

It only looks, right? There's no magic button to help one clean up?

Share this post


Link to post
1 hour ago, Squazz said:

It only looks, right? There's no magic button to help one clean up?

And there will never be.  Problem is that when reading the file from /mnt/user/share/..., unRaid will give you the version from the lowest numbered drive (cache, disk1, disk2, etc)  The version which you may be expecting, or which may be the latter version could be on the highest numbered drive.  Hard to say, and I don't want the plugin to attempt to guess what the user wants to actually do.

 

You can manually clean them up via Krusader (if you map /mnt instead of /mnt/user in the template) by navigating to the appropriate drive(s) and inspect the files accordingly

 

Should be noted that under normal circumstances, duplicated files do not happen under unRaid.

Edited by Squid

Share this post


Link to post
7 hours ago, Squid said:

And there will never be.  Problem is that when reading the file from /mnt/user/share/..., unRaid will give you the version from the lowest numbered drive (cache, disk1, disk2, etc)  The version which you may be expecting, or which may be the latter version could be on the highest numbered drive.  Hard to say, and I don't want the plugin to attempt to guess what the user wants to actually do.

 

You can manually clean them up via Krusader (if you map /mnt instead of /mnt/user in the template) by navigating to the appropriate drive(s) and inspect the files accordingly

 

Should be noted that under normal circumstances, duplicated files do not happen under unRaid.

I totally understand, and I only have this problem due to me using unBalance :/

I was hoping there was a way to tell a program "if you find any dupes on diskX, delete them", so that I was in charge of what files was removed. Guess I'll have to do it the hard way :P

Share this post


Link to post
15 hours ago, Squazz said:

As I see this docker could not help with looking for duplicates on the disk-level,

I'm confused by this statement. It works just fine for me to do this exact task. Why do you not want to use this docker?

Share this post


Link to post
7 minutes ago, jonathanm said:

I'm confused by this statement. It works just fine for me to do this exact task. Why do you not want to use this docker?

I forgot to edit the template, got it working :)

Share this post


Link to post

Is there a reason the"fuzzy image searching functionality is not included in the docker? I see exif data and content scans only.

Share this post


Link to post
18 hours ago, xhaloz said:

Is there a reason the"fuzzy image searching functionality is not included in the docker? I see exif data and content scans only.

In Picture mode, when you select the Content scan type, you can adjust the "Filter hardness".  I think this is what you mean by "fuzzy image search".

Share this post


Link to post

Hi Djoss, thanks for the app, I can use a tool to combat dupes on my server! Question though, I've got the 'ignore hardlinks' option checked, but a scan still returns hardlinks in the results. Is this a known bug or am I missing something on my end?

 

Edit: retested. dupeGuru marks a hardlink as a dupe, regardless of the option to ignore.

 

E2: the documentation for dupeGuru says it checks if inodes are the same. Somehow, even though they're hardlinks, the inodes are different. I'm stumped for now.

Edited by thatsthefrickenlightning

Share this post


Link to post
On 5/24/2019 at 5:34 PM, thatsthefrickenlightning said:

Hi Djoss, thanks for the app, I can use a tool to combat dupes on my server! Question though, I've got the 'ignore hardlinks' option checked, but a scan still returns hardlinks in the results. Is this a known bug or am I missing something on my end?

 

Edit: retested. dupeGuru marks a hardlink as a dupe, regardless of the option to ignore.

 

E2: the documentation for dupeGuru says it checks if inodes are the same. Somehow, even though they're hardlinks, the inodes are different. I'm stumped for now.

When you say that inodes are different, is it from unRAID or from the container?

Share this post


Link to post
On 5/26/2019 at 5:55 AM, thatsthefrickenlightning said:

I check the inodes in Krusader by opening a terminal and entering 'stat file.ext' for both files.

Do the same thing from unRAID terminal.  If you still have different inodes, then the files are definitely not hard linked.

Share this post


Link to post
3 hours ago, Djoss said:

Do the same thing from unRAID terminal.  If you still have different inodes, then the files are definitely not hard linked.

You're right. When checking out the two files with stat in an unRaid terminal, the system returns the same inode. Strange that Krusader returns something different. I suppose I'll believe unRaid over Krusader, though. Doubly so because deleting one of the two hardlinks yields no extra free drive space, as one would expect from working hardlinks.

 

Here's my next puzzle, then. When I 'unleash' dupeGuru on my cache drive, where there currently resides a downloaded file and a hardlink to that same data in another folder, dupeGuru says there are no dupes. So far, so good. Now when I have dupeGuru scan the share that those two files are in, among the results are the two hardlinks that dupeGuru previously said were not dupes. How come?

 

Appreciate the replies. :)

Edited by thatsthefrickenlightning

Share this post


Link to post
6 hours ago, thatsthefrickenlightning said:

Now when I have dupeGuru scan the share that those two files are in, among the results are the two hardlinks that dupeGuru previously said were not dupes. How come?

The fuse filesystem that makes up the user shares keeps its own inodes list AFAIK. Hardlinking only works inside the same filesystem, each disk has its own individual filesystem, and the user share fuse is another filesystem all its own.

Share this post


Link to post
16 hours ago, jonathanm said:

The fuse filesystem that makes up the user shares keeps its own inodes list AFAIK. Hardlinking only works inside the same filesystem, each disk has its own individual filesystem, and the user share fuse is another filesystem all its own.

Alright, so basically I need to point dupeGuru to drives and not shares, then? Noted. Thank you!

Share this post


Link to post

So, seems like this docker has stopped working.  I'm using Unraid 6.7.0 and everytime I try and launch the app, it gives:

 

Execution Error

Server Error

 

I've uninstalled the docker, deleted the dupeguru folder from appdata, and reinstalled.  Same issue.

 

Anyone else having this problem?

Share this post


Link to post
6 minutes ago, JasonK said:

Execution Error

Server Error

Usually, that means that while the app was stopped, you've added another and it is using the same port(s) as dupeGuru.

 

Easy way to see what the error actually is is to edit the template, make a change, revert the change, then hit apply.  The error will be in the docker run command.

Share this post


Link to post
17 minutes ago, Squid said:

Usually, that means that while the app was stopped, you've added another and it is using the same port(s) as dupeGuru.

 

Easy way to see what the error actually is is to edit the template, make a change, revert the change, then hit apply.  The error will be in the docker run command.

Squid - thanks for the reply.  Did as you suggested, and it did give a port conflict on the run line.  Which is strange, because when I look at the docker allocations for the 4 dockers I have, there's no conflicts:

 

image.png.dea4e117fa108691ad36ae39be44029c.png

 

I'll give a reboot to see if that helps.  Stand by :)

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.