Duplicate files

[Blade]

Is there a quick way in Unraid 6 to find duplicate files on the disks?

I know I have some disks that have some duplicates in the same folder structures but across multiple disks.

Is there an easy way to find them and remove them from a specific disk if found?

 

Thanks

[switchman]

I don't know about built into R6, but here are a couple threads/utilites that should help:

 

http://lime-technology.com/forum/index.php?topic=35183.0

 

http://lime-technology.com/forum/index.php?topic=33689.0

 

and this one

 

http://lime-technology.com/forum/index.php?topic=35183.msg328600#msg328600

[SpecFroce]

This would be such a nice feature to have integrated into the web GUI. Are there better methods to discovering duplicates now in 2019?

[lymbada]

@SpecFroce, I'm not sure how good it will be, but I'm currently adding an app 'dupeGuru' which appears to have a web-base GUI for performing some form of scanning function. It looks promising, hopefully it has the ability to delete all duplicates from a selected folder (which is what I mostly require).

[jpotrz]

On 7/11/2019 at 6:46 AM, lymbada said:

@SpecFroce, I'm not sure how good it will be, but I'm currently adding an app 'dupeGuru' which appears to have a web-base GUI for performing some form of scanning function. It looks promising, hopefully it has the ability to delete all duplicates from a selected folder (which is what I mostly require).

It seems dupeGuru only looks for logical duplicates. Not physical duplicates on different physical HDDs. I'm still struggling to find an "easy" method in finding these files myself.

 

[itimpi]

4 hours ago, jpotrz said:

t seems dupeGuru only looks for logical duplicates. Not physical duplicates on different physical HDDs. I'm still struggling to find an "easy" method in finding these files myself.

It might be quite easy (and efficient) to build something on top of the File Integrity plugin's checksum output as by their very nature identical fines should have identical checksums.

 

EDIT:  Looks like there may already be a capability for checking for duplicates in the File Integrity plugin.  I must try it out to see how well in works.