kopuz Posted August 24, 2016 Share Posted August 24, 2016 I have the following setup with 2 disks Tower > Movies > A What happens if I was to copy the same file to Disk1 > Movies > A and Disk2 > Movies > A Both disks are in the user share Movies. The same exact file is copied directly too 2 different disks to the same user share with the same path. Would the file take up double the space in the array? Hope I explained my scenario clearly enough. Thanks in advance for the help! Quote Link to comment
itimpi Posted August 24, 2016 Share Posted August 24, 2016 Yes, the file uses twice the space, even though you will only see it once at the User Share level. I think unRAID shows the one on the lowest number disk but I am not sure about that. You can get strange behaviour in that if you try to delete (or rename) the file from the share it still appears to remain as unRAID will now present the second copy. Quote Link to comment
kopuz Posted August 24, 2016 Author Share Posted August 24, 2016 Anyone recommend a good duplicate file scanner? Docker maybe? Quote Link to comment
kopuz Posted August 24, 2016 Author Share Posted August 24, 2016 Dug this up. Working very well so far. http://lime-technology.com/forum/index.php?topic=35183.0 Quote Link to comment
itimpi Posted August 24, 2016 Share Posted August 24, 2016 I wrote that script originally to address exactly this issue when I was moving data around and managed to make mistakes and ended up with duplicates. I think I have seen that the "Fix Common Problems" plugin will also now show if you have duplicates. Quote Link to comment
Sn3akyP3t3 Posted August 25, 2017 Share Posted August 25, 2017 Two applications I often turn to in duplicate file removal situations are: Duplicate Cleaner (Digital Volcano Software) - https://www.digitalvolcano.co.uk/duplicatecleaner.html (Free or Paid versions available) Its pretty amazing software! Its flexible in finding duplicates in ways that work for you covering different possible scenarios. It groups found duplicates together for verification. Cleanup processing has plenty of options available to the user to decide which file survives with criteria like age, size, or location. The pro version unlocks features such as finding dupes in photos which can be tedious to do manually. VisiPics - http://www.visipics.info/index.php?title=Main_Page (Free) Another one I find useful from time to time, but mostly now on other users computers and often after migration from old computers to new computers. It pretty much restricts duplicate file cleanup to pictures only, but its really good at it! I usually do this cleanup in steps starting with strict mode then knocking it down a notch to find like matches that may have been exports of the original or reduced for sharing. Quote Link to comment
Philip Gore Posted February 1, 2018 Share Posted February 1, 2018 Try Duplicate Files Deleter, a free and good software. Quote Link to comment
steve1977 Posted April 18, 2020 Share Posted April 18, 2020 On 8/24/2016 at 3:25 PM, itimpi said: I wrote that script originally to address exactly this issue when I was moving data around and managed to make mistakes and ended up with duplicates. I think I have seen that the "Fix Common Problems" plugin will also now show if you have duplicates. You sure about the "fix common problems" plugin? I suspect I have duplicates (same file on different disks, shows up once in user share). What's the best way to look into this? Quote Link to comment
superloopy1 Posted April 18, 2020 Share Posted April 18, 2020 Dynamix file integrity plugin can give you a list is all duplicate files, I use it regularly, never had a problem. Quick and simple to use. Quote Link to comment
steve1977 Posted April 18, 2020 Share Posted April 18, 2020 2 hours ago, superloopy1 said: Dynamix file integrity plugin can give you a list is all duplicate files, I use it regularly, never had a problem. Quick and simple to use. Thanks. I installed it, but not fully clear to me what it does and how I get to the list of duplicate files. Is this looking more broadly for "real" duplicates? I am looking for files with same folder/file name across disks (that then only show up once when in same user share). Thanks! Quote Link to comment
Squid Posted April 18, 2020 Share Posted April 18, 2020 8 hours ago, steve1977 said: You sure about the "fix common problems" plugin? I Yup. Extended tests 1 Quote Link to comment
superloopy1 Posted May 4, 2020 Share Posted May 4, 2020 Sorry, missed this one a while back, oops! Yes, dynamix file integrity has a 'Find' option which, once you've built/exported the array disks, will identify duplicates across all disks producing a logfile. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.