Duplicate files across array drives

[Aegisnir]

Hey there everyone.  I just recently migrated my unraid server to new hardware and downsized my array to remove my slow WD Red drives.  I moved data from disk to disk and just discovered I may have duplicated the same drive on two others.  I know if the same file exists in two drives, the actual SHARE would only present it once so not sure whats the best way to figure this out.  Does unraid have any built in detection to know when it has the same file on 2 disks?  Obviously I dont want to take up double the space on my array.  I did this with over 4TBs of data it seems so it may have taken up 8TB in total if i duplicated them.....I tried DupeGuru but didnt have great results.

[Frank1940]

I will assume that you know what two disks you spread the content to.   Further assuming that you have one of the latest versions of Unraid installed. 

 

I know of no tools which will help you figure which files are duplicated.  You will have to do it by grunt work.  I would use the GUI built-in file manager in the disk mode that you can find here: 

 

Open up two GUI windows and select the file manager for each disk.   Start by looking for common directories.  You will have to  investigate each common directory and all common sub-directories.  (I would use screen captures as much as possible and organize thing using marking pens so you are sure you have gone down each branch.)  IF you have to, analyze  each one.  As an example, if I have a share called "Backup" which contains folders for periodic backups of individual computers and I find two identical folders, I can probably delete one of the folders without much further investigation.  (This might not be true if my data disks are almost completely full...)  There will be cases where you have to compare things file by file looking for the duplicates.  You can also use your browser's print function to get printouts to work from.

 

I think you have probably figured out at this point that when facing a project like this to make out a task list and check off each item as you start/complete it...

[ChatNoir]

If you are using the File Integrity plugin, there is an option for that.

But if you are not, I am not sure it is the best solution.

[remotevisitor]

If the duplicates are in the same place on various data drives then the script on this posting may help