On 6/23/2023 at 3:46 AM, JorgeB said:

You restore from a backup, same for xfs with the plugin. 

But what happens if your corrupted file was copied to your last back up? I was hoping I would be able to repair the file ala QPAR or MULTIPAR which have this ability.

Is there a good video on this utility? I could not find any on youtube. Again thanks...

45 minutes ago, Vetteman said:

I was hoping I would be able to repair the file ala QPAR or MULTIPAR which have this ability.

This plugin can only verify files, not repair them.

3 hours ago, JorgeB said:

This plugin can only verify files, not repair them.

Thanks for the reply. Much appreciated. However restoring from backups may not be a salvation IF the file was already corrupted at time of backup, unless you have something like file history enabled.

 

Is there a QPAR/MULTIPAR type of plugin for Unraid?

 

Again thanks for the reply...

21 minutes ago, Vetteman said:

Is there a QPAR/MULTIPAR type of plugin for Unraid?

Not that I'm aware.

I installed this plug-in with some hope of file integrity checking.

 

But so far it's just been a very confusing, resource hungry journey.

 

Apart from trying to understand why "build up-to-date" is now all green ticked, when I have only performed the manual 'Build' for 1 of my drives (so far), is very confusing.  If I then try to do the "Export" command, it says nothing to export on all the drives that say "build up-to-date" (except for the one I know I manually did a Build for).

 

Then, today I tried the "Find" command, and it said duplicate files found and output a 'duplicate_file_names.txt' file.

On looking at this file it makes no sense.

e.g. (a portion of the text file):

disk1,disk1,disk1,disk1 [equal content] SDTVShows/SDTV Series/Star Trek Voyager/Star Trek- Voyager.S01E01 - Caretaker (1).576p.mkv
disk1,disk1,disk1,disk1 [equal content] SDTVShows/SDTV Series/Star Trek Voyager/Star Trek- Voyager.S01E02 - Caretaker (2).576p.mkv
disk1,disk1,disk1,disk1 [equal content] SDTVShows/SDTV Series/Star Trek Voyager/Star Trek- Voyager.S01E03 - Parallax.576p.mkv
disk1,disk1,disk1,disk1 [equal content] SDTVShows/SDTV Series/Star Trek Voyager/Star Trek- Voyager.S01E04 - Time and Again.576p.mkv
disk1,disk1,disk1,disk1 [equal content] SDTVShows/SDTV Series/Star Trek Voyager/Star Trek- Voyager.S01E05 - Phage.576p.mkv
disk1,disk1,disk1,disk1 [equal content] SDTVShows/SDTV Series/Star Trek Voyager/Star Trek- Voyager.S01E06 - The Cloud.576p.mkv
disk1,disk1,disk1,disk1 [equal content] SDTVShows/SDTV Series/Star Trek Voyager/Star Trek- Voyager.S01E07 - Eye of the Needle.576p.mkv
disk1,disk1,disk1,disk1 [equal content] SDTVShows/SDTV Series/Star Trek Voyager/Star Trek- Voyager.S01E08 - Ex Post Facto.576p.mkv
disk1,disk1,disk1,disk1 [equal content] SDTVShows/SDTV Series/Star Trek Voyager/Star Trek- Voyager.S01E09 - Emanations.576p.mkv
disk1,disk1,disk1,disk1 [equal content] SDTVShows/SDTV Series/Star Trek Voyager/Star Trek- Voyager.S01E10 - Prime Factors.576p.mkv
disk1,disk1,disk1,disk1 [equal content] SDTVShows/SDTV Series/Star Trek Voyager/Star Trek- Voyager.S01E11 - State of Flux.576p.mkv
disk1,disk1,disk1,disk1 [equal content] SDTVShows/SDTV Series/Star Trek Voyager/Star Trek- Voyager.S01E12 - Heroes and Demons.576p.mkv

 

This makes no sense to me?  There is no indication of what the CSV columns mean.  What are the duplicate filenames?   The episode files listed (along with all the other files included in the 'duplicate_file_names.txt' file), are clearly all different filenames.

 

Should I just uninstall this plug-in?  Is there a better plug-in for detecting file level corruption?

 

Even a plug-in that is a little more user-friendly in terms of how to use it, and what it is doing, would be helpful!

21 hours ago, KoolKiwiNZ said:

 

 

Apart from trying to understand why "build up-to-date" is now all green ticked, when I have only performed the manual 'Build' for 1 of my drives (so far), is very confusing. 

could it be that this is enabled? if so, it will automatically hash new files and then its just natural that the build is up to date.


For your duplicate issue, it does look confusing.


Edit:

The duplicate hash output is certainly a lot more clear

 

hash: 008d7cfa771fda2262418bb32b46978a8ce3a1b48211ea2f363d1f4aa8b70329
>disk3 Magnon-Box/Download Backup/Pre/elvui-12.74/ElvUI/Classic/Modules/Skins/Debug.lua
>disk3 Magnon-Box/Download Backup/Pre/elvui-12.74/ElvUI/TBC/Modules/Skins/Debug.lua
>disk3 Magnon-Box/Download Backup/Pre/elvui-12.76/ElvUI/Classic/Modules/Skins/Debug.lua
>disk3 Magnon-Box/Download Backup/Pre/elvui-12.76/ElvUI/TBC/Modules/Skins/Debug.lua
hash: 0091067a3546fbe739817e6df458a1fc6bd0c9df49dc83285f8eb917ea8923c0
>disk2 Magnon-Box/Tools/UT_PCE_AC68_2088/UT_PCE-AC68_2115/0x041e.ini
>disk2 Magnon-Box/Tools/UT_PCE_AC68_2088/UT_PCE_AC68_2088/0x041e.ini
>disk5 OneDrive/Treiber/UT_PCE_AC68_2088/UT_PCE-AC68_2115/0x041e.ini
>disk5 OneDrive/Treiber/UT_PCE_AC68_2088/UT_PCE_AC68_2088/0x041e.ini

 

Edited February 21, 20251 yr by Mainfrezzer

2 hours ago, Mainfrezzer said:

could it be that this is enabled? if so, it will automatically hash new files and then its just natural that the build is up to date.

Yes, I have Enabled "Automatic protect new and modified files".  I don't actually understand why you wouldn't have this Enabled, if you're trying to identify the integrity of the files on your system.  Otherwise, any new / changed files would presumably not be included?

 

Are you suggesting that by Enabling this setting it then disregards the fact that all of the existing files have not yet had a hash built as yet?  This doesn't really make any sense to me.   If this was the case, the option should be called: "ONLY protect new and modified files". 

 

Also, I hadn't added or modified any files on the disks that it decided were now "build up-to-date" on (even though I hadn't done an initial manual 'Build' yet).

 

Perplexing...

8 hours ago, Mainfrezzer said:

For your duplicate issue, it does look confusing.

Edit:
The duplicate hash output is certainly a lot more clear
 

hash: 008d7cfa771fda2262418bb32b46978a8ce3a1b48211ea2f363d1f4aa8b70329
>disk3 Magnon-Box/Download Backup/Pre/elvui-12.74/ElvUI/Classic/Modules/Skins/Debug.lua
>disk3 Magnon-Box/Download Backup/Pre/elvui-12.74/ElvUI/TBC/Modules/Skins/Debug.lua
>disk3 Magnon-Box/Download Backup/Pre/elvui-12.76/ElvUI/Classic/Modules/Skins/Debug.lua
>disk3 Magnon-Box/Download Backup/Pre/elvui-12.76/ElvUI/TBC/Modules/Skins/Debug.lua
hash: 0091067a3546fbe739817e6df458a1fc6bd0c9df49dc83285f8eb917ea8923c0
>disk2 Magnon-Box/Tools/UT_PCE_AC68_2088/UT_PCE-AC68_2115/0x041e.ini
>disk2 Magnon-Box/Tools/UT_PCE_AC68_2088/UT_PCE_AC68_2088/0x041e.ini
>disk5 OneDrive/Treiber/UT_PCE_AC68_2088/UT_PCE-AC68_2115/0x041e.ini
>disk5 OneDrive/Treiber/UT_PCE_AC68_2088/UT_PCE_AC68_2088/0x041e.ini

 

 

Yes, very confusing.   I've now found that after nearly completing all of my manual "Builds' and 'Exports' (12 of 13 disks completed), when I now run 'Find', it reports no duplicates have been found!  So, no output file.

 

But, if I tick the duplicate hash box, I get a 292MB 'duplicate_file_hashes.txt' output file! 

 

In addition, even though all of my manual 'Exports' have been done / updated (and are green ticked), for the 12 of 13 disks that have completed the 'Build', the 'Find' initially rpeorts: 'Hash files are not up-to-date'.  Even though the Exports (for the completed 'Build' disks), are ALL green ticked!

 

I'm really struggling to understand this.  It's really looking like a buggy Plugin!  Or, at the very least, a buggy User Interface?

 

17 minutes ago, KoolKiwiNZ said:

I've now found that after nearly completing all of my manual "Builds' and 'Exports' (12 of 13 disks completed), when I now run 'Find', it reports no duplicates have been found!  So, no output file.

 

But, if I tick the duplicate hash box, I get a 292MB 'duplicate_file_hashes.txt' output file! 

 

Interestingly, although NO duplicate filenames are now being found, the 292MB 'duplicate_file_hashes.txt' file includes obvious duplicate filenames!  Huh?

 

e.g.

Quote

hash: 000adeeba872143af07c6ba3dce5dd178a3c69036c3315a07f4546903c5a3bad
>disk6 Photos/2003/TradeMe/112-1209_IMG.JPG
>disk6 Photos_Archive/2003/TradeMe/112-1209_IMG.JPG

 These are obviously duplicate files (with the same filename!).   But 'Find' still says "No duplicate file names found".

 

I'm really at a loss as to how this Plugin can be trusted to actually work?

Yet more weirdness, that defies my logical understanding of this plugin...

 

I setup the verification schedule, as per the help instructions. 

i.e. Specifically: "When multiple tasks exist then each task is executed in the next schedule in a rotating fashion. "

 

I put Disks 1 - 4 in TASK 1, Disks 5 - 7 in TASK 2 etc. etc. (as below).

 

I then expected the first weekly schedule run to do disks 1 - 4 (TASK 1), as per the help.  Then next week would be Diisks 5 - 7 (in TASK 2), etc.

 

However, the first scheduled verification has just started and it is currently working on Disk 11 & 12 (from TASK 4), and also showing Disk 1 (from TASK 1), is also in progress!

But, it also has only notified me that Disks 11 & 12 are running, but the below also shows Disk 1 as a (presumably) "in progress" coloured box???

 

I mean, WTF?   How is this verification schedule really supposed to work?  Does anything work as it's supposed to?

 

Edited February 22, 20251 yr by KoolKiwiNZ

Okay, another question:

My first scheduled verify task finished overnight. 

 

But, how do I know:

a). That a disk has actually done a verify pass?  Nothing has changed in the user interface to indicate a verfy pass has been done!

b). Whether the verification was successful, and how many files were verified?

 

The only indication that a verify pass has happened (that I can find), is in the sytem log (mirrored in my notification).

i.e.

Quote

Feb 23 03:15:32 mediaserver bunker: Verify task for disk12 finished, duration: 5 hr, 15 min, 31 sec.

 

The single log entry only tells me the task finished.  No indication of whether it was successful or not.  How many errors it found, or confirmation of zero errors found.  Or, how many files it processed etc.

 

So are we just supposed to assume it processed all the files on the disk successfully, and no errors were found???

 

Apologies if my posts are coming across a bit negative, but so far every step of the way with this plugin has raised more questions, and less confidence that it is actually doing what it is supposed to be doing.

I'm sure the developer is very capable, but so far my user experience just seems full of bugs, or at least full of user interface bugs?

 

So, I thought I'd setup my verification schedule so that the second scheduled verification would kick-off.

 

The thinking was that since the first verififcation run seemed to do TASK 4 (for some reason), that maybe the next (2nd) verification run, would wrap around to doing TASK 1 (Disks 1 - 4)?

 

But, NO.  Instead the 2nd verification scheduled run is again doing TASK 4, so my disks 11 - 13 are getting verified once again.

 

In addition, while it's running, the Settings page is once again showing an activity on Disk 1 (in TASK 1), even though Disk 1 is not being verified.  According to the notifications only Disks 11 - 13 are being verified. 

 

So, I have no idea:

a).  Why TASK 4 is once again being run (instead of wrapping around to do TASK 1 next).

b).  Why Disk 1 (scheduled in TASK 1) icon in Settings, appears to be showing it has some verification activity?

 

Here's the verification TASK schedule screenshot, which verification of TASK 4 (Disks 11 - 13) is again in progress (and showing Disk 1 seems to also indicate activity?): 

 

Yes, I am still persevering with this plugin, even though I'm yet to find anything that does what I'd expected it to do!    

Edited February 23, 20251 yr by KoolKiwiNZ

on the surface this plugin seems to be like a good idea, but since there is an inode limit, it can't even protect the drives i actually care about.  i'm removing it and will stop following this thread because plugin serves no useful purpose.

Okay, All Builds complete.  All Exports complete.  3 Disks have been verified (twice now - see above).

 

So, my first attempt at moving some new files to the server...  and you guessed it... more questions.

 

The mover moved some new files onto Disk 6.

 

But when I now go to the "Integrity" page, I get the below!

 

Why "Error Operation aborted" on Disk 1 ??   What Operation?  What Error?   Nothing in the logs!

 

Also: Why have Disks 10, 12, and 13 now lost their 'Build up-to-date' green ticks, when there have been NO write activity on these disks?  Do I need to build them again?

 

12 hours ago, nerbonne said:

on the surface this plugin seems to be like a good idea, but since there is an inode limit, it can't even protect the drives i actually care about.  i'm removing it and will stop following this thread because plugin serves no useful purpose.

Couldn't agree more. A working File Integrity Plugin is indeed a good idea.   I've really persevered with this so far for days, but the issues and questions keep coming.

 

Unfortunately, it seems from the lack of help on this forum, that the plugin is no longer supported.  Perhaps the developer gave up, with issues they couldn't fix?

 

I guess to uninstall it, you first use the manual "Remove" button to delete all the file hash extended attributes, then uninstall?

With the experiences so far with this plugin, I'm kinda nervious to see what happens if I click "Remove". 

 

A warning should be posted somewhere that this plugin is no longer supported, and appears to have numerous issues.

Can you run Dynamix Integrity Checker on drives not in the array? I certainly can't see them?

Something that I have learned recently is that 16GB is just not enough for me to run this plugin.   I had tried a few times to run manual build on an 8TB drive only to find several hours later the unraid webgui completely stopped running.  I could still SSH into the machine and my container based services were still running fine.  But there was nothing I could do to revive the webgui.   Eventually I tracked it down to running out of system memory during the file scanning phase of the integrity plug in.  I stopped most of my containers to free up the maximum available memory, but it was still not enough.  I upgraded to 32GB and the problem went away.   

 

So if you run into weird unraid gui behaviors (missing drive temp data, pools disappearing, general sluggishness) that ultimately leads to the webgui page refusing to load at all after starting a File Integrity build, it could very well be that you need more system memory for the plugin to operate.

Is this plugin still recommended?

 

There have been quite a few recent negative posts about it; and I am wondering if I should give it a go; if it still works on other people's systems; or if there is an alternative please?

 

Thank you

On 4/27/2025 at 12:53 AM, Zee-Unraid said:

Is this plugin still recommended?

 

There have been quite a few recent negative posts about it; and I am wondering if I should give it a go; if it still works on other people's systems; or if there is an alternative please?

 

Thank you

Well, I've persevered with it, despite all the issues so far.

 

But, last week I got my first verify alert! - "Found 1 file with BLAKE3 hash key corruption"

 

Interestingly, this reported a corruption to a file in my Backup share, on a file I had backed up just a few weeks ago.

This was great, as I still had the original file on my PC.

 

So, I copied the so called "corrupt" file from unRAID back to my PC, and did a Binary File Compare with the original file that was still on my PC.

Well, interestingly, you guessed it:  FC: No differences encountered!

 

So, this was a bogus "file corruption" alert.

 

Makes me question that if it can alert for a file that isn't corrupt (is identical to it's original copy), then what confidence do I have that a file that has actually been corrupted, actually gets detected?

 

I think I persevere out of amusement.  Like, how bad can this plugin get?  To be fair, most of my issues to date have been user interface related.  I was just hoping that the underlying code would reliably do it's function.  🙄

2 hours ago, KoolKiwiNZ said:

Well, I've persevered with it, despite all the issues so far.

 

But, last week I got my first verify alert! - "Found 1 file with BLAKE3 hash key corruption"

 

Interestingly, this reported a corruption to a file in my Backup share, on a file I had backed up just a few weeks ago.

This was great, as I still had the original file on my PC.

 

So, I copied the so called "corrupt" file from unRAID back to my PC, and did a Binary File Compare with the original file that was still on my PC.

Well, interestingly, you guessed it:  FC: No differences encountered!

 

So, this was a bogus "file corruption" alert.

 

Makes me question that if it can alert for a file that isn't corrupt (is identical to it's original copy), then what confidence do I have that a file that has actually been corrupted, actually gets detected?

 

I think I persevere out of amusement.  Like, how bad can this plugin get?  To be fair, most of my issues to date have been user interface related.  I was just hoping that the underlying code would reliably do it's function.  🙄

Yeah, some of us have mentioned things like this for years now.  I don't know why anyone still uses it.  It clearly doesn't work properly, rendering it pointless to use.

I think this plugin is partly the reason why my flash drive is erroring and is starting to fail in less than a year, all because all the hashes and logs and whatnot are being stored on it.

 

Any reason not to have the option to change the location of all the files this plugin generates to the SSD cache?

So this plugin was on my micro project list for some time now and after having looked through the recent posts in this thread... Good grief. I'm seeing:

* Confusion around the setup and configuration of the plugin
* A non-intuitive UI (I definitely agree)
* Resource demanding (which I imagine any hash integrity tool will be with a massive file collection)
* Unreliable error reporting (this is the straw that broke it for me - if error notifications are invalid then why use this tool)

Is there a reliable alternative or are there plans from the LimeTech Team to release a reliable tool in the future? Is this tool even still actively being developed by @bonienl


 

Edited May 20, 20251 yr by xDmOx

I've been using this plugin foreverish (some time in 2016 was the first file date). My /boot drive is now filled to the point where I can't install the unraid 7.1.3 update.

Looking at disk space usage on my flash drive, FIP is one of the largest users at > 1GB of storage. Looking back at the last 2 years of comments, there seem to be lots of problems and little support/resolution. I've never had FIP tell me that a file was corrupted, so... It's time for FIP to go.

I've just re-installed the file integrity plugin, as it had disappeared for some reason! However, I'm not able to select blake3 as the hashing method. I understand that it's disabled if the processor doesn't support it, but I still have the blake3 hashes from when it was previously installed so I can only assume the CPU does support it! It's an Intel 12600k. Is there another reason for blake3 to be disabled?