1 hour ago, JustinChase said:

I couldn't find any "log tab", but there is a "log files" folder on the plugin page.  The only thing in that folder is a file called duplicate_file_hashes.txt

 

That only lists all the duplicate hashes it found, but nothing about any errors, or help figuring out which file triggered the error.

 

This is an example in log tab, it show a file "test" at /mnt/disk12 have problem, not in plugin tab (page).

 

 

And you need enable logging in plugin setting.

 

Edited April 27, 20206 yr by Benson

Ah, that helps; thank you!

 

doesn't make as much sense as being in the log folder on the plugin page, but now I know.

 

it actually shows tons of warnings and errors about SHA256 mismatches and/or missing files.

 

I may have been doing some moving of files when the process was originally running, so maybe that's the issue.  It took about a day to run, and I'm not sure when I can not use the server for that long to run it again, but I'll try to do that soon, and have it re-evaluate all files during that time and see if errors continue.

 

I just installed a brand new parity drive and system drive, both of which passed the preclear, so I'd be surprised if there was bit rot happening already.

 

I guess we'll see soon enough.

 

thanks again for your help.

9 minutes ago, JustinChase said:

it actually shows tons of warnings and errors about SHA256 mismatches and/or missing files.

That would be normal if file move to other path. You should perform export to update it.

I deleted all data, and am just rebuiling now, while I use the server, but am not moving files around this time.  We'll see how it goes.  If any issues, I'll try the export function next time.  I read all the descriptions of all the buttons, and there was no obvious 'rebuild' function I could decipher.

 

thanks again for the help.

I recently ran a dynamix file integrity test on a drive that has read errors. According to the log there are over 90 MKV files with a SHA256 hash key mismatch but even though it states that the files are corrupt they play perfectly. I used a hash file manager to rewrite the hash keys for the files but this did not work. How can I fix these files?

nvm

 

 

Edited June 11, 20206 yr by snuffy47

Hi!

 

During my build of one harddisk the green checkmark next to "Build up-to-date" appeared, but the build process is still ongoing. Should I click the cancel button?

I ran build and export on one of the discs in my server. During the export process many files were skipped and many corrupt files were found. I am unable to locate any corrupt files because no log was posted after the export process. After this,  I ran verification on disc 4. The process ran for 31 hours and no log was posted. How can I locate the corrupt files?

On 5/21/2020 at 8:42 AM, cdixon said:

I used a hash file manager to rewrite the hash keys for the files but this did not work. How can I fix these files?

I am not sure what means ( or how ) the hash file manager do it. If too much file show corrupt, then best way was "Remove" extended attributes and "Build" it again.

 

13 hours ago, cdixon said:

During the export process many files were skipped

Export won't skip file, only Import will have this warning.

 

13 hours ago, cdixon said:

How can I locate the corrupt files?

If no logging enable, then you may never know which file have hash mismatch.

 

 

Pls understand this plugin won't work perfect in some case which already discuss before and plugin should stop develop.

You need understand why wrong hash will generate for a file .... if file still in writing / transfer but FIP hash it, then wrong hash value will generate. I would simple suggest never hash file during file transfer, so best stop auto hash function ( manual execute hash or periodic during no any file in transfer ) . Otherwise, there are no meaning always "Remove" and "Build".

 

In my use case, I store large size media file always, those file are write once, so I haven't issue for those file. But something like frequency update file (i.e. subtitle) will auto-update (correct) during hash check.

I also ( advance manual operation ) export the hash to backup, every-time "Import" hash for all backup file from master. i.e. 10000+ file in backup, this also can confirm does any file in missing backup too.

 

* PS : I also use checksum plugin, so I have two ref. source for file(s) really corrupt or not. *

 

 

Edited August 11, 20205 yr by Benson

20 hours ago, cdixon said:

How can I locate the corrupt files?

See the setting "Error logging" under Settings -> File Integrity

3 minutes ago, bonienl said:

See the setting "Error logging" under Settings -> File Integrity

That's the problem. In the settings, I have Error logging set to Syslog and Output file but I am still unable to find the corrupt files with this option.

@bonienl

 

I ran a manual check on a drive and got a lot of BLAKE2 hash key mismatches (mostly .nfo files modified).  I received the details in a notification email, but there's no log file in /boot/config/plugins/dynamix.file.integrity/logs , even though I do have error logging set to "Syslog & Output File".  I also have log files and email notifications from 2018/2019, so I know it was working correctly at some point.  

 

Are log files not created when the check is run manually (as opposed scheduled daily/weekly/monthly)?  I have 14GB free on my flash drive, so it's not an issue of usable space.  What do you suggest?

Edited August 19, 20205 yr by nukem492

I  am interested in starting using this plugin, but i have some questions:

 

1. If the hashing reults are NOT saved to flash, where are they stored?

2. Is it possible to store hash results log to another path then flash? I have for example used fstab to statically mount a smb share on a remote logserver which holds all my logs, i would want the logs to go directly there instead of moving them from the flash each day with a script (avoiding writing as much as possible to flash) is this possible or do i need to modify the plugin to do this?

Are folder exclusions recursive?

So for example if I exclude folder _abc, will that exclude /mnt/user/x/_abc/some_file.zip ?

If not, how can I achieve this, thanks.

Hi, thanks for the plugin, just a minor bug I noticed when running the Find Duplicates command, it says it's checking Disk 8, I've actually removed that disk from the array as it was faulty. Screenshot:

1 hour ago, dgwharrison said:

Hi, thanks for the plugin, just a minor bug I noticed when running the Find Duplicates command, it says it's checking Disk 8, I've actually removed that disk from the array as it was faulty. Screenshot:

It sort/check all hash files, it won't check actual disk exist or not. You can simple delete disk8 hash file.

Edited September 12, 20205 yr by Benson

Hi, just wondering how I reset the File Integrity plugin? I deleted the hash files however that seems to have procuded unexpected behaviour... (my bad).

Deleting the hash files should have no effect, so what is the unexpected behaviour?

Sorry @itimpi just realised I forgot to export. 🤦‍♂️

 

Edited September 15, 20205 yr by dgwharrison

I searched the thread and didn't find an answer so apologies if this question's been asked.

 

I'm new to this plugin and have my first verification running after the initial check. It's been going for around 8 hours and I'm wondering how far it's progressed. Is there an indicator somewhere? The initial check showed a handy progress bar in the page under Tools.

 

I'm also confused about whether "check" and "verify" are used interchangeably in the documentation. The help says Use the Check command to verify files against a previously exported file but the plugin can be configured to verify with Save new hashing results to flash (which I assume is the referenced "export") disabled, suggesting they're not interchangeable.

Edited September 23, 20205 yr by CS01-HS
typo

got a question

i had a few blake2  corruptions now i replaced those files. and did a normal check which says ok. SO i did a check with the plugin again it took 19 hours to do all 45k files. The check says finish and it has less corruptions now but when i check the disk2.export.20201011.bad.log

it still shows the old count and the date is also not updated on it. While it finsh the check files this morning.

 

Is there something i missed why it didnt updated this bad.log file? i also dont see any other with a newer date.

Found 25 files with BLAKE2 hash key mismatch
warning
BLAKE2 hash key mismatch (updated), blah/blah/aom-u-move/kowai/catacodec.data was modified
BLAKE2 hash key mismatch (updated), blah/blah/aom-u-move/kowai/local_files.data was modified

 

i had in the plugin excluded teh aom-u-move directory but it still shows the directory under that.  these files change often zo i excluded the whole directory but that didnt seems to work.

 

3 hours ago, KoNeko said:

Found 25 files with BLAKE2 hash key mismatch
warning
BLAKE2 hash key mismatch (updated), blah/blah/aom-u-move/kowai/catacodec.data was modified
BLAKE2 hash key mismatch (updated), blah/blah/aom-u-move/kowai/local_files.data was modified

 

i had in the plugin excluded teh aom-u-move directory but it still shows the directory under that.  these files change often zo i excluded the whole directory but that didnt seems to work.

 

If file already hashed before exclusion, you need clear its attribute. You can clear it manually or edit the export file then clear all by GUI and import again.

19 hours ago, Vr2Io said:

If file already hashed before exclusion, you need clear its attribute. You can clear it manually or edit the export file then clear all by GUI and import again.

sounds a very big hassle? How to do that ? or is there a other way.

@bonienl

I started using this plugin on a different array the other day and I think I found a bug with the reporting mechanism for the Export being up-to-date with the Build.

 

When mid-Build, I accidently hit the Export button the the webUI with all the disks selected.  The build continued, but an export was also done on all the devices in my array with the current files where hashes were computed.  I figured, these would almost certainly contain incomplete data, but I'd make a new export when the build completed the next day.  Before I did however, I wanted to see if the Export was incomplete AND the webUI reported that the state was stale.  When checking the export files, I was correct that the data was incomplete and not all the hashes were in the export files.  BUT, the webUI greeted me with all green check marks on the export status for each device.

 

This left me with a bigger question.  How does the plugin determine if the build and export are up-to-date?