Dynamix File Integrity plugin


bonienl

Recommended Posts

I uninstalled this plugin, but today is the day I had it scheduled to run (the first of the month) and my server is going crazy, so I highly suspect it's still running.

 

I found this in my system log today:

 

Mar 1 05:44:39 Tower inotifywait[5734]: Failed to watch /mnt/disk4; upper limit on inotify watches reached!

Mar 1 05:44:39 Tower inotifywait[5734]: Please increase the amount of inotify watches allowed per user via `/proc/sys/fs/inotify/max_user_watches'.

 

Does that mean it is in fact running? If so, how do I disable it? Do I need to install it again and clear the schedule or something?

Link to comment
On 12/29/2015 at 10:04 AM, bonienl said:

There is no need to stay at the same page or keep the browser open all the time. Whenever the control page is revisited from the same browser, it will update itself to the latest state.

How can I view the status of the task when using another browser? I initiated the check on my phone but I want to view the status on my laptop and my PC.

Link to comment
3 hours ago, Rendeaust said:

How can I view the status of the task when using another browser? I initiated the check on my phone but I want to view the status on my laptop and my PC.

Status view is limited to the same browser (system) it was started from.

  • Upvote 1
Link to comment

I ran File Integrity today. Long story short, power went down today and I want to see if that caused file corruption. If it got the hashes from when the array was started earlier this morning, then those hashes are useless. Hashes should be from yesterday. It all went well but I want to know what date it got the hashes are from?

 

Also PS. I'm still confused what's the difference between build and export. Can someone ELI5? Thanks a lot.

Link to comment
  • 3 weeks later...

Greetings ..

Using the File Integrity plugin on 6.6.7

Impressive work..

 

Have been doing a fair amount of testing to try and get it to fail / see where it misses stuff.

E.g. Parity check running, 6 data drives monitored, parallel transfer of 5GB files from ramdisk to each of the monitored disks.

 

Everything passed, as said before: Impressive.

 

There are a number of changes / additions I would like to see, but realise that they would only apply to my use-case, and it would not make sense to ask for them to be implemented.

 

There are two possible changes that I would ask for that would / could be of benefit to the community as a whole (possibly).

 

1. An option to have the log files directory elsewhere (cache etc.) as I am uncomfortable with such potentially frequently changed files stored on the flash drive, I suspect there will be some who will opine that is nothing to worry about (wear leveling, flash drives are better quality nowadays etc. etc.), but if I am using the plugin as an aid to monitor / reduce the effects of bit-rot / failure then it makes sense to avoid the relatively fragile flash drive (in my opinion).

 

2. A means of triggering / running a script when processing has completed, enabling reliable post-processing.
In my case I would use that script to perform the unique processing for my use-case.


I hope you can consider this request.

If only one can be implemented, please can it be option 2 (end-of-processing script triggering).

 

Cheers..

Link to comment

I'm see the following error in my log, but no reference to any bad files.

 

BLAKE2 hash key mismatch,  is corrupted

 

This was the only entry in the log.  Is there somewhere else I need to check?

 

Also, if we do see errors, do we need to reformat the file or just recopy the indicated files?  I do see errors on a couple of the other disks with file names indicated.

 

Thanks in advance!

Edited by jeffreywhunter
Link to comment
  • 2 weeks later...
12 hours ago, nuhll said:

Where i can see which files are bad?

In the syslog. If you enable notifications and have e-mail notification set up you also get a list of all the files by mail. And the files aren't necessarily "bad", the files could have just been updated and thus changed the hash. So it's a good idea to exclude files that gets updated a lot by various apps, like .nfo files.

Link to comment
  • 2 weeks later...
  • 1 month later...

How does one exclude files based on extension? I.e.

*.log *.nfo *.tgz

(the box in the config form doesn't say anything about what the syntax should be)

 

In rsync I often use

--min-size=1 --no-specials --no-devices --no-links

is there a way to also exclude these from the file integrity checks?

Edited by fluisterben
Link to comment
  • 2 weeks later...

Is there anything to be concerned about here? Anything I should do

SHA256 hash key mismatch (updated), /mnt/disk2/domains/Ubuntu/vdisk1.img was modified
SHA256 hash key mismatch (updated), /mnt/disk2/domains/UnR-Win-10/vdisk1.img was modified
SHA256 hash key mismatch (updated), /mnt/disk1/back_ups/libvert_back_up/libvirt.img was modified
SHA256 hash key mismatch (updated), /mnt/disk1/system/docker/docker.img was modified
SHA256 hash key mismatch (updated), /mnt/disk1/system/libvirt/libvirt.img was modified
SHA256 hash key mismatch (updated), /mnt/disk1/system/libvirt/libvirt.img was modified

 

Link to comment
7 hours ago, Fiservedpi said:

Is there anything to be concerned about here? Anything I should do


SHA256 hash key mismatch (updated), /mnt/disk2/domains/Ubuntu/vdisk1.img was modified
SHA256 hash key mismatch (updated), /mnt/disk2/domains/UnR-Win-10/vdisk1.img was modified
SHA256 hash key mismatch (updated), /mnt/disk1/back_ups/libvert_back_up/libvirt.img was modified
SHA256 hash key mismatch (updated), /mnt/disk1/system/docker/docker.img was modified
SHA256 hash key mismatch (updated), /mnt/disk1/system/libvirt/libvirt.img was modified
SHA256 hash key mismatch (updated), /mnt/disk1/system/libvirt/libvirt.img was modified

 

No concern, those file change time to time, best ignore them in FIP. Update / check their hash just waste resources.

Edited by Benson
  • Like 1
Link to comment
On 5/26/2019 at 9:33 PM, fluisterben said:

How does one exclude files based on extension? I.e.

*.log *.nfo *.tgz

(the box in the config form doesn't say anything about what the syntax should be)

 

In rsync I often use

--min-size=1 --no-specials --no-devices --no-links

is there a way to also exclude these from the file integrity checks?

add it in " custom files: " field in form " *.log, *.nfo, *.tgz "

Link to comment

Dont know if this has been asked before, but would it be possible to set the location of the export, logs, and saved folders? (mainly the export folder). I dont like the idea of having the entire list of files/hashes stored in plain text on the flash drive. I'd much rather have the option to store it somewhere on my encrypted cache/array and just exclude it. 

 

I thought I could maybe just create a folder symlink for those folders, but you can't create symlinks on a FAT32 formatted (flash) drive.

  • Upvote 1
Link to comment
On 6/8/2019 at 9:55 AM, shiarua said:

Dont know if this has been asked before, but would it be possible to set the location of the export, logs, and saved folders? (mainly the export folder). I dont like the idea of having the entire list of files/hashes stored in plain text on the flash drive. I'd much rather have the option to store it somewhere on my encrypted cache/array and just exclude it. 

 

I thought I could maybe just create a folder symlink for those folders, but you can't create symlinks on a FAT32 formatted (flash) drive.

Well, looking through the source, I guess I could just make some wrapper scripts around the bunker script for exporting hash files and running the verification checks. Seems like a feasible workaround. 

Link to comment

I could use another set of eyes on this issue:  Integrity check ran last night and created an bad.log export.  Listed below is the exact message:

BLAKE2 hash key mismatch,  is corrupted
BLAKE2 hash key mismatch,  is corrupted
BLAKE2 hash key mismatch,  is corrupted

 

No path or file name to reference.  I checked the SysLog and the following messages are logged:

 

Jun 10 23:34:47 Tower bunker: error: BLAKE2 hash key mismatch, is corrupted
Jun 10 23:50:56 Tower bunker: error: BLAKE2 hash key mismatch, is corrupted

Jun 11 03:47:27 Tower bunker: error: BLAKE2 hash key mismatch, is corrupted

 

Apparently there are 3 files that require attention, but no reference to the specific files.  Does anyone have any idea how I can proceed?

 

Thanks

 

 

  • Upvote 1
Link to comment

I seem to get a hash key mismatch on every file which is a hard link (at the User Share level) to another file in the same User Share.   Is this expected behaviour?    Is there anything I can do to avoid getting this error?  

 

The links are in a sub-folder of a user share - do I have to use the Custom folder option to exclude the sub-folders containing these hard links.  If so can I have a folder name of the form FILMS/!3D.

Edited by itimpi
Link to comment
  • 2 weeks later...
On 6/11/2019 at 6:06 PM, DivideByZero said:

I could use another set of eyes on this issue:  Integrity check ran last night and created an bad.log export.  Listed below is the exact message:

BLAKE2 hash key mismatch,  is corrupted
BLAKE2 hash key mismatch,  is corrupted
BLAKE2 hash key mismatch,  is corrupted

 

No path or file name to reference.  I checked the SysLog and the following messages are logged:

 

Jun 10 23:34:47 Tower bunker: error: BLAKE2 hash key mismatch, is corrupted
Jun 10 23:50:56 Tower bunker: error: BLAKE2 hash key mismatch, is corrupted

Jun 11 03:47:27 Tower bunker: error: BLAKE2 hash key mismatch, is corrupted

 

Apparently there are 3 files that require attention, but no reference to the specific files.  Does anyone have any idea how I can proceed?

 

Thanks

 

 

This is strange, because normally the file name is included in the message, but it seems the file name is empty here.

 

You could make a selective "check" per disk and try to pinpoint the file(s).

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.