@Vr2Io

This didn't work, however I'm guessing I need to server restart for this to take effect. I'm also in the middle of a parity check. I'll have to try again tomorrow after the parity ends.

Thanks for responding, it looks like there are a few people raising this issue.

4 hours ago, Vr2Io said:

 

Sample check files by "getfattr -d <file>"

Thank you.

I did check one of the files:

getfattr -d noiseman_sound_insect.avi

And it looks like the blake3 checksum is still there (see screenshot below).

But I did follow the instructions:
"for those who want to uninstall this plugin,
 before uninstalling the plugin itself you need to remove the calculated hashes,
 this is done by the "Clear" operation (select all disks and then Clear, see also the built-in Help)."
And after I did this I removed the plugin and deleted the folder /config/plugins/dynamix.file.integrity

 

Edited September 25, 20223 yr by DataCollector

14 minutes ago, DataCollector said:

Thank you.

I did check one of the files:

getfattr -d noiseman_sound_insect.avi

And it looks like the blake3 checksum is still there (see screenshot below).

But I did follow the instructions:
"for those who want to uninstall this plugin,
 before uninstalling the plugin itself you need to remove the calculated hashes,
 this is done by the "Clear" operation (select all disks and then Clear, see also the built-in Help)."
And after I did this I removed the plugin and deleted the folder /config/plugins/dynamix.file.integrity

 

 

Strange, clear operation haven't clear extend attribute. If you want clear it, pls try install FIP again ( set blake3 ) for that or left those attribute there should be no hurt.

42 minutes ago, Vr2Io said:

 

Strange, clear operation haven't clear extend attribute. If you want clear it, pls try install FIP again ( set blake3 ) for that or left those attribute there should be no hurt.

Yes. Strange.

 

Now I installed FI again,
"Automatically protect new and modified files": Enabled
Method: "BLAKE3"
<Apply> (see screenshot 1)
==
Then I spun up the first 5 Disks of my '1st System' (look at signature) manually and started in FI "Clear" for those Disks #1-5.
(see screenshot 2)
==
FI toook around 1 Second and then showed in green "cleared" (see screenshot 3)
==
And I then looked again at the attributes and Blake3 is there.
(see screenshot 4)

 

I am very sad.

 

 

Pls try

 

/bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/bunker -r -b3 /mnt/diskX

 

or

 

/bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/bunker -R -b3 /mnt/diskX

 

, where X is disk no.

 

 

/bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/bunker -h , would got bunker help

 

Edit : Overlook you should use remove instead clear

 

Edited September 25, 20223 yr by Vr2Io

Unfortunately this plugin isn't working correctly for me. Today it gave me a notification saying that 82 files on my array were corrupt. When I checked the files deemed corrupt by the plugin the hashes matched up to the values in the extended file attributes, so I have no idea where the plugin got the idea that my files were corrupt. It honestly shows how unreliable the plugin is when these "corruption" notifications don't even faze me in the slightest.

On 9/25/2022 at 2:03 PM, DataCollector said:

Yes. Strange.

 

Now I installed FI again,
"Automatically protect new and modified files": Enabled
Method: "BLAKE3"
<Apply> (see screenshot 1)
==
Then I spun up the first 5 Disks of my '1st System' (look at signature) manually and started in FI "Clear" for those Disks #1-5.
(see screenshot 2)
==
FI toook around 1 Second and then showed in green "cleared" (see screenshot 3)
==
And I then looked again at the attributes and Blake3 is there.
(see screenshot 4)

 

I am very sad.

 

 

 

 

EDIT: it seems "Clear" is the wrong command.   Today I tried "Remove" and that command did work on the disks for some time.

 

I wanted to added ".Recycle.Bin,backup/SQL" to the excluded folders and i accidentally hit my enter key. Doing so resets all settings to default.

Hi,

 

I have this plugin installed with Automatically protect new and modified files: enabled. but when I check new files with "getfattr -d <file>" the hash information is missing until I go and run the build again manually. Am I doing something wrong or can I check if it's ruining somehow?

I've been trying to track down why so many NEW files do not have a hash. I get emails every single time due to either hash mismatch (usually on nextcloud.log despite excluding *.log files) or hash missing altogether. I dug into the source code of the plugin and replicated the inotifywait command so that i could watch for myself what was going on.

 

inotifywait -mr -e close_write --format '%w%f' /mnt/disk1 /mnt/disk2

and after running mover here is what i see

/mnt/disk2/documents/ubuntu-18.04.1-server-amd64.iso.partial
/mnt/disk1/backup/unraid/STORAGE/flash/config/editor.cfg.partial
/mnt/disk1/backup/unraid/STORAGE/flash/config/super.dat.CA_BACKUP.partial
/mnt/disk1/backup/SQL/mariadb/vmosa/2022-11-12-08.00.01.sql.tgz.partial
/mnt/disk1/backup/SQL/postgres11/airsonic/airsonic_2022-11-12-08.05.02.sql.tgz.partial
/mnt/disk1/backup/SQL/postgres11/authelia/authelia_2022-11-12-08.05.03.sql.tgz.partial
/mnt/disk1/backup/SQL/postgres11/family_photos_dev/family_photos_dev_2022-11-12-08.05.03.sql.tgz.partial
/mnt/disk1/backup/SQL/postgres11/family_photos_prod/family_photos_prod_2022-11-12-08.05.04.sql.tgz.partial
/mnt/disk1/backup/SQL/postgres11/film_convert_dev/film_convert_dev_2022-11-12-08.05.04.sql.tgz.partial
/mnt/disk1/backup/SQL/postgres11/film_convert_prod/film_convert_prod_2022-11-12-08.05.05.sql.tgz.partial
/mnt/disk1/backup/unraid/STORAGE/flash/config/plugins/dynamix.file.manager/dynamix.file.manager.txz.partial
/mnt/disk1/backup/unraid/STORAGE/flash/config/plugins/dynamix.file.manager.plg.partial
/mnt/disk1/backup/unraid/STORAGE/flash/config/plugins/unassigned.devices.plg.partial
/mnt/disk1/backup/unraid/STORAGE/flash/config/plugins/dynamix.file.integrity/disks.ini.partial

As you can see, every file moved by mover ends with .partial and if i run getfattr on the actual file (since there is no .partial file)

root@storage:~# getfattr -d /mnt/disk1/backup/SQL/postgres11/airsonic/airsonic_2022-11-12-08.05.02.sql.tgz
root@storage:~# 

When I navigate into the airsonic backup folder and run getfattr on any file in there (because backup always saves to cache and then mover moves) getfattr -d airsonic* is blank for all files.

I restarted the plugin after months of pause and I face the same issue.

I tried to exclude .partial files in the hope that the plugin will skip it and only consider the real one once the transfer is done.

I'll report back.

It won't. The move script that mover uses is not standard, its custom written and seems to copy to .partial, deletes the original and then renames the partial to the real name. That renaming does not trigger inotifywait.

Has anyone else ever gotten this error?

 

clean install, getting this error when doing an initial build:

 

 Finished - added 0 files. Duration: 00:00:01

 

I had the plugin installed before and it successfully did an initial build and export.  But after upgrading the plugin I began getting this error.  I removed the plugin, deleted the plugin folder under \boot\config\plugins and rebooted, but still getting the error.  Logs are empty.

 

 

Just want to jump in here too, I get the same thing. I manually this on all my disks to remove previous hashes:

/bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/bunker -r -b2 /mnt/diskX
/bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/bunker -r -b3 /mnt/diskX 

Removed all old preferences and started fresh but every time I hit build its completes immediately with the same log as the previous post.

 

For now the plugin stays disabled. I'm not quite sure what's going on.

 

EDIT:

it also looks like "bunker -R -b3 /mnt/diskX" is broken? it only clears the hash but not the other "user." attrs looking in to the script. Wonder if the new unraid version is causing issues?

Edited February 16, 20233 yr by JimmyGerms

Plugin just call bunker to perform different task, pls check does file permission issue, try use tools - NEW PERMISSIONS to fix that and re-run bunker.

 

Daily add & verify file ( bunker CLI ) without issue.

 

 

 

3 hours ago, JimmyGerms said:

it only clears the hash but not the other "user."

This normal.

Edited February 16, 20233 yr by Vr2Io

I've run "New Permissions" not too long ago and everything matches what the help docs have for those Shares.

 

Looking at this code, I'm reading it that if the "user.scandate" attr is empty then using -R should remove the three extra attrs but it doesn't seem to...Not sure where my setup is going wrong.

      if [[ -z $userdate || $((userdate/86400)) -le $epoch ]]; then
        ((count++))
        setfattr -x user.$hash "$file" 2>/dev/null
        [[ $cmd == R ]] && setfattr -x user.scandate "$file" 2>/dev/null
        [[ $cmd == R ]] && setfattr -x user.filedate "$file" 2>/dev/null
        [[ $cmd == R ]] && setfattr -x user.filesize "$file" 2>/dev/null

 

EDIT: Yeah, not seeing anything wrong in the code. Even if the user.scandate was 0 it still should respect the flags...What's going on with my system then. Cannot get this to kick off at all anymore.

Edited February 16, 20233 yr by JimmyGerms

Ok, rolled back the plugin to this version and it's working again...

 

https://raw.githubusercontent.com/bergware/dynamix/fbd863f64ce1af8ea97ac2d13cc906a410925b9f/unRAIDv6/dynamix.file.integrity.plg

11 hours ago, JimmyGerms said:

Ok, rolled back the plugin to this version and it's working again...

 

https://raw.githubusercontent.com/bergware/dynamix/fbd863f64ce1af8ea97ac2d13cc906a410925b9f/unRAIDv6/dynamix.file.integrity.plg

Nice find, will try that as I am also getting the "0 files added" and no log files created....

Edited February 17, 20233 yr by Kretsche
Edit: Does not work for me as its not compatible with Unraid 6.11.5

On 2/17/2023 at 3:21 AM, Kretsche said:

Nice find, will try that as I am also getting the "0 files added" and no log files created....

I'm on 6.11.5 too. The trick for me was I downloaded the corresponding txz with that version of the plugin and placed it on the flash drive. Next I manually installed the plugin by using that link and it worked. Here's the link to the .txz for ya!

https://github.com/bergware/dynamix/raw/fbd863f64ce1af8ea97ac2d13cc906a410925b9f/archive/dynamix.file.integrity.txz

 

EDIT: Honestly, it's still not functioning right. New files don't get hashed even though I have the setting "save new files hashes to flash" enabled. Think overall I'm just going to remove this and write a little script to remove left over hashes that are stuck on files.

Edited February 19, 20233 yr by JimmyGerms

New 6.11.5 Unraid install with the plugin freshly installed and the build command is not doing anything for me, it keeps repeating that 0 files have been added even though the disk is not empty

Same issue as above, no files being added. 6.11.5

Unlike Jimmy, I was unable to revert to a previous version;

 

plugin: installing: dynamix.file.integrity.plg
Executing hook script: pre_plugin_checks
plugin: downloading: dynamix.file.integrity.plg ... done

Executing hook script: pre_plugin_checks
plugin: skipping: inotify-tools-3.14-x86_64-1.txz - Unraid version too high, requires at most version 6.2.9
plugin: skipping: b2sum-20130305-x86_64-1.txz - Unraid version too high, requires at most version 6.3.9

+==============================================================================
| Skipping package b3sum-1.0.0-x86_64-1 (already installed)
+==============================================================================
plugin: downloading: dynamix.file.integrity.txz ... done

plugin: bad file MD5: /boot/config/plugins/dynamix.file.integrity/dynamix.file.integrity.txz
Executing hook script: post_plugin_checks

 

Hello, Same issue for me for the newer version of this plugin on UnRaid 6.11.5

 

no files been added, also tried with every hashing methods

 

No Files Added (it run when we do Build and after few minutes, it said 0 file added

No Export files (file at 0 so we can click on Export, it run but nothing been exported)...

 

it seem to just scan all files without scanning the hash

same issue no files been added - no files added etc.

fresh installed the plugin, never used before. 

 

I'm also on Unraid 6.11.5 and the updated plugin was not working properly and hashing new files added to the array.

 

I did an uninstall and rolled back to the version below.  The older version of the plugin immediately picked up and started hashing thousands of files that had been added/changed recently.

 

https://raw.githubusercontent.com/bergware/dynamix/fbd863f64ce1af8ea97ac2d13cc906a410925b9f/unRAIDv6/dynamix.file.integrity.plg

 

On 3/5/2023 at 3:51 PM, remowilliams said:

I'm also on Unraid 6.11.5 and the updated plugin was not working properly and hashing new files added to the array.

 

I did an uninstall and rolled back to the version below.  The older version of the plugin immediately picked up and started hashing thousands of files that had been added/changed recently.

 

https://raw.githubusercontent.com/bergware/dynamix/fbd863f64ce1af8ea97ac2d13cc906a410925b9f/unRAIDv6/dynamix.file.integrity.plg

 

Interesting. I'm experiencing the same issue of no files being added. I tried to install the older plugin but it comes back with a bad checksum so it won't install...