Jump to content
bonienl

Dynamix File Integrity plugin

779 posts in this topic Last Reply

Recommended Posts

Started 3 build operations for disks 1, 2 and 3 (see first picture)

 

After a few minutes canceled disk 3, both disk 1 and 2 continue to work (see second picture)

 

You might want to clear your browser cookies, and start fresh, perhaps that helps?

 

Also try the latest version of today. Several corrections have been made in the past days.

file-build-3.png.49dbd6a83fcbc67ee1c9f8d871151c7d.png

file-build-2.png.edbb84dd509513078e180566535a73a3.png

Share this post


Link to post

Can o please repeat your test but canceling  disk1 instead of 3?  I'm not close to a server right now but I believe it didn't happened if I canceled the last disk.

Share this post


Link to post

Can o please repeat your test but canceling  disk1 instead of 3?  I'm not close to a server right now but I believe it didn't happened if I canceled the last disk.

 

Good catch, canceling the first disk 1 gives your situation. :P

 

Let me check and correct that ...

Share this post


Link to post

Made a correction and fixed the incorrect display update when a running disk operation is canceled. There is a mistake corrected too in the process termination logic (it didn't always find the corresponding pid).

 

Please update to version 2016.01.01b if you want to use the Cancel functionality on the control page.

 

 

Share this post


Link to post

I am getting the following from my unRAID terminal on upon reboot:

unRAID login: /etc/inotifywait.conf: line 3: /mnt/disk2: Is a directory

 

I am guessing it has to do with this plugin?

Share this post


Link to post

I am getting the following from my unRAID terminal on upon reboot:

unRAID login: /etc/inotifywait.conf: line 3: /mnt/disk2: Is a directory

 

I am guessing it has to do with this plugin?

 

Yes, but wondering why the message appears. Can't find a clear reason.

 

Can you show the content of this file on your system (below is mine):

# cat /etc/inotifywait.conf
cmd="a"
method="-b2"
exclude=""
disks="/mnt/disk1 /mnt/disk2 /mnt/disk3 /mnt/disk4 /mnt/disk5 /mnt/disk6 /mnt/disk7 /mnt/disk8 /mnt/disk9 /mnt/disk10 /mnt/disk11"

 

Share this post


Link to post
# cat /etc/inotifywait.conf

method=-b2

exclude=

disks=/mnt/disk1 /mnt/disk2

 

I also got an email from my server. Same message but a little more info:

Subject:

cron for user root /usr/bin/run-parts /etc/cron.daily 1> /dev/null

Body:

/etc/inotifywait.conf: line 3: /mnt/disk2: Is a directory

Share this post


Link to post

# cat /etc/inotifywait.conf

method=-b2

exclude=

disks=/mnt/disk1 /mnt/disk2

 

I also got an email from my server. Same message but a little more info:

Subject:

cron for user root /usr/bin/run-parts /etc/cron.daily 1> /dev/null

Body:

/etc/inotifywait.conf: line 3: /mnt/disk2: Is a directory

 

In the latest version I have quoted the values of the conf file, e.g. disks="/mnt/disk1 /mnt/disk2". This is needed when spaces are present.

 

You may want to upgrade. As an extra measure I suppress error readings.

 

Thanks for testing.

 

Share this post


Link to post

In the latest version I have quoted the values of the conf file, e.g. disks="/mnt/disk1 /mnt/disk2". This is needed when spaces are present.

 

You may want to upgrade. As an extra measure I suppress error readings.

 

Thanks for testing.

 

Thanks, I was 1 version off  ;D Latest version added the "".

Share this post


Link to post

I'm trying to upgrade from 2016.0.01a to 2016.01.02 but receive the following error:

 

plugin: bad file MD5: /boot/config/plugins/dynamix.file.integrity/dynamix.file.integrity.txz

 

Share this post


Link to post

There appears to be something wrong in latest version, build works ok but trying to run a check completes without any file being verified, syslog fills with file not found errors, file names on log are incomplete, probably the cause of the problem.

Share this post


Link to post

There appears to be something wrong in latest version, build works ok but trying to run a check completes without any file being verified, syslog fills with file not found errors, file names on log are incomplete, probably the cause of the problem.

 

You may want to use the "really" latest version which is 2016.01.03.

 

Can you check the format of your export files, is it still in the old format with 5 fields separated by a | character, or is it the new format which is 2 fields separated by a * character.

 

All commands now expect the new format and are not able to read the old format.

 

Thanks for testing.

 

Share this post


Link to post

I'm trying to upgrade from 2016.0.01a to 2016.01.02 but receive the following error:

 

plugin: bad file MD5: /boot/config/plugins/dynamix.file.integrity/dynamix.file.integrity.txz

 

This may be a bad download. There is a newer version available now. Do a "Check for updates" and do an installation. Should go all fine.

 

Thanks for testing.

Share this post


Link to post

Updated but still not working, I’m only using new format only, from a quick test:

 

Exported file:

fe893c472a452790f85f7bb720fea856 */mnt/disk4/test/james.corden.2015.11.12.saoirse.ronan.hdtv.x264-crooks.mp4
b2acaee8d6152c5fbaf27ad1abca9074 */mnt/disk4/test/james.corden.2015.11.16.natalie.dormer.hdtv.x264-crooks.mp4
61b6e399a721af7aa6ee53e55e5e05b8 */mnt/disk4/test/james.corden.2015.11.17.ethan.hawke.hdtv.x264-crooks.mp4
4e66607e129855e777f68c7c40e128cd */mnt/disk4/test/james.corden.2015.11.18.anthony.mackie.hdtv.x264-crooks.mp4
52b644ea8298b38b2891c7d89339db90 */mnt/disk4/test/james.corden.2015.11.19.bryan.cranston.hdtv.x264-crooks.mp4

 

From syslog after pressing check:

Jan  3 10:22:53 Testv6 bunker: warning: 5.11.12.saoirse.ronan.hdtv.x264-crooks.mp4 is missing
Jan  3 10:22:53 Testv6 bunker: warning: 5.11.16.natalie.dormer.hdtv.x264-crooks.mp4 is missing
Jan  3 10:22:53 Testv6 bunker: warning: 5.11.17.ethan.hawke.hdtv.x264-crooks.mp4 is missing
Jan  3 10:22:53 Testv6 bunker: warning: 5.11.18.anthony.mackie.hdtv.x264-crooks.mp4 is missing
Jan  3 10:22:53 Testv6 bunker: warning: 5.11.19.bryan.cranston.hdtv.x264-crooks.mp4 is missing

 

Share this post


Link to post

Can you give me a step by step overview how to produce?

 

I just started a 'check' here and works as expected.

 

Share this post


Link to post

Select a disk and press build, wait for build to finish.

 

Select same disk and press check.

 

I even tried with simpler file names just in case and got same result:

 

disk4.export.hash

fe893c472a452790f85f7bb720fea856 */mnt/disk4/test/1.mp4
b2acaee8d6152c5fbaf27ad1abca9074 */mnt/disk4/test/2.mp4
61b6e399a721af7aa6ee53e55e5e05b8 */mnt/disk4/test/3.mp4
4e66607e129855e777f68c7c40e128cd */mnt/disk4/test/4.mp4
52b644ea8298b38b2891c7d89339db90 */mnt/disk4/test/5.mp4

 

syslog

Jan 3 10:39:43 Testv6 bunker: added 5 files from /mnt/disk4 with mask *. Duration: 00:00:06 Average speed: 329. MB/s
Jan 3 10:39:49 Testv6 bunker: warning: is missing
Jan 3 10:39:49 Testv6 bunker: warning: is missing
Jan 3 10:39:49 Testv6 bunker: warning: is missing
Jan 3 10:39:49 Testv6 bunker: warning: is missing
Jan 3 10:39:49 Testv6 bunker: warning: is missing

 

Share this post


Link to post

Can you try:

 

build - export -check

 

Build means the extended attributes are created.

Export means extended atrributes are exported to a file

Check means export file is used to check against calculated hash of file(s)

 

Also: after upgrading to the latest version, you need to stop/start the protect function on the settings page, in order to make the update work.

Share this post


Link to post

OK, believe I found something, check works on hashs done with BLAKE2, fails on hashs done with MD5 or SHA256.

 

Ok, need to investigate that (I am using BLAKE2 all the time).

 

Thanks for testing and helping out.

 

Share this post


Link to post

Well a 'stupid' mistake from my side. The key lengths of MD5 and SHA256 were mixed up. Causing both methods to use the wrong key size when performing their operations.

 

I've made a correction for this, plus some more updates. See the release notes of version 2013.01.03a

 

Please update to this version.

 

 

Share this post


Link to post

That was it, thanks.

 

One little suggestion if you don’t mind, at the moment when the check is finish it only displays “Check finished”, I’d like to request more info, for example you could take the info displayed during a check and modify it a litle to display at the end:

 

Check finished, processed xxx of xxx files. Skipped: 0 files. Found: 0 mismatches, 0 corruptions

 

I believe you’re taking a simpler approach and just notifying in case of errors but I would feel better having more info of what was actually checked in the end.

 

Share this post


Link to post

One little suggestion if you don’t mind, at the moment when the check is finish it only displays “Check finished”, I’d like to request more info, for example you could take the info displayed during a check and modify it a litle to display at the end:

 

Sure, that is possible and available. Please update to version 2016.01.04.

 

Below a screenshot of what it looks like.

operation-progress.png.e10d712c47ea5bc7bfc5323322fd71f5.png

Share this post


Link to post

Really excited for this - initial check nearly done on my i5 server (3 4Tb data disks).  Is it normal for a lower specd server to become fully unresponsive during the initial check (specs in signature)

Share this post


Link to post

Really excited for this - initial check nearly done on my i5 server (3 4Tb data disks).  Is it normal for a lower specd server to become fully unresponsive during the initial check (specs in signature)

 

If you started all three disks simultaneously, it can introduce a full load on your processor. Alternatively you can build one disk at the time to lower the load, but it will take longer to complete all disks.

 

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.