archedraft Posted September 23, 2015 Share Posted September 23, 2015 Added new extended attribute: file size We will need to re-run bunker on all our existing files to add the new extended attribute correct? bunker -u /mnt/disk1 is all that is needed to update the existing checksums? Quote Link to comment
bonienl Posted September 23, 2015 Author Share Posted September 23, 2015 Added new extended attribute: file size We will need to re-run bunker on all our existing files to add the new extended attribute correct? bunker -u /mnt/disk1 is all that is needed to update the existing checksums? bunker -u /mnt/disk1 will work, but requires to recalculate the hash value for each file, a lengthy process. There is a hidden command -T to just add file size to the extended attributes without recalculating the hash: bunker -T /mnt/disk1 will add the missing attribute. Afterwards you may want to do an export command to save all attributes including the new one in a file, e.g. bunker -e -f /boot/hashes/disk1.txt /mnt/disk1 Quote Link to comment
archedraft Posted September 23, 2015 Share Posted September 23, 2015 There is a hidden command -T to just add file size to the extended attributes without recalculating the hash: bunker -T /mnt/disk1 will add the missing attribute. Very nice, I feel so cool using hidden commands Afterwards you may want to do an export command to save all attributes including the new one in a file, e.g. bunker -e -f /boot/hashes/disk1.txt /mnt/disk1 I have been saving exports of my files on a regular basis per your advice, but I do have a question for you. Is the point of the export to compare the extended attributes checksum with the exported text file? Or what is the reasoning behind having a separate export file? Quote Link to comment
bonienl Posted September 23, 2015 Author Share Posted September 23, 2015 I have been saving exports of my files on a regular basis per your advice, but I do have a question for you. Is the point of the export to compare the extended attributes checksum with the exported text file? Or what is the reasoning behind having a separate export file? The exported file can be used for different purposes: 1. Restore the extended attributes when they went missing (import command) 2. Check the extended attributes after file moves (check command) An example: I converted my data disks from RFS to XFS, in this process I started with an empty disk formatted in XFS, then copied the contents of the next disk to the empty disk and repeated this action for all data disks. I had made an export file of each disk before starting the above procedure. After a disk was copied I run bunker -a /mnt/diskX to reapply the extended attributes (altenatively you can use a copy method which preserves the extended attributes). Next I replaced in the export file the original disk number for the new disk number to reflect the new location Next step was to check the files after the copy. i.e. bunker -c /mnt/diskX and ensure no file corruption Quote Link to comment
itimpi Posted October 2, 2015 Share Posted October 2, 2015 It looks like a small modification is required for 6.1 compatibility. Currently the script is looking for 'notify' under /usr/local/sbin. In the 6.1 release this is now at /usr/local/emhttp/scripts. Sounds like an upfront check is needed for the location of this command to handle running on different unRAID versions? Quote Link to comment
bonienl Posted October 2, 2015 Author Share Posted October 2, 2015 It looks like a small modification is required for 6.1 compatibility. Currently the script is looking for 'notify' under /usr/local/sbin. In the 6.1 release this is now at /usr/local/emhttp/scripts. Sounds like an upfront check is needed for the location of this command to handle running on different unRAID versions? Thanks for noticing. I've made an update which verifies the location of the 'notify' script and use appropriately. Quote Link to comment
toby9999 Posted October 3, 2015 Share Posted October 3, 2015 This utility looks really useful. Is there any GUI front-end (dynamix, docker??) for this? Quote Link to comment
itimpi Posted October 3, 2015 Share Posted October 3, 2015 This utility looks really useful. Is there any GUI front-end (dynamix, docker??) for this? No, not at the moment. I put the idea of creating a plugin to run this on my list of 'nice-to-try-creating' items a few weeks ago. However demands on my time have have meant I have not got any further, but it is still something I would like to try my hand at if time becomes available. Quote Link to comment
toby9999 Posted October 5, 2015 Share Posted October 5, 2015 This utility looks really useful. Is there any GUI front-end (dynamix, docker??) for this? No, not at the moment. I put the idea of creating a plugin to run this on my list of 'nice-to-try-creating' items a few weeks ago. However demands on my time have have meant I have not got any further, but it is still something I would like to try my hand at if time becomes available. It actually seems fairly easy to use via CLI, but a GUI would probably open it up to a lot more non-techie users. I'm looking around for an easy solution to protect my unraid data from data degradation. I have a full copy of every unraid hard drive on another hard drive, but there's no integrity checking, hence my interest in this solution or something else (snapraid?). Quote Link to comment
tr0910 Posted December 21, 2016 Share Posted December 21, 2016 Is anyone still using this in conjunction with the excellent dynamix plugin? I have an external server that I want to verify the rsyncing process via bunker command line after exporting the keys from the production server and doing a bunker -c -f /tmp/disk1_keys.txt Quote Link to comment
bonienl Posted December 23, 2016 Author Share Posted December 23, 2016 Is anyone still using this in conjunction with the excellent dynamix plugin? I have an external server that I want to verify the rsyncing process via bunker command line after exporting the keys from the production server and doing a bunker -c -f /tmp/disk1_keys.txt bunker version 2 comes together with the plugin Dynamix File Integrity. There is no need to install it separately. The export files of bunker v1 and v2 are compatible. Quote Link to comment
tr0910 Posted December 23, 2016 Share Posted December 23, 2016 bunker version 2 comes together with the plugin Dynamix File Integrity. There is no need to install it separately. The export files of bunker v1 and v2 are compatible. Thank-you. For those following this, it would appear that bunker no longer exists on your flash drive in executable format. It gets installed here at boot time. You can find bunker in: root@Tower:/usr/local/emhttp/plugins/dynamix.file.integrity/scripts# bunker bunker v2.5.1 - Copyright (c) 2015-2016 Bergware International Usage: bunker -a|A|v|V|u|e|t|i|c|C|r|R [-fdDsSlLnq] [-md5|-b2] path [!] [mask] -a add hash key attribute for files, specified in path and optional mask -A same as -a option with implicit export function (may use -f) -v verify hash key attribute and report mismatches (may use -f) -V same as -v option with updating of mismatched keys (may use -f) -u update mismatched or corrupted hash keys with new hash key attribute (may use -f) -e export hash key attributes to the export file (may use -f) -t touch file, i.e. copy file modified time to extended attribute -i import hash key attributes from file and restore them (must use -f) -c check hash key attributes from input file (must use -f) -C same as -c option and correct mismatched hash key in extended attribute (must use -f) -r remove hash key extended attribute from specified selection (may use -f) -R same as -r option and remove all other values too (may use -f) -f <file> optional set file reference to <file>. Defaults to /tmp/bunker.store.log -d <days> optional only verify/update/remove files which were scanned <days> or longer ago -D <time> optional only add/verify/update/export/remove files newer than <time>, time = NNs,m,h,d,w -s <size> optional only include files smaller than <size> -S <size> optional only include files greater than <size> -l optional create log entry in the syslog file -L optional, same as -l but only create log entry when changes are present -n optional send notifications when file corruption is detected -q optional quiet mode, suppress all output. Use for background processing -md5 optional use md5 hashing algorithm instead of sha256 -b2 optional use blake2 hashing algorithm instead of sha256 path path to starting directory, mandatory with some exceptions (see examples) mask optional filter for file selection. Default is all files when path or mask names have spaces, then place names between quotes precede mask with ! to change its operation from include to exclude Examples: bunker -a /mnt/user/tv add SHA key for files in share tv bunker -a -S 10M /mnt/user/tv add SHA key for files greater than 10 MB in share tv bunker -a /mnt/user/tv *.mov add SHA key for .mov files only in share tv bunker -a /mnt/user/tv ! *.mov add SHA key for all files in share tv except .mov files bunker -A -f /tmp/keys.hash /mnt/user/tv add SHA key for files in share tv and export to file keys.hash bunker -v -n /mnt/user/files verify SHA key for previously scanned files and send notifications bunker -V /mnt/user/files verify SHA key for scanned files and update any mismatches bunker -v -d 90 /mnt/user/movies verify SHA key for files scanned 90 days or longer ago bunker -v -f /tmp/errors.hash /mnt/user/movies verify SHA key and save mismatches in file errors.hash bunker -u /mnt/disk1 update SHA key for mismatching files bunker -u -D 12h /mnt/disk1 update SHA key for mismatching files created in the last 12 hours bunker -e -f /tmp/disk1_keys.hash /mnt/disk1 export SHA key to file disk1_keys.hash bunker -i -f /tmp/disk1_keys.hash import and restore SHA key from user defined file - no path bunker -c -f /tmp/disk1_keys.hash check SHA key from user defined input file - no path bunker -C -f /tmp/disk1_keys.hash check SHA key and correct mismatched attribute (omit corruptions) - no path bunker -r /mnt/user/tv remove SHA key for files in share tv bunker -r -f /tmp/errors.hash remove SHA key for files listed in file errors.hash - no path root@Tower:/usr/local/emhttp/plugins/dynamix.file.integrity/scripts# Quote Link to comment
rampage Posted January 29, 2021 Share Posted January 29, 2021 Where is the hash file saved location? Quote Link to comment
JonathanM Posted January 29, 2021 Share Posted January 29, 2021 3 hours ago, rampage said: Where is the hash file saved location? Answer is in the first post of the thread. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.