CS01-HS Posted December 7, 2020 Share Posted December 7, 2020 I recently changed Error Logging from Syslog Only to Syslog and Output file. After a check I see the entries in syslog but can't find the "output file" - any idea where it's stored? (I saw a couple of posters ask but didn't see an answer.) Quote Link to comment
Mihle Posted December 9, 2020 Share Posted December 9, 2020 Now I get mismatches because Nextcloud files_trashbin too. This plugin likes to give false positives when you have Nextcloud. Quote Link to comment
jasonfox Posted December 17, 2020 Share Posted December 17, 2020 (edited) I have 3 servers and this plug in has been installed on each for years. Everything has worked well. So much so that I frequently forget to check back on them for months. I happened to be doing other things in the tools tab and had an 'oh yeah FIP' moment. So I started just looking around to be sure everything was still running smoothly and found that 2 of my machines were running MD5 and one was running SHA2 and I couldn't for the life of me remember why. I turned on the help setting (consistently amazed at how awesome the help is) and it reminded me of the blake2 option which I think I had passed on originally because it said it was proprietary and I had some aspirations of checking my backups externally or some such that of course I never did. I did a little quick googling of blake2 and learned that there is now a blake3 that is approx 5x as fast as blake2. Has any thought been given to adding this as an option? Edited December 17, 2020 by jasonfox typos Quote Link to comment
Fizz Posted December 17, 2020 Share Posted December 17, 2020 My scheduled verify job reported "bunker verify command Found 1 file with BLAKE2 hash key corruption" but doesn't tell me what file is corrupted. I had error logging set to syslog in settings, but I don't see the file listed there either. How can I see what file is causing the problem? Here is the relevant part of syslog: bunker: error: BLAKE2 hash key mismatch, is corrupted bunker: Verify task for disk1 finished, duration: 12 hr, 2 min, 0 sec. bunker: verified 455797 files from /mnt/disk1. Found: 0 mismatches, 1 corruption. Duration: 12:02:00. Average speed: 98.3 MB/s 1 Quote Link to comment
Vr2Io Posted December 17, 2020 Share Posted December 17, 2020 (edited) 12 hours ago, jasonfox said: I have 3 servers and this plug in has been installed on each for years. Everything has worked well. So much so that I frequently forget to check back on them for months. I happened to be doing other things in the tools tab and had an 'oh yeah FIP' moment. So I started just looking around to be sure everything was still running smoothly and found that 2 of my machines were running MD5 and one was running SHA2 and I couldn't for the life of me remember why. I turned on the help setting (consistently amazed at how awesome the help is) and it reminded me of the blake2 option which I think I had passed on originally because it said it was proprietary and I had some aspirations of checking my backups externally or some such that of course I never did. I did a little quick googling of blake2 and learned that there is now a blake3 that is approx 5x as fast as blake2. Has any thought been given to adding this as an option? Speedup should be multiple core/thread only. For array even SSD ( single disk limitation ), expect no much benefit. ( current max speed ~500MB/s+ ) Edited December 17, 2020 by Vr2Io Quote Link to comment
pman Posted December 20, 2020 Share Posted December 20, 2020 (edited) I received an unraid alert for "Found 1 file with SHA256 hash key corruption". How do I tell which file is corrupt? No log file was created. My last dynamix log file is from 2019. Thanks. EDIT: I ran this cmd to find it, but surely there has to be a better way? Why is it not logging? "cat /var/log/syslog | grep corrupt" Edited December 20, 2020 by pman Quote Link to comment
Fizz Posted December 20, 2020 Share Posted December 20, 2020 I posted about a similar problem above. For me, syslog does not say what the corrupted file is. 1 Quote Link to comment
CS01-HS Posted December 28, 2020 Share Posted December 28, 2020 On 4/3/2019 at 8:24 PM, jeffreywhunter said: I'm see the following error in my log, but no reference to any bad files. BLAKE2 hash key mismatch, is corrupted Was this ever resolved? I'm seeing it now. 1 Quote Link to comment
vakilando Posted January 6, 2021 Share Posted January 6, 2021 same here: two of the corrupt files are not mentioned and I don't know what/where the third one is Jan 6 03:31:59 bunker: error: BLAKE2 hash key mismatch, is corrupted Jan 6 03:51:41 bunker: error: BLAKE2 hash key mismatch, is corrupted Jan 6 03:58:50 bunker: error: BLAKE2 hash key mismatch, ng/fs.json is corrupted Jan 6 04:02:59 bunker: Verify task for disk1 finished, duration: 1 hr, 2 min, 58 sec. Jan 6 04:02:59 bunker: verified 198104 files from /mnt/disk1. Found: 0 mismatches, 3 corruptions. Duration: 01:02:58. Average speed: 36.1 MB/s 2 Quote Link to comment
Ranmacanada Posted February 6, 2021 Share Posted February 6, 2021 Can confirm what others have said about false positives. Said one of my bluray images was corrupt, and I rechecked it against the original and the image is 100% fine. Quote Link to comment
gacpac Posted February 7, 2021 Share Posted February 7, 2021 Me i'm kind of confused with this plugin. I've used it since forever. But it's one of those that well, if you don't get errors then you are good. But I'm kind of hoping to see a log with all good. All I have are the exports from 2019 and no logs. Currently i'm doing a manual check to see. I'm hoping to see a notification or something. Quote Link to comment
gacpac Posted February 8, 2021 Share Posted February 8, 2021 On 12/17/2020 at 2:51 AM, Fizz said: My scheduled verify job reported "bunker verify command Found 1 file with BLAKE2 hash key corruption" but doesn't tell me what file is corrupted. I had error logging set to syslog in settings, but I don't see the file listed there either. How can I see what file is causing the problem? Here is the relevant part of syslog: bunker: error: BLAKE2 hash key mismatch, is corrupted bunker: Verify task for disk1 finished, duration: 12 hr, 2 min, 0 sec. bunker: verified 455797 files from /mnt/disk1. Found: 0 mismatches, 1 corruption. Duration: 12:02:00. Average speed: 98.3 MB/s Mine got sent in an email from the smtp that i previously setup Quote Link to comment
iarp Posted February 8, 2021 Share Posted February 8, 2021 (edited) I'm looking for some clarification on the Disk Verification Schedule and the Tasks. So the help says Quote when you have set up a weekly schedule with 3 tasks then every week another task is executed and after 3 weeks the verification function has done all selected disks. So in this example you have 3 disks and you have put each disk on a separate task. My understanding of the above is that Task 1 executes on the Monday, then the following Monday Task 2, and the following Monday after that Task 3. But then when does it restart? The next Monday after the week with Task 3 finishing its back to Task 1? If I select Monthly, is that going to run Task 1 in Jan, Task 2 in Feb, Task 3 in March? Is there anyway to set it up so it checks all disks once a month but separate the tasks by a day or a week? Also on the Integrity page in the help it states "0 represents an open operation", does that mean a running operation? Like it's actively doing something in the background? Edited February 8, 2021 by iarp Quote Link to comment
mani3321 Posted February 13, 2021 Share Posted February 13, 2021 Is there a solution regarding false positives? Quote Link to comment
shEiD Posted February 18, 2021 Share Posted February 18, 2021 I've just installed and started using the File Integrity plugin today. I manually started a `Build` process on 7 (out of 28) drives in my array disk1 had least amount of files, so it has already finished But, the UI shows some nonsense 🤔 it shows disk1 as a circle, not green checkmark, even though it has just finished the build, and is up-to-date it shows disks 4, 5, 9 and 10 with a green checkmark, even though the builds are clearly still running and aren't finished it shows disks 7, 12, 17, 18, 19, 22, 23, 24, 26, 27 and 28 with a green checkmark, even though the build process has never been run on these disk... I mean, WAT? 😳 Can it be, that am I really not understanding what the circle/checkmark/cross means? Or is this a bug? Quote Link to comment
servidude Posted March 1, 2021 Share Posted March 1, 2021 I'm seeing similar notifications about 3 supposedly corrupted files, but no names! BLAKE2 hash key mismatch, is corrupted BLAKE2 hash key mismatch, is corrupted BLAKE2 hash key mismatch, is corrupted The corresponding syslog entries are also missing file info, for example: Feb 28 18:17:44 Tower bunker: error: BLAKE2 hash key mismatch, is corrupted What is the cause of this (apparently I'm not the only one seeing it), and how can I fix it - is there a way to reset the file hashes? Quote Link to comment
Ranmacanada Posted March 6, 2021 Share Posted March 6, 2021 Another false positive on a bluray dump (different image). I've verified it with the original and everything hashes 100%. I'm only getting 1 mismatch every month, but this is 2 for 2 where it has been a false positive. Is there any way to address this or am I just stuck at having to double check files when it says they are corrupt? Quote Link to comment
Vr2Io Posted March 6, 2021 Share Posted March 6, 2021 20 minutes ago, Ranmacanada said: Another false positive on a bluray dump (different image). I've verified it with the original and everything hashes 100%. I'm only getting 1 mismatch every month, but this is 2 for 2 where it has been a false positive. Is there any way to address this or am I just stuck at having to double check files when it says they are corrupt? This is know issue, I naver have bluray image ( or static file ) false positive, because I hash file by "build" in manual not automatic. Quote Link to comment
touz Posted April 22, 2021 Share Posted April 22, 2021 On 10/13/2019 at 10:15 PM, Ouze said: Happened again - did a check, out the output: "bunker: error: SHA256 hash key mismatch, rnal.720p.bluray.x264-reward.mkv is corrupted" According to a search, that narrows it down to 125 files. There has got to be a way to get this plugin to output more useful (specific) information, right? Why are these error messages being truncated this way? I did an md5 check against all 125 files individual checksums and of course, they pass just fine. These files got dropped onto the server once and have not been touched since then. If this plugin is going to generate output vague as it is, and as prone to false positives as it seems to be; then ultimately it's just contributing noise, not value. Did you finally found out why file names/paths are truncated? It also happens to me, and can be very annoying to find the exact file which is "supposed" to be corrupted. Quote Link to comment
Fizz Posted May 15, 2021 Share Posted May 15, 2021 Is there a way to change the location where the hash results for each file are stored? I see they're stored in plain text on my usb drive. I'd like the results to be in an encrypted location such as my cache or array. Quote Link to comment
DuzAwe Posted May 16, 2021 Share Posted May 16, 2021 Files dont seem to be ignored in my set up I have tried .log, .nfo, .tgz, .tmp and *.log, *.nfo, *.tgz, *.tmp. Keep getting alerts about .nfos changing. Either ts a bug or I am doing something wrong. Quote Link to comment
Supershocker Posted June 9, 2021 Share Posted June 9, 2021 MD5 hash key mismatch, /stats/combnk.html is corrupted MD5 hash key mismatch, box/fixedpoint/ui/nedVisuals/fxpHistogram/release/images/toolstrip/liveeditor/left-align-24.png is corrupted MD5 hash key mismatch, box/matlab/datatypes/duration/+matlab/+internal/+duration/getDetectionFormats.m is corrupted MD5 hash key mismatch (updated), /mnt/disk1/system/docker/docker.img was modified MD5 hash key mismatch (updated), /mnt/disk1/system/libvirt/libvirt.img was modified I got the above response from an automated check. I understand docker and libvirt normally change but the corruptions I don't understand. I cant figure out what these files are and how to resolve this. Quote Link to comment
sunbear Posted June 11, 2021 Share Posted June 11, 2021 Does anyone know why my scans always fail to scan any files? I have tried different folder exclusions and even removed all exclusions altogether any yet anytime I select "BUILD" or "EXPORT" for any disk, it scans 0 files for several hours and always has zero files at the end of the scan. I assume I'm doing something wrong but I've tried all different types of settings and hash methods and no dice. Screenshot below. Quote Link to comment
sunbear Posted June 11, 2021 Share Posted June 11, 2021 Attached Diagnostics. minint-petb45-diagnostics-20210610-2021.zip Quote Link to comment
Vr2Io Posted June 11, 2021 Share Posted June 11, 2021 2 hours ago, sunbear said: I assume I'm doing something wrong but I've tried all different types of settings and hash methods and no dice. Screenshot below. Your system got OOM ( out of memory ) call trace, you need found out what cause this. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.