Jump to content

jammer

Members
  • Content Count

    6
  • Joined

  • Last visited

Community Reputation

0 Neutral

About jammer

  • Rank
    Newbie
  1. bonienl, I hadn't thought of that, but it makes sense now. Thanks for pointing that out! That pausing issue was something I was going to look at separately since it was occasional and I only thought it affected speed and not functionality. Strangely, only one of the 8 CPU readings on the Dashboard would suddenly shoot up to 100% for a few seconds while all 8 would normally hover around 10% or so during the copy operation. It wasn't as if one or more were hovering around 70+%. I think that at least for my case, the problem occurs in the automatic hashing part rather than hash verification. Scheduled hash verifications perform consistently and run off hours but I'll decrease the frequency nonetheless once I verify that everything is working properly. Is there a way to schedule the hash Build operation to happen automatically say on a nightly basis off for any new unhashed files? This way, I can leave auto real-time protection disabled. Thanks!
  2. bonienl, thanks for taking the time to help. Here are some details and additional observations (i.e. incomplete filenames/paths in the logs, and pausing during copy). I'm guessing your #1, #3, and #4 recommendations don't apply in my situation. Setup: 1. Unraid 6.6.5 running on Dell PowerEdge T110 II, Intel® Xeon® CPU E3-1240 @ 3.30GHz, 16GB ECC, 2x4TB WD Red, 2TB WD, 1TB Hitachi, 128GB SSD (Unassigned). CyberPower CP1000PFCLCD UPS. Wired network. 2. Unraid array filesystem is encrypted XFS using passphrase 3. Memtest86 passed. SMART and Parity tests pass. 4. Windows 10 Desktop PC (NTFS) containing family photos/videos to be backed up. JPG and MOV files. 5. Created a Public SMB Share (/photos) that only includes one empty disk of the array. (keeping it simple for testing purposes). Copy-on-write on Auto. Cache disk disabled. Enhanced OS X interoperability disabled. 6. Plugins installed: Unassigned Devices, Community Applications, Dynamix File Integrity, 7. Docker and VM enabled but none installed. 8. Dynamix File Integrity options: Auto protect new files=Enabled. Hashing method=BLAKE2 (also tried SHA2). Save hash to flash=Disabled. Excluded folders/files=docker.img,libvirt.img and Apple metadata. Verification schedule=Daily. Procedure: 1. On Windows PC, I navigate to //towerl/photos and then drag/copy a directory of photos (inside are subdirectories of photos separated by date/event) from the local drive to the Unraid share. There are ~40,000 files totalling 160GB. This takes a few hours. 2. Wait for the daily hash verification check and the parity check (I set this to be more frequent for testing purposes) Symptoms: SHA256 hash key mismatch, is corrupted SHA256 hash key mismatch, /Jimmy Samsung/20160518_215300.jpg is corrupted SHA256 hash key mismatch, bbean cruise/00013.MTS is corrupted SHA256 hash key mismatch, ion/IMG_0408.JPG is corrupted SHA256 hash key mismatch, ion/IMG_0815.JPG is corrupted SHA256 hash key mismatch, thering at Janice/DSC_8135.JPG is corrupted SHA256 hash key mismatch, up shots/DSC_7719.JPG is corrupted 1. Firstly, not sure if it's related but while copying I notice that the transfer speed is 40+MB/s but drops to 0MB/s for several seconds every few minutes or so. During those times when it's dropped to 0MB/s I notice that one to two CPU cores is at 100% in the Unraid dashboard. 2. Daily hash verification shows ?? files hash mismatch. (see log excerpt above). These are for JPG and MTS (Video) files that don't change. 3. Note that in the first entry the filename is empty and for the other files, the filepaths are incomplete. 4. Subsequent hash verifications show identical errors and log output. 5. I've compared file hashes using other tools and also bitwise compare (cmp) to confirm that these so-called corrupted files are indeed identical to the source files. 6. Parity check shows 0 errors. I repeated the above. That is, I deleted the files and copied the same directory again. This time a different set of files show errors and subsequent hash verifications consistently show errors on that new set of files. As another test, I used mc on Unraid to make a copy of the folder locally on the array (i.e. taking Windows out of the loop). Similar result, the new copy had another set of files that show errors consistently. Again, file compares showed that the files were bitwise identical. I then changed the hash method from SHA2 to BLAKE2 and did a manual Build in the Integrity menu to recalculate the hashes for all files. The next daily hash verification showed no errors. However, after copying more files from the Windows PC, a few hash errors were found again in the new files. Any other things for me to try? Thanks.
  3. _Shorty, thanks for responding. That's unfortunate to hear. bonienl, I don't doubt that it works perfectly for the vast majority of users otherwise I would expect to find more discussion about issues. Can you please suggest next steps on how to troubleshoot? I suspect it may have something to do with lots of files with long Windows filepaths and additional attributes that may lead up to this issue happening. But then after copying to Unraid, I've even tried using mc to make a copy and other files encounter the same issue. Thanks in advance.
  4. I believe I'm experiencing the same issue as _Shorty. I tested by making a copy of a directory onto the same drive. Got hash errors on some files but the files are identical to the source (used other hashing tools and a binary compare using cmp). Did a hash export from the Dynamix Plugin and the stored hashes were indeed different from each other. _Shorty, did you end up finding a solution or alternative for file integrity? Anyone else?
  5. Hi there, This is my first post as I started to trial Unraid a few weeks ago. Thanks for this plugin. I'm encountering some weirdness that I hope the community can help me understand. I am notified of SHA256 hash errors after the daily integrity check after backing up photos from a Windows PC over SMB. I enabled "Automatically protect new files". This is in the log: SHA256 hash key mismatch, is corrupted SHA256 hash key mismatch, /Jimmy Samsung/20160518_215300.jpg is corrupted SHA256 hash key mismatch, bbean cruise/00013.MTS is corrupted SHA256 hash key mismatch, ion/IMG_0408.JPG is corrupted SHA256 hash key mismatch, ion/IMG_0815.JPG is corrupted SHA256 hash key mismatch, thering at Janice/DSC_8135.JPG is corrupted SHA256 hash key mismatch, up shots/DSC_7719.JPG is corrupted SMART attributes are good (two new WD Red 4TB). Parity check (no auto correct) is fine too. When manually checking SHA256 hash values for the above files on Unraid and of the source files, they come back identical. Subsequent integrity checks show the same errors so it doesn't seem like a one-time read error. Questions: 1. What could cause the plugin to report these as errors even though the files seem to be written correctly? 2. Is it normal for the file paths to be cut off? I had to search for the names of the files to find their paths. 3. What could cause the file name to be missing entirely (i.e. the first entry)? Any help would be much appreciated!