SantyC93 Posted April 7, 2023 Share Posted April 7, 2023 (edited) After looking for a bit in the Bunker forum page I couldn't really find the solution I wanted (I'd like to run the FileIntegrity "automatic check" so that it uses my task settings and excluded folders), but I think I got something close enough. This command to check the folder I need and export (one script for each folder): /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/bunker -A -l -f /mnt/user/logs/Folder1build.log -b3 /mnt/user/Folder1 And this one to verify them: /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/bunker -v -l -f /mnt/user/logs/Folder1check.log -n -b3 /mnt/user/Folder1 Just wanted to ask if there's somithing wrong with them since I'm pretty inexperienced with this stuff, for example the command does not save any log in the folder I specified. If anyone knows what I did wrong and can help me fix this error or can tell me if I made any more mistakes I would be really grateful, ty Edited April 7, 2023 by SantyC93 spelling Quote Link to comment
Vr2Io Posted April 8, 2023 Share Posted April 8, 2023 11 hours ago, SantyC93 said: After looking for a bit in the Bunker forum page I couldn't really find the solution I wanted (I'd like to run the FileIntegrity "automatic check" so that it uses my task settings and excluded folders), but I think I got something close enough. This command to check the folder I need and export (one script for each folder): /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/bunker -A -l -f /mnt/user/logs/Folder1build.log -b3 /mnt/user/Folder1 And this one to verify them: /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/bunker -v -l -f /mnt/user/logs/Folder1check.log -n -b3 /mnt/user/Folder1 Just wanted to ask if there's somithing wrong with them since I'm pretty inexperienced with this stuff, for example the command does not save any log in the folder I specified. If anyone knows what I did wrong and can help me fix this error or can tell me if I made any more mistakes I would be really grateful, ty "-f" not use to export log, it use to specify where are the hash file. Quote Link to comment
SantyC93 Posted April 8, 2023 Share Posted April 8, 2023 4 hours ago, Vr2Io said: "-f" not use to export log, it use to specify where are the hash file. What would be the correct way to export the logs? Could you make an example pls? Tyvm Quote Link to comment
Vr2Io Posted April 8, 2023 Share Posted April 8, 2023 (edited) 5 hours ago, SantyC93 said: What would be the correct way to export the logs? Could you make an example pls? Tyvm You could export the result ( not detail log ) to a file, by add "&>test.log" at command ending. cat test.log Finished - verified 6 files, skipped 0 files. Found: 0 mismatches, 0 corruptions. Duration: 00:00:09. Average speed: 2.48 GB/s Edited April 8, 2023 by Vr2Io Quote Link to comment
SantyC93 Posted April 8, 2023 Share Posted April 8, 2023 8 minutes ago, Vr2Io said: You could export the result ( not detail log ) to a file, by add "&>test.log" at command ending. cat test.log Finished - verified 6 files, skipped 0 files. Found: 0 mismatches, 0 corruptions. Duration: 00:00:09. Average speed: 2.48 GB/s That could be useful, but logs are necessary in the event of a corrupted file. If there's no way to export the logs I guess I can't use a script to fix my problem. Ty anyway Quote Link to comment
Vr2Io Posted April 8, 2023 Share Posted April 8, 2023 (edited) 47 minutes ago, SantyC93 said: That could be useful, but logs are necessary in the event of a corrupted file. If there's no way to export the logs I guess I can't use a script to fix my problem. Ty anyway If file corrupt, it should show out the details in output. Edited April 8, 2023 by Vr2Io 1 Quote Link to comment
SantyC93 Posted April 8, 2023 Share Posted April 8, 2023 (edited) 7 hours ago, Vr2Io said: If file corrupt, it should show out the details in output. Ty! Worked great After a bit of tests I ended up with this script: /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/bunker -v -l -n -b3 /mnt/user/Folder1 &>"Folder1_Verify_log_$(date +"%Y_%m_%d_%I_%M_%p").log" /usr/local/emhttp/webGui/scripts/notify -e "Unraid Server Notice" -s "Bunker Verify" -d "Folder1 Verify complete" -i "normal" mv "/root/Folder1_Verify_log_"* "/mnt/user/logs/" it works perfectly when I manually run it, but if I run it in background or if I schedule the script the .log file does not get generated Any idea on what could be causing the issue? Edited April 8, 2023 by SantyC93 Quote Link to comment
Vr2Io Posted April 9, 2023 Share Posted April 9, 2023 2 hours ago, SantyC93 said: but if I run it in background or if I schedule the script the .log file does not get generated No such problem, does job still in running and not finish ? Quote Link to comment
SantyC93 Posted April 9, 2023 Share Posted April 9, 2023 (edited) 7 hours ago, Vr2Io said: No such problem, does job still in running and not finish ? I'm sure the script finished (I tried on a very small folder so it takes few seconds). I tried running it many times and the log file does not get generated if the script runs in the background. Tried googling but can't find anything about this problem. Edited April 9, 2023 by SantyC93 Quote Link to comment
Vr2Io Posted April 9, 2023 Share Posted April 9, 2023 (edited) You should set the full path for redirection i.e. "/Folder1/Verify_log_$(date +"%Y_%m_%d_%I_%M_%p").log" Edited April 9, 2023 by Vr2Io 1 Quote Link to comment
SantyC93 Posted April 9, 2023 Share Posted April 9, 2023 (edited) 1 hour ago, Vr2Io said: You should set the full path for redirection i.e. "/Folder1/Verify_log_$(date +"%Y_%m_%d_%I_%M_%p").log" Adding the path fixed the problem Tyvm! This is the working script: /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/bunker -v -l -n -b3 /mnt/user/Folder1 &>"/mnt/user/logs/Folder1_Verify_log_$(date +"%Y_%m_%d_%I_%M_%p").log" /usr/local/emhttp/webGui/scripts/notify -e "Unraid Server Notice" -s "Bunker Verify" -d "Folder1 Verify complete" -i "normal" Edited April 9, 2023 by SantyC93 1 Quote Link to comment
Gray J Posted April 11, 2023 Share Posted April 11, 2023 Is the program actively ignoring excluded shares for anyone else? I mean it is still checking shares that have been marked as excluded. Quote Link to comment
Vr2Io Posted April 11, 2023 Share Posted April 11, 2023 (edited) 1 hour ago, Gray J said: Is the program actively ignoring excluded shares for anyone else? I mean it is still checking shares that have been marked as excluded. Exclude only means on add / build hash. If exclude path file's have hash attribute or the hash record file have it, then it will verify too. You can use "clear" function to clear the hash attribute in exclude path. Edited April 11, 2023 by Vr2Io Quote Link to comment
SantyC93 Posted April 18, 2023 Share Posted April 18, 2023 On 4/9/2023 at 11:00 PM, SantyC93 said: Adding the path fixed the problem Tyvm! This is the working script: /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/bunker -v -l -n -b3 /mnt/user/Folder1 &>"/mnt/user/logs/Folder1_Verify_log_$(date +"%Y_%m_%d_%I_%M_%p").log" /usr/local/emhttp/webGui/scripts/notify -e "Unraid Server Notice" -s "Bunker Verify" -d "Folder1 Verify complete" -i "normal" Update: Tested the script on a large folder and the exported log file exceeded 300MB, causing the log to only show this error message: <b>Fatal error</b>: Allowed memory size of 134217728 bytes exhausted (tried to allocate 355982752 bytes) in <b>/usr/local/emhttp/plugins/dynamix.file.manager/include/Control.php</b> on line <b>121</b><br /> It seems to be caused by the PHP memory limit, which is set to 128MB by default. Googling the problem I found many post on StackOverflow suggesting to increse the memory limit to solve the problem. Anyone knows if it is possible to change the PHP memory limit in Unraid? And would it be a bad idea to increase it to 512MB? Are there other ways to fix this problem without increasing the PHP memory limit? Ty very much for the help Quote Link to comment
T0rqueWr3nch Posted May 3, 2023 Share Posted May 3, 2023 On 3/19/2023 at 6:16 AM, bonienl said: I don't see this happening on my system and need to investigate. Currently busy with the Unraid 6.12 release, but will look into this later. @bonienlI ran into the same problem when I attempted to implement this plugin with the "automatic" functionality today. I believe I have figured out the problem (I actually found two problems; one I've already submitted the PR for) and I have the automatic functionality working as expected on my Unraid development server as we speak. I just wanted to do some further testing before I submit the additional PR. tl;dr: Focus on your 6.12 release. You can take this off your mind. I'll submit the PR for your review. We appreciate your work! Quote Link to comment
T0rqueWr3nch Posted May 3, 2023 Share Posted May 3, 2023 Pull requests in: Implement auto-hashing for cache-to-disk moves by Torqu3Wr3nch · Pull Request #79 · bergware/dynamix (github.com) Fix Automatic Checksums - Don't exclude everything when nothing is set to be excluded. by Torqu3Wr3nch · Pull Request #81 · bergware/dynamix (github.com) Quote Link to comment
bonienl Posted May 3, 2023 Author Share Posted May 3, 2023 33 minutes ago, T0rqueWr3nch said: Pull requests in: Thanks, I will look into this when time permits. Currently the focus is on Unraid 6.12, which should be released "soon" Quote Link to comment
cpxazn Posted May 17, 2023 Share Posted May 17, 2023 (edited) Whenever I drag a folder into VLC to queue a playlist, for some reason it triggers rehashing for all files in that directory. I've seen at least 20 b3sum processes get kicked off by this which kills server performance. Is there a reason why the plugin thinks the files have been modified? EDIT: This is with automatic protection enabled. b3sum processes will run even when parity check is running. Edited May 17, 2023 by cpxazn Quote Link to comment
T0rqueWr3nch Posted May 18, 2023 Share Posted May 18, 2023 (edited) On 5/17/2023 at 9:52 AM, cpxazn said: Whenever I drag a folder into VLC to queue a playlist, for some reason it triggers rehashing for all files in that directory. I've seen at least 20 b3sum processes get kicked off by this which kills server performance. Is there a reason why the plugin thinks the files have been modified? EDIT: This is with automatic protection enabled. b3sum processes will run even when parity check is running. We're really only watching for a close_write, so the fact that you're seeing this implies that VLC is opening the file like it intended to write to it. My bet is that it's related to this: VLC drop linux FS-Cache when playing (#27848) · Issues · VideoLAN / VLC · GitLab. Specifically this line: Quote I found problem: taglib open file for writing when read metadata. Try updating VLC and let us know how that goes. Update (5/23/23): It doesn't appear this is fixed until 3.0.19 (current release is 3.0.18), so you may have to wait for VLC to release 3.0.19 or seek out an alpha/beta version. Edited May 23, 2023 by T0rqueWr3nch 1 Quote Link to comment
Hobbes_Is_Real Posted June 3, 2023 Share Posted June 3, 2023 I just installed unRaid for the first time this last weekend. Bitflips is a concern of mine. Considering the warning at the top of the first post, is this safe to use with Btrfs encrypted? I did read through at least a 1/3 of the 47 pages of this thread, but I am just want to make sure before I use it. Quote Link to comment
itimpi Posted June 4, 2023 Share Posted June 4, 2023 3 hours ago, Hobbes_Is_Real said: I just installed unRaid for the first time this last weekend. Bitflips is a concern of mine. Considering the warning at the top of the first post, is this safe to use with Btrfs encrypted? I did read through at least a 1/3 of the 47 pages of this thread, but I am just want to make sure before I use it. If you are using btrfs then you may not need this plugin as it has built-in checksumming for detection of bitrot. Quote Link to comment
OmegaXis Posted June 5, 2023 Share Posted June 5, 2023 I have 3 disks that always refuse to export properly, even though the plugin says otherwise. Is there something I am missing? Quote Link to comment
isvein Posted June 22, 2023 Share Posted June 22, 2023 On 6/4/2023 at 2:06 AM, itimpi said: If you are using btrfs then you may not need this plugin as it has built-in checksumming for detection of bitrot. I wonder the same thing, but for an ZFS formatted array Quote Link to comment
itimpi Posted June 22, 2023 Share Posted June 22, 2023 12 minutes ago, isvein said: I wonder the same thing, but for an ZFS formatted array In terms of having them as array drives then ZFS and BTRFS offer very similar capabilities. in protecting against bitrot. An unresolved question at the moment is whether ZFS or BTRFS will prove more resilient in such a configuration if hardware problems are encountered, and if performance is the same (I expect this is likely). Quote Link to comment
isvein Posted June 22, 2023 Share Posted June 22, 2023 So this plugin wont add much on zfs formatted array drives as it has the scrub and checksum already? and if a drive goes down, we always have good old parity as usual Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.