I can make that an option in bunker to send notifications when corruption is detected.

 

 

That sounds like a much better option than me riggin something up. Although i will say I did manage to figure out the command I probably need is "notifiy smpt-init" so I was getting somewhere! But it makes more sense for it to be built-in. Do you think it would be possible for the email to print the log in the body of the email? That way you could just read the email and find the file without having to look through logs.

OK so here are the two scripts I created. One question one the first script.

$bunker -a -q -l -f $log/new.disk1.$day /mnt/disk1 && $bunker -U -q -l -f $log/updates.disk1.$day /mnt/disk1 &

Will this first add checksums to new files AND THEN update any user modified files? Just wanted to make sure bunker will not add new and update at the same time. Although I guess the & at the end ins't going to allow disk2 to start that the same time as disk1 because of the &&?

 

bunker-newupdate

#!/bin/bash

bunker=/boot/bunker
var=/var/local/emhttp/var.ini
log=/boot/logs/bunker
day=$(date +%Y%m%d)
array=$(awk -F'"' '/^fsState=/{print $2}' $var)
rsync=$(awk -F'"' '/^mdResync=/{print $2}' $var)

# Every day add new and update user modified checksums
if [[ $array == Started  && $rsync -eq 0 ]]; then
        $bunker -a -q -l -f $log/new.disk1.$day /mnt/disk1 && $bunker -U -q -l -f $log/updates.disk1.$day /mnt/disk1 &
        $bunker -a -q -l -f $log/new.disk2.$day /mnt/disk2 && $bunker -U -q -l -f $log/updates.disk2.$day /mnt/disk2 &
        $bunker -a -q -l -f $log/new.disk3.$day /mnt/disk3 && $bunker -U -q -l -f $log/updates.disk2.$day /mnt/disk3 &
fi

 

bunker-verify

#!/bin/bash

bunker=/boot/bunker
var=/var/local/emhttp/var.ini
log=/boot/logs/bunker
day=$(date +%Y%m%d)
array=$(awk -F'"' '/^fsState=/{print $2}' $var)
rsync=$(awk -F'"' '/^mdResync=/{print $2}' $var)

# Every month verify checksums
if [[ $array == Started  && $rsync -eq 0 ]]; then
        $bunker -V -q -l -f $log/verify.disk1.$day /mnt/disk1 &
        $bunker -V -q -l -f $log/verify.disk2.$day /mnt/disk2 &
        $bunker -V -q -l -f $log/verify.disk3.$day /mnt/disk3 &
fi

I can make that an option in bunker to send notifications when corruption is detected.

 

 

That sounds like a much better option than me riggin something up. Although i will say I did manage to figure out the command I probably need is "notifiy smpt-init" so I was getting somewhere! But it makes more sense for it to be built-in. Do you think it would be possible for the email to print the log in the body of the email? That way you could just read the email and find the file without having to look through logs.

 

Ok, new version 1.8 of bunker is available (see OT).

 

This version has the new option -n (notify) which let bunker send notifications when file corruption is detected.

 

File corruption is sent as an "alert" message, so make sure you have configured the correct recipients (browser, email or agents) to receive the alert.

 

Only emails will have a message body detailing the corrupted file names, therefore I would advice to select at least email as a recipient.

 

I can make that an option in bunker to send notifications when corruption is detected.

 

 

That sounds like a much better option than me riggin something up. Although i will say I did manage to figure out the command I probably need is "notifiy smpt-init" so I was getting somewhere! But it makes more sense for it to be built-in. Do you think it would be possible for the email to print the log in the body of the email? That way you could just read the email and find the file without having to look through logs.

 

Ok, new version 1.8 of bunker is available (see OT).

 

This version has the new option -n (notify) which let bunker send notifications when file corruption is detected.

 

File corruption is sent as an "alert" message, so make sure you have configured the correct recipients (browser, email or agents) to receive the alert.

 

Only emails will have a message body detailing the corrupted file names, therefore I would advice to select at least email as a recipient.

 

Awesome! Thank you very much!

Any thoughts on why I am getting this error:

 

I run this command:

bunker -U -l -f /boot/logs/bunker/new.disk3.$(date +%Y%m%d) /mnt/disk3

Specified file doesn't exist

 

If I remove

-f /boot/logs/bunker/new.disk3.$(date +%Y%m%d)

then it runs.

 

ALSO:

I typed this and got this message:

bunker -a -l /mnt/disk3 && bunker -U -l /mnt/disk3

./bunker: line 414: plural$: command not found

 

Bunker -a lockup.  Processing file 152 of 195168, and rather than processing as normal, it just stopped there for several hours.  Ctrl C would not break out of it.

 

Htop reports the following:

 

Any thoughts on why I am getting this error:

 

I run this command:

bunker -U -l -f /boot/logs/bunker/new.disk3.$(date +%Y%m%d) /mnt/disk3

Specified file doesn't exist

 

If I remove

-f /boot/logs/bunker/new.disk3.$(date +%Y%m%d)

then it runs.

 

ALSO:

I typed this and got this message:

bunker -a -l /mnt/disk3 && bunker -U -l /mnt/disk3

./bunker: line 414: plural$: command not found

 

You've found two bugs, I have corrected them and will issue version 1.9 with bug fixes.

 

Thanks.

 

Bunker -a lockup.  Processing file 152 of 195168, and rather than processing as normal, it just stopped there for several hours.  Ctrl C would not break out of it.

 

Htop reports the following:

 

 

I had similar issues when using ReiserFS as the file system, once in a while the writing of the extended attributes causes a hang up. Since moving to XFS I never had this problem anymore.

 

Version 1.9 now available.

 

Bunker -a lockup.  Processing file 152 of 195168, and rather than processing as normal, it just stopped there for several hours.  Ctrl C would not break out of it.

 

Htop reports the following:

 

 

I had similar issues when using ReiserFS as the file system, once in a while the writing of the extended attributes causes a hang up. Since moving to XFS I never had this problem anymore.

I have run bunker on 30tb of rfs and this was the first I've seen this. Would the new versions (post 1.5) be more likely to see this?

I have run bunker on 30tb of rfs and this was the first I've seen this. Would the new versions (post 1.5) be more likely to see this?

 

All versions prior to 1.7 did write the same extended attributes, and no change in coding for that part.

 

From version 1.7 onwards the new extended attribute filedate is added, though coding-wise the same approach is still done.

 

Personally I lost a bit faith in RFS under v6 and moved all to XFS, so far I am quite pleased with the result.

 

Yes I plan to move to xfs too, but need a life changing reason to do so. So far v6 has new disks xfs old disks rfs.

Thanks for 1.9

 

I can confirm that both messages go away. I think something is still messed up with the -f flag. It does not create the log in the folder I have specified.

I have tried all the variations of but it never creates the log file in the folder (even when I made new files and made changes to the files)

bunker -a -l -f /boot/logs/bunker/new.disk3.$(date +%Y%m%d) /mnt/disk3

bunker -U -l -f /boot/logs/bunker/new.disk3.$(date +%Y%m%d) /mnt/disk3

bunker -V -l -f /boot/logs/bunker/new.disk3.$(date +%Y%m%d) /mnt/disk3

 

Also, I think I found another bug as well 

bunker -a -l -f /boot/logs/bunker/new.disk3.$(date +%Y%m%d) /mnt/disk3

Scanning for files to add... - awk: cmd. line:1: fatal: division by zero attempted

 

Yes I plan to move to xfs too, but need a life changing reason to do so. So far v6 has new disks xfs old disks rfs.

 

FWIW, I have also moved all my disk (I only have 3) to xfs and have been very pleased with it as well.

Also, I think I found another bug as well 

 

I released version 1.10 which has a small correction that may prevent this bug (can't really reproduce myself) and also in v1.10 is a more comprehensive reporting, telling the execution duration and average hashing speed after the task is finished.

 

Addendum: you can use the command -e (export) to create a file explicitely.

 

bunker -e -f /boot/logs/bunker/new.disk3.$(date +%Y%m%d) /mnt/disk3

 

I have created two scripts for daily and monthly processing. These might be useful for you, but you need to adapt them to your needs, see the red sections. Make sure you are running the latest version of bunker.

 

Update: adjusted bunker-daily to use command -U -D 1 (which speeds up execution by limiting to files changed in the last day)

 

bunker-daily is copied to folder /etc/cron.daily and checks for new additions, reports about errors for changed files  and keeps the export file up to date.

#!/bin/bash

[color=red]bunker=/boot/custom/bin/bunker[/color]
[color=red]log=/boot/custom/hash[/color]
var=/proc/mdcmd
day=$(date +%Y%m%d)
array=$(grep -Po '^mdState=\K\S+' $var)
rsync=$(grep -Po '^mdResync=\K\S+' $var)

mkdir -p $log

# Daily check on new files, report errors and create export file
if [[ $array == STARTED && $rsync -eq 0 ]]; then
  for i in [color=red]1 2 3 4 5 6 7 8 9 10[/color] ; do
    $bunker -A -D 1 -q -l -f $log/disk$i.$day.new.txt /mnt/disk$i
    $bunker -U -D 1 -q -l -n -f $log/disk$i.$day.bad.txt /mnt/disk$i
    if [[ -s $log/disk$i.export.txt ]]; then
      if [[ -s $log/disk$i.$day.new.txt || -s $log/disk$i.$day.bad.txt ]]; then
        mv $log/disk$i.export.txt $log/disk$i.$day.txt
        $bunker -e -q -l -f $log/disk$i.export.txt /mnt/disk$i
      fi
    else
      $bunker -e -q -l -f $log/disk$i.export.txt /mnt/disk$i
    fi
  done
fi

bunker-monthly is copied to folder /etc/cron.monthly and verifies a group of disks on file corruption.

#!/bin/bash

[color=red]bunker=/boot/custom/bin/bunker[/color]
[color=red]log=/boot/custom/hash[/color]
var=/proc/mdcmd
day=$(date +%Y%m%d)
array=$(grep -Po '^mdState=\K\S+' $var)
rsync=$(grep -Po '^mdResync=\K\S+' $var)

mkdir -p $log

# Monthly verification of different group of disks (quarterly rotation)
if [[ $array == STARTED && $rsync -eq 0 ]]; then
  case $(($(date +%m)%4)) in
  0) for i in [color=red]1 2[/color] ; do
       $bunker -v -n -q -l -f $log/disk$i.$day.bad.txt /mnt/disk$i &
     done ;;
  1) for i in [color=red]3 4 5[/color] ; do
       $bunker -v -n -q -l -f $log/disk$i.$day.bad.txt /mnt/disk$i &
     done ;;
  2) for i in [color=red]6 7 8[/color] ; do
       $bunker -v -n -q -l -f $log/disk$i.$day.bad.txt /mnt/disk$i &
     done ;;
  3) for i in [color=red]9 10[/color] ; do
       $bunker -v -n -q -l -f $log/disk$i.$day.bad.txt /mnt/disk$i &
     done ;;
  esac
fi

bunker-daily.txt

bunker-monthly.txt

For the brave users ...

 

You can adjust the script bunker-daily to check for files changed in the last hour and copy it to /etc/cron.hourly (might change the name to bunker-hourly).

  $bunker -A [color=red]-D 1h[/color] -q -l -f $log/disk$i.$day.new.txt /mnt/disk$i
  $bunker -U [color=red]-D 1h[/color] -q -l -n -f $log/disk$i.$day.bad.txt /mnt/disk$i

 

Thanks for the example scripts.

 

Looks like you set it up for a 10 disk array?  I could use your monthly script on a weekly basis for a larger server couldn't I ?

 

Thanks for the example scripts.

 

Looks like you set it up for a 10 disk array?  I could use your monthly script on a weekly basis for a larger server couldn't I ?

 

Sure you can. Below a weekly example for 16 drives (the reason it is split over several weeks is because a complete scan/verification per disk is done). Running 4 disks concurrently is about the max my processor can do, yours may be different.

#!/bin/bash

bunker=/boot/custom/bin/bunker
log=/boot/custom/hash
var=/proc/mdcmd
day=$(date +%Y%m%d)
array=$(grep -Po '^mdState=\K\S+' $var)
rsync=$(grep -Po '^mdResync=\K\S+' $var)

mkdir -p $log

# Weekly verification of different group of disks (monthly rotation)
if [[ $array == STARTED && $rsync -eq 0 ]]; then
  case $(($([color=red]date +%U[/color])%4)) in
  0) for i in 1 2 3 4 ; do
       $bunker -v -n -q -l -f $log/disk$i.$day.bad.txt /mnt/disk$i &
     done ;;
  1) for i in 5 6 7 8 ; do
       $bunker -v -n -q -l -f $log/disk$i.$day.bad.txt /mnt/disk$i &
     done ;;
  2) for i in 9 10 11 12 ; do
       $bunker -v -n -q -l -f $log/disk$i.$day.bad.txt /mnt/disk$i &
     done ;;
  3) for i in 13 14 15 16 ; do
       $bunker -v -n -q -l -f $log/disk$i.$day.bad.txt /mnt/disk$i &
     done ;;
  esac
fi

 

Thanks for the scripts. If we manually copy the scripts to their respective cron folders do they survive a reboot or do we need to set up a cp command in our go files to copy them in on every reboot?

Thanks for the scripts. If we manually copy the scripts to their respective cron folders do they survive a reboot or do we need to set up a cp command in our go files to copy them in on every reboot?

 

After a reboot files will be gone, you need to copy them back. I have the following in my go file

# Copy scheduler programs
cp /boot/custom/bin/bunker-daily /etc/cron.daily
cp /boot/custom/bin/bunker-monthly /etc/cron.monthly

 

The bunker-daily script threw up a bunch of errors before I canceled it.

 

EDIT: I think I had a / at the end of the log path. I removed it and it appears to be running OK now.

I have a question with the bunker-monthly script. Does it ever get triggered at the same time as parity check? If so how do you handle that?

sooo I am doing some testing with the bunker-daily script (sorry) and whenever I add a test file to disk1 and run bunker-daily I get the following errors. I think it is because it is trying and failing to create the new txt log of the added file. I will also attach my modified bunker-daily file, just in case I messed it up.

bunker-daily.txt