Latest (super easy) method for automated flash zip backup!


Recommended Posts

In my previous post found here, i outlined a multi-step process with 2 different scripts and 2 cron jobs for automating the unraid flash zip backup process using the User Scripts Plugin. Thanks to @nitewolfgtr and @sreknob i've refined it down to one simple "fill in the blanks" script. I've also added an optional UI notification.

 

This script backs up a zipped copy of your unraid flash disk to a location of your choice, removes old backups after specified number of days, as well as the optional notification upon completion

 

Due to the changes, i've marked the other OP outdated and now refer to this script. While i have tested (version 6.8.3 only) and currently run this on all 3 of my unraid servers with reasonable confidence, i stress for anyone that decides to try, MANUALLY BACKUP YOUR FLASH DRIVE FIRST BEFORE TRYING. if you are at all nervous or unfamiliar, please don't try this - i don't assume any liability here for data loss. Use at your own risk, this is still a work in progress.

 

Here's the script, with explanation underneath currently for unraid version 6.8.3 AND 6.9.0-rc2 (version 6.9b30 see below)

#!/bin/bash

#### SECTION 1 ####------------------------------------------------------------------------------------------------------
#dir = WHATEVER FOLDER PATH YOU WANT TO SAVE TO
dir="/insert/your/path/here"

echo 'Executing native unraid backup script'
/usr/local/emhttp/webGui/scripts/flash_backup

#### SECTION 2 ####------------------------------------------------------------------------------------------------------
echo 'Remove symlink from emhttp'
find /usr/local/emhttp/ -maxdepth 1 -name '*flash-backup-*.zip' -delete

sleep 5

#### SECTION 3 ####------------------------------------------------------------------------------------------------------
if [ ! -d "$dir" ] ; then
 
			echo "making directory as it does not yet exist"

			# make the directory as it doesnt exist
			mkdir -vp "$dir"
		else
			echo "As $dir exists continuing."
			fi


#### SECTION 4 ####------------------------------------------------------------------------------------------------------
echo 'Move Flash Zip Backup from Root to Backup Destination'
mv /*-flash-backup-*.zip "$dir"

sleep 5

#### SECTION 5 ####------------------------------------------------------------------------------------------------------
echo 'Deleting Old Backups'

#ENTER NUMERIC VALUE OF DAYS AFTER "-MTIME +"
find "$dir"* -mtime +10 -exec rm -rfv {} \;

echo 'All Done'

#### SECTION 6 ####------------------------------------------------------------------------------------------------------

#UNCOMMENT THE NEXT LINE TO ENABLE GUI NOTIFICATION UPON COMPLETION 
#/usr/local/emhttp/webGui/scripts/notify -e "Unraid Server Notice" -s "Flash Zip Backup" -d "A copy of the YOURTOWERNAME unraid flash disk has been backed up" -i "normal"

exit

 

Here's the slightly modified script if you're running unraid version 6.9b30:

 

Copy and paste the appropriate version above into a new blank script in the User Scripts Plugin, then proceed to edit per below.

 

Some final changes on your end to implement the script. I broke the script down into sections for an easier explanation here: 

 

  • Section 1 - you need enter the path to where you want to save your zipped flash copy (local or remote).
  • Section 5 - be sure to enter the number of days you want to keep old flash backups, default is 10 days.
  • Section 6 - optional, to enable notifications, uncomment the bottom line and replace "YOURTOWERNAME" with your actual tower name (this is for notification only)

 

And that's it, all there is to it. Just copy/paste the script into the user scripts plugin, set how often you want it to run, and you're done. If you have any questions or suggestions, let me know. Enjoy.

 

 

EDIT 1: 12/21/20 - Merged two previous scripts into one script for aforementioned versions

 

EDIT 2: 12/21/20 - Added quotes to $dir in Section 4 that could cause directories with spaces to get lost

 

EDIT 3: 2/5/21 - Step by step instructions found on my youtube channel here

 

Edit 4: 2/8/21 - Inserted new Section 3 with mkdir command to prevent errors if directory does not yet exist, bumped subsequent sections down. No user interaction required

                     - Added quotes to primary directory path to help prevent syntax errors

 

 

Edited by Cpt. Chaz
improvements
  • Like 7
  • Thanks 4
Link to comment
  • 1 month later...

So I did run this and this happened:

image.thumb.png.0c27bf04629d94c830c11f0bc2bc23af.png

I think it somehow deleted more things than it should have done

EDIT:
Reboot fixed it, so I think it caused a memory leak or what its called that made unRAID itself not able to run from memory anymore as it usually do. Had to turn it off by the power button.

(I do have a manual flash backup already btw)

 

This is what the script looks like for me right now:

 

#!/bin/bash

#### SECTION 1 ####------------------------------------------------------------------------------------------------------
#dir = WHATEVER FOLDER PATH YOU WANT TO SAVE TO
dir=/mnt/user/NCloud/Mihle/files/Filer/Backup/NAS Flash Backup/

echo 'Executing native unraid backup script'
/usr/local/emhttp/webGui/scripts/flash_backup

#### SECTION 2 ####------------------------------------------------------------------------------------------------------
echo 'Remove symlink from emhttp'
find /usr/local/emhttp/ -maxdepth 1 -name '*flash-backup-*.zip' -delete

sleep 5

#### SECTION 3 ####------------------------------------------------------------------------------------------------------
echo 'Move Flash Zip Backup from Root to Backup Destination'
mv /*-flash-backup-*.zip "$dir"

sleep 5

#### SECTION 4 ####------------------------------------------------------------------------------------------------------
echo 'Deleting Old Backups'

#ENTER NUMERIC VALUE OF DAYS AFTER "-MTIME +"
find "$dir"* -mtime +90 -exec rm -rfv {} \;

echo 'All Done'

#### SECTION 5 ####------------------------------------------------------------------------------------------------------

#UNCOMMENT THE NEXT LINE TO ENABLE GUI NOTIFICATION UPON COMPLETION 
/usr/local/emhttp/webGui/scripts/notify -e "Unraid Server Notice" -s "Flash Zip Backup" -d "A copy of the NAS unraid flash disk has been backed up" -i "normal"

exit

 

When I run it, it did say something about it could not find directory " " over and over and over I think? dont quite remember

 

Did I write something in to the script wrongly?

Edited by Mihle
Link to comment

Tested again, but I removed the / at the end of the directory, it says this:

1635072809_Skjermbilde2021-02-06124337.thumb.jpg.abb439277bc0b2d25a225ea8f55ac02a.jpg

Then remove some other things

then continues like this, probably until the NAS runs out of memory and crashes?

(I run it manually)

1509379657_Skjermbilde2021-02-06124551.jpg.e87b71817f7be08e9345cd71ad31ac3d.jpg

 

Is there something I have done that causes it?

Edited by Mihle
Link to comment

Does your backup directory (/mnt/user/NCloud/Mihle/files/Filer/Backup/NAS Flash Backup/) already exist?

 

If not, it needs to be created before the script runs.  I learned this making my Plex script that you cannot move a file to a destination that does not exist. 

 

Also, I'm not clear on whether it's actually needed or not, but it's good practice to enclose your backup path in quotes since it has spaces in the path:

dir="/mnt/user/NCloud/Mihle/files/Filer/Backup/NAS Flash Backup/"

 

The error message on the mv command indicates that one or both of these are the problem.

 

It's probably also why the rm is failing as well.  During the creation of my Plex script I had this rm command mis-formed and had the same errors as you, and the GUI was totally unresponsive due to it.. had to pull the power plug to reset the server since I couldn't restart it any other way... was not a good time!

 

So I'd change your section 1 to this and see if it helps:

 

#dir = WHATEVER FOLDER PATH YOU WANT TO SAVE TO

dir="/mnt/user/NCloud/Mihle/files/Filer/Backup/NAS Flash Backup/"

mkdir -p "$dir"

 

 

The mkdir -p just ensures that if the path does not exist that it is created.

 

  • Thanks 1
Link to comment
1 hour ago, Energen said:

Does your backup directory (/mnt/user/NCloud/Mihle/files/Filer/Backup/NAS Flash Backup/) already exist?

Yes.

Quote

If not, it needs to be created before the script runs.  I learned this making my Plex script that you cannot move a file to a destination that does not exist. 

 

Also, I'm not clear on whether it's actually needed or not, but it's good practice to enclose your backup path in quotes since it has spaces in the path:

dir="/mnt/user/NCloud/Mihle/files/Filer/Backup/NAS Flash Backup/"

I did not know that

Quote

The error message on the mv command indicates that one or both of these are the problem.

 

It's probably also why the rm is failing as well.  During the creation of my Plex script I had this rm command mis-formed and had the same errors as you, and the GUI was totally unresponsive due to it.. had to pull the power plug to reset the server since I couldn't restart it any other way... was not a good time!

 

So I'd change your section 1 to this and see if it helps:

 

#dir = WHATEVER FOLDER PATH YOU WANT TO SAVE TO

dir="/mnt/user/NCloud/Mihle/files/Filer/Backup/NAS Flash Backup/"

mkdir -p "$dir"

 

 

The mkdir -p just ensures that if the path does not exist that it is created.

Ah, ok, only adding the quotes seem to have fixed it!

Well, It does not show up in Nextcloud as I ideally want, but that might be Nextcloud. At least it doesnt make the folder it is moved to uneditable for Nextcloud as the appdata backup do.

Edited by Mihle
Link to comment
19 hours ago, Mihle said:

Well, It does not show up in Nextcloud as I ideally want, but that might be Nextcloud. At least it doesnt make the folder it is moved to uneditable  and unsyncable for Nextcloud as the appdata backup plugin do.

Probably Nextcloud fault, maybe it dont like putting files to its database without being through its GUI or its own code.


Work around of having a own share just to store the backup and adding it as external storage worked, it then shows up in Nextcloud and syncs to other devices just fine.
Not ideal but it works. Thanks.

Edited by Mihle
Link to comment
  • 9 months later...

Thanks! That’s a good question… might be worth adding a check for that…

 

I’ve got a few irons in the fire at the moment, but I’ll try to write something up for it over the coming weekend if schedule permits. Anyone is free to take a stab at it as well and I’ll add it. 

Edited by Cpt. Chaz
Link to comment
  • 1 month later...

Does someone has a version of this script that just copy the files without making a zip file?

The next time the backup is done is ok if it deletes everything in that folder and put the new files.

 

I'm doing backups with Kopia (highly recommended after trying many) and this would be wonderful to have a decent versioning without taking almost any additional space.

With Kopia docker I don't have permissions to access to the boot folder so this is the only solution so far

 

For example for each snapshot

imagen.thumb.png.4a4b0f363698ba4e1b3b154fc98513ab.png

You can browse it individually and get any file, restore everything... it has encryption deduplication, compression, and it is way faster than duplicati.

imagen.thumb.png.72667bd797b15023a9dc5d7eec647129.png

Edited by L0rdRaiden
Link to comment
3 minutes ago, L0rdRaiden said:

Does someone has a version of this script that just copy the files without making a zip file?

The next time the backup is done is ok if it deletes everything in that folder and put the new files.

 

I'm doing backups with Kopia (highly recommended after trying many) and this would be wonderful to have a decent versioning without taking almost any additional space.

With Kopia docker I don't have permissions to access to the boot folder so this is the only solution so far

The CA Backups app will do uncompressed backups to the location of your choice IIRC.

Link to comment
9 minutes ago, wgstarks said:

The CA Backups app will do uncompressed backups to the location of your choice IIRC.

I know the problem with CA backup is that it can not only do flash backup, you have to do it together with appdata (docker) although I guess I could you a dummy folder and

imagen.png.9351bb8f4194c0d7fbace4c3826ef9d0.png

 

In any case if I backup the USB will CA remove all the files a backup the current status or will only replace o create files but won't delete older ones?

Edited by L0rdRaiden
Link to comment
6 hours ago, L0rdRaiden said:

Does someone has a version of this script that just copy the files without making a zip file?

The next time the backup is done is ok if it deletes everything in that folder and put the new files.

 

This script is built on the native unraid script that includes the zip function. However, it would be easy enough to add an unzip function to my backup script for you to use. I wrote this fairly quickly and only tested once to make sure it worked. Looks like it did, but please use caution until you've tested and are certain. Also, this assumes you only have the one single zip file in $dir. the way it's written, it will unzip every zip file in $dir. Just copy and paste to the end of your script. Hope it helps.

 

#### SECTION 7 ####------------------------------------------------------------------------------------------------------

echo 'Unzip Flash Backup'

cd "$dir"
unzip *.zip

#UNCOMMENT THE NEXT LINE IF YOU WISH TO DELETE THE ORIGINAL ZIP FILE AFTER EXTRACTION
#rm -rfv *.zip

 

  • Thanks 1
Link to comment
4 hours ago, Cpt. Chaz said:

 

This script is built on the native unraid script that includes the zip function. However, it would be easy enough to add an unzip function to my backup script for you to use. I wrote this fairly quickly and only tested once to make sure it worked. Looks like it did, but please use caution until you've tested and are certain. Also, this assumes you only have the one single zip file in $dir. the way it's written, it will unzip every zip file in $dir. Just copy and paste to the end of your script. Hope it helps.

 

#### SECTION 7 ####------------------------------------------------------------------------------------------------------

echo 'Unzip Flash Backup'

cd "$dir"
unzip *.zip

#UNCOMMENT THE NEXT LINE IF YOU WISH TO DELETE THE ORIGINAL ZIP FILE AFTER EXTRACTION
#rm -rfv *.zip

 

Thanks a lot, I would also need to delete all the content of the folder before the unzip, so the old files won't appear anymore and the unzip or the copy of the file starts always in a clean folder

Link to comment
59 minutes ago, Cpt. Chaz said:

No problem.
 

Does section 5 not do this for you?

 

Yes you are right, so I used my advanced skills in scripting programming 😁 and swapped section 4 and 5, so the files are deleted (mtime +0) and then the zip file is moved and unzipped. I won't delete the original zip (section 6) file but I will exclude that file in the backup system.

 

Now looks perfect for my use case, thanks a lot :)   (EDIT: see next posts, still doesn't work)

 

#!/bin/bash

#### SECTION 1 ####------------------------------------------------------------------------------------------------------
#dir = WHATEVER FOLDER PATH YOU WANT TO SAVE TO
dir="/mnt/user/Backup/UnraidFlash/"

echo 'Executing native unraid backup script'
/usr/local/emhttp/webGui/scripts/flash_backup

#### SECTION 2 ####------------------------------------------------------------------------------------------------------
echo 'Remove symlink from emhttp'
find /usr/local/emhttp/ -maxdepth 1 -name '*flash-backup-*.zip' -delete

sleep 5

#### SECTION 3 ####------------------------------------------------------------------------------------------------------
if [ ! -d "$dir" ] ; then
 
			echo "making directory as it does not yet exist"

			# make the directory as it doesnt exist
			mkdir -vp "$dir"
		else
			echo "As $dir exists continuing."
			fi


#### SECTION 4 ####------------------------------------------------------------------------------------------------------
echo 'Deleting Old Backups'

#ENTER NUMERIC VALUE OF DAYS AFTER "-MTIME +"
find "$dir"* -mtime +0 -exec rm -rfv {} \;

echo 'All Done'

sleep 5

#### SECTION 5 ####------------------------------------------------------------------------------------------------------
echo 'Move Flash Zip Backup from Root to Backup Destination'
mv /*-flash-backup-*.zip "$dir"

sleep 5

#### SECTION 6 ####------------------------------------------------------------------------------------------------------
echo 'Unzip Flash Backup'
cd "$dir"
unzip *.zip

#UNCOMMENT THE NEXT LINE IF YOU WISH TO DELETE THE ORIGINAL ZIP FILE AFTER EXTRACTION
#rm -rfv *.zip

#### SECTION 7 ####------------------------------------------------------------------------------------------------------

#UNCOMMENT THE NEXT LINE TO ENABLE GUI NOTIFICATION UPON COMPLETION 
/usr/local/emhttp/webGui/scripts/notify -e "Unraid Server Notice" -s "Flash Zip Backup" -d "A copy of the Unraid flash disk has been backed up" -i "normal"

exit

 

Edited by L0rdRaiden
Link to comment

@Cpt. Chaz  well I am not as good as I though

 

imagen.thumb.png.1b25f2755061b5ea4f30d5aa70f35eaf.png

 

For some reason

  • Section 4 for doesn't delete previous zip files but deletes uncompressed files
  • Section 6 fails to unzip the file because I guess there are more than 1 file that matches, right?

If we can fix section 4 I guess section 6 will work, it should be easy but I am afraid to experiment with the rm command.

 

Maybe replacing section 4 with this will fix everything?

cd "$dir"
rm -rfv
echo 'All files deleted'

 

Edited by L0rdRaiden
Link to comment

you're on the right track! however the rm syntax requires a target. try this instead

 

rm -rfv "$dir"*
echo 'All files deleted'

 

 

if you want to confirm this works before trying on "mission critical" files, take everything out of $dir. Then cd into the directory, and run this

touch test1.txt test2.txt
mkdir testfolder

this just simply puts in a couple basic files and a directory. If the commands i gave you for section 4 work correctly, it should remove these test files, and the rest of the flow should result in the latest unzipped backup copy of your flash drive.

 

  • Thanks 1
Link to comment
46 minutes ago, Cpt. Chaz said:

you're on the right track! however the rm syntax requires a target. try this instead

 

rm -rfv "$dir"*
echo 'All files deleted'

 

 

if you want to confirm this works before trying on "mission critical" files, take everything out of $dir. Then cd into the directory, and run this

touch test1.txt test2.txt
mkdir testfolder

this just simply puts in a couple basic files and a directory. If the commands i gave you for section 4 work correctly, it should remove these test files, and the rest of the flow should result in the latest unzipped backup copy of your flash drive.

 

 

Thanks, it works perfectly  :) I have learnt a lot with you

 

Together with this Script (the first one in my life) and Kopia I have tune up my backup system 

 

#!/bin/bash
/usr/local/emhttp/webGui/scripts/notify -e "Unraid Server Notice" -s "OneDrive backup started" -d "Synchronizing Kopia backup to OneDrive" -i "normal"
echo 'OneDrive backup started'

rclone sync /mnt/user/Backup/Kopia OneDrive:"/Backup/Kopia"

/usr/local/emhttp/webGui/scripts/notify -e "Unraid Server Notice" -s "OneDrive backup finished" -d "Synchronization completed" -i "normal"
echo 'One Drive backup completed'

 

  • Like 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.