Skip to content
View in the app

A better way to browse. Learn more.

Unraid

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

rsync Incremental Backup

Featured Replies

Wow, many thanks for the complete solution!

I set it up but I am not sure if that really works on my server, because tcpdump is not recognized as command.

I presume I have to install this first, right?

  • Replies 426
  • Views 132.4k
  • Created
  • Last Reply

Top Posters In This Topic

Most Popular Posts

  • Released v1.4 with the following new features:   1.) The script can be now called with source and destination path as arguments   Example: /usr/local/bin/incbackup /src_path /ds

  • My plan is to realize a plugin out of it. So it's not a good idea to publish it at Github at this time. It will be done if it reaches the plugin state.

  • At the moment I'm working on the next release. Many people asked for SSH destinations and I'm still working on that. I created some really nice hacks to reach this:   As an example I use thi

Posted Images

  • Author
31 minutes ago, toasti said:

I presume I have to install this first, right?

Yes sorry. Nerd Pack is what you need ;)

Awesome! Works like a charm!

I will setup another one with another port for my wife's desktop and we are done ;-)

 

Many thanks! 😄

Another question... how did you solve it that your Backup user can access your Windows profile and things like desktop, documents, etc?

Did you add the user to the admin group? Add NTFS read permissions for the profile folder?

All not ideal...

  • Author

Add a new local user through the settings with a strong password and name it "backup". After that right mouse click on "documents" > share with... Select the new user from the drop-down. It should be "read only". Repeat this step for all dirs you want to share. Do not share the root username dir. It contains the hidden "appdata" dir and a massive amount of other files which aren't needed.

 

After that you can mount this "smb server" through the Unassigend Disks Plugin.

That is clear ;-) I did the same, but the problem is that other users have no NTFS permissions for your profile folder.

I always use advanced sharing, maybe the easier sharing option also sets NTFS permissions. Not sure. Never used it the last years.

 

EDIT: Yes, tested it: NTFS permissions are automatically set with the easier sharing option.

I will create a group, add the Unraid-Backup-User to the group and give the permissions to the group.

Then I am flexible if I want to change anything for the user accounts...

It's always best practice to use groups.

 

By the way, I created a share for the complete C$ because I want also to backup other folders in my profile, also some program folders in AppData etc. In the script I just added the path behind and all is fine...

Edited by toasti

  • Author
18 minutes ago, toasti said:

By the way, I created a share for the complete C$ because I want also to backup other folders in my profile, also some program folders in AppData etc. In the script I just added the path behind and all is fine...

Ok, but it takes much more time and the logs will contain several permission errors because of hidden redirects. But you will find them in the backup log ;)

No problem, I will check this as soon as I did add the first backups within AppData.

 

How this works now is really brilliant.

 

I just thinking about how to do a second copy which I save in the office and refresh every week with a new backup.

Probably the easiest one is to create 2 more scripts that will copy the data to the second and third external drive.

Same config, just another target path and manually started.

 

 

Edited by toasti

Any idea how to send the external backup hard disk to stand by when no backup is running?

Edited by toasti

  • Author

The UD Plugin should spindown the disk I think. Please ask in the support thread of UD.

Actually the latest versions of UD relinquished spin control to unRAID so disks in UD spindown according to the default unRAID spindown timer.

 

Maybe you just have to wait longer or maybe you're encountering the same problem I did - where the USB drive returns an error when spindown's called so even though it's spun down unRAID thinks it's active.

 

I have a fix but I'm not confident enough in it to really recommend it:

 

 

  • Author

I found a bug. My PC is mounted through SMB. The backup runs every 5 hours. Now I found out that my sleeping PC returned a succesful backup, although rsync copied only the root dir of the SMB mount and returned no other errors:

 

Create backup of /mnt/remotes/DESKTOP-I0HHMD9_Users
Backup path has been set to /mnt/user/Backup/marc/remotes/DESKTOP-I0HHMD9_Users
Create incremental backup 20210321_000001 by using last backup 20210320_200001
sending incremental file list
DESKTOP-I0HHMD9_Users/

Number of files: 1 (dir: 1)
Number of created files: 0
Number of deleted files: 0
Number of regular files transferred: 0
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 75
Total bytes received: 27

sent 75 bytes  received 27 bytes  204.00 bytes/sec
total size is 0  speedup is 0.00

 

 

The backup dir is empty:

1017178472_2021-03-2211_11_48.thumb.png.ee6073d86dd089efe275a9b38a6047b9.png

 

The last backup exists:

1533757857_2021-03-2211_12_27.thumb.png.cc506c0048b254ec70fd209d2b505544.png

 

This bug is not dangerous, because after powering on my PC, the new backup is created as usual:

611151545_2021-03-2211_14_01.thumb.png.e77de9735116bafe2e42d798c2fbd4b5.png

 

But its not incremental anymore as it was compared with the last empty backup.

 

I don't know why the SMB mount returns an empty result. Shouldn't it return "host is offline"?! I need to check that further while my PC is sleeping.

 

Maybe its the best to mark backups as failed if they contain only one empty dir.

 

 

Just to be sure, I need a BTRFS formatted external hard disk that it works correctly (hard links, inodes, etc) - right?

Yesterday I had the idea to map my external disk (connected to my W10 machine, NTFS formatted and Bitlocker encrypted) to Unraid and use your script to make a backup copy which I want to store in my office.

On 1/3/2021 at 2:40 AM, mgutt said:

Ah ok. You passed multiple source paths and by that it uses one log for all source paths. This bug will be solved in the next version. I already fixed it, but I'm still testing the next release ;)

Hello,

 

I use today the script in v0.6 and i still have the problem with the log files when using multiple source paths :

Same result as in v0.3:

- the first log file is full with the logs of all source path backups

- only the last log file is OK with only the log of the last source path backup.

 

Fred.

  • Author
1 hour ago, hf00 said:

i still have the problem with the log files when using multiple source paths :

Yes, this bug is still present. I tried to find out how to stop forwarding the script output to a file, but wasn't successful. I think I will try to forward the output to an own function. Maybe I can replace only the filename so "exec" isn't called multiple times.

  • 2 weeks later...

Hi all.....I have noticed that I am unable to unmount the USB disk even after the backup script finishes. I run it in background mode and it apparently affects the unmounting of the drive. I click unmount on the UD entry and it pauses a split second and stays mounted. Looking at the drive settings below it shows the backup script. Is there a command I can run to stop the script manually?

 

 

Capture.JPG

  • 2 weeks later...

Hi,

i tried the script an it works wonderful.

One thing i would like to see added, is the possibility to exclude Folders.

For example ".Recycle.Bin". Or other specified folders like home videos in user shares.

On 3/22/2021 at 11:19 AM, mgutt said:

I found a bug. My PC is mounted through SMB. The backup runs every 5 hours. Now I found out that my sleeping PC returned a succesful backup, although rsync copied only the root dir of the SMB mount and returned no other errors:

 





Create backup of /mnt/remotes/DESKTOP-I0HHMD9_Users
Backup path has been set to /mnt/user/Backup/marc/remotes/DESKTOP-I0HHMD9_Users
Create incremental backup 20210321_000001 by using last backup 20210320_200001
sending incremental file list
DESKTOP-I0HHMD9_Users/

Number of files: 1 (dir: 1)
Number of created files: 0
Number of deleted files: 0
Number of regular files transferred: 0
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 75
Total bytes received: 27

sent 75 bytes  received 27 bytes  204.00 bytes/sec
total size is 0  speedup is 0.00

 

 

 

 

where to get that summary ? I doubt the backup went through ... on a very large Folder ( ~ 22TB)

The log what i got looks like this

 

reate backup of /mnt/remotes/192.168.1.3_Pics
Backup path has been set to /mnt/HDD/Pics/remotes/192.168.1.3_Pics
Create full backup 20210508_062306
sending incremental file list
192.168.1.3_Pics/
192.168.1.3_Pics/Treesize.xml
192.168.1.3_Pics/Untitled.dfd
192.168.1.3_Pics/duplicates.xml
192.168.1.3_Pics/-- new entries !/
192.168.1.3_Pics/-- new entries !/2021-07.05 001.jpg
192.168.1.3_Pics/-- new entries !/2021-07.05 002.jpg

....

~ roughly 1TB copied over then suddenly stopped

script stopped somewhere without error entry ?

**

Update script makes Browser Tab Crash

after thats script is closed ...

where to dig out the error ?

 

 

 

 

 

 

Edited by Dtrain

  • Author
58 minutes ago, Dtrain said:

Update script makes Browser Tab Crash

after thats script is closed ...

where to dig out the error ?

 

Do not execute the script through the browser. Only in the background oder through Cron. The user scripts plugin is not memory friendly enough to display a huge amount of logs in the popup.

Nice work.  Can we make this script get a file or folder from a unix like system that allows ssh / scp / sftp?  I need to make back-ups of some remote linux machines directories 

  • Author
7 hours ago, johnwhicker said:

allows ssh / scp / sftp?

I'm working on that. Please do not modify my script. Someone tried that and because of the last cleanup through "rm -r" all his data gone lost. I want to change everything to rsync. After that SSH / rsync daemon will work, too.

5 hours ago, mgutt said:

I'm working on that. Please do not modify my script. Someone tried that and because of the last cleanup through "rm -r" all his data gone lost. I want to change everything to rsync. After that SSH / rsync daemon will work, too.

Thanks much Sir

On 5/15/2021 at 10:13 AM, mgutt said:

I'm working on that. Please do not modify my script. Someone tried that and because of the last cleanup through "rm -r" all his data gone lost. I want to change everything to rsync. After that SSH / rsync daemon will work, too.

 

Do you mean, you want to change everything to rclone?

Is it possible to make Backups encrypted to my Cloud Storage?

 

 

Thank you very much for your great Work!!!

Edited by schoppehermann

  • Author
2 hours ago, schoppehermann said:

Do you mean, you want to change everything to rclone?

Rclone? I wrote rsync ;)

 

2 hours ago, schoppehermann said:

Is it possible to make Backups encrypted to my Cloud Storage?

Not a the moment. 

Short question. What happens, if there is not enough free space on the destination?

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.