rsync Incremental Backup

[Stupot]

2 hours ago, mgutt said:

Does not really make sense as you said you used -a, which already includes -o and -g on your first sync?! Another strange thing: If you use rsync though the terminal, it is executed as root, so if -o and -g wouldn't be active, the files should get root:root and not nobody:users.

 

Hey,

 

I actually did the copying using a Krusader docker.  When I first started using Unraid, I watched a lot of SpaceInvader One's guides - one of which was about moving data around. I needed to move data from my old NAS to the new one, and the method SIO explained was using Krusader, and it has worked fine whenever I needed to move data.  Since it was so much data I was moving, I wanted a way of being able to see what is happening in the terminal window. As I understand it, if I had run the command in a shell window from Unraid GUI, I would have had to keep that window open for it to keep going.  Using Krusader just seemed like a more stable way of doing it.

 

I am making an educated guess here: In order for the Krusader docker to be able to copy the files while maintaining user and group, the docker needs root access, which is not enabled by default...and was not enabled when I copied the da;ta over. So I would guess that is why the files copied, but the user and group was not.  Would likely also explain why it was nobody:users?  

 

When I ran "rsync -avhH" WITHOUT root access (as per the instruction from one of SIO's videoes) - everything copied with no errors. When I tried "rsync -avhogH" without access, there were loads of errors saying it didn't have permission to do something (I can't recall the exact wording) - trying the same command with root access enabled it worked fine and backups seem to be working spot on now.

 

I'll be honest, I find it confusing understanding what a lot of the arguments for command line applications mean. I had no idea that "a" would essentially include o and g with it.  Since it didn't error when copying the data, I assume that copying the user:group is an optional thing?  Like "try and copy the user and group if you can but if you can't just skip it and give it a default user:group" type deal?

 

Thanks for taking the time to read.

 

One other thing: is there anything one can add to the end of your script that once it's finished it will change the owner of the .log files the script generates?  They are owned by "root" as default and so am unable to open the log files over the network.  Hopefully just a simple chown command for the log file(s) at the end of the script?

 

Thanks!

[mgutt]

1 hour ago, Stupot said:

Would likely also explain why it was nobody:users?  

Yes, this is the reason.

 

1 hour ago, Stupot said:

trying the same command with root access enabled it worked fine

Then -o and -g is not needed, but it does not cause problems to use it in combination with -a. Rsync does not warn you, if you use options twice.

 

1 hour ago, Stupot said:

As I understand it, if I had run the command in a shell window from Unraid GUI, I would have had to keep that window open for it to keep going.  Using Krusader just seemed like a more stable way of doing it.

Two hints:

- The File Explorer Plugin has a move feature included, acts as root and works even if the browser window has been closed. But I'm not sure if Hardlinks are preserved 🤷

Downside: It checks for each file if there is access, which makes it a little bit slower than rsync alone. But I'm not sure r

- You can execute rsync as follows to be able to close the Terminal:

rsync -aH src/ dst & disown

Downside: You need to use kill <pid> to stop rsync.

 

[Stupot]

On 10/1/2024 at 10:08 AM, mgutt said:

Yes, this is the reason.

 

Then -o and -g is not needed, but it does not cause problems to use it in combination with -a. Rsync does not warn you, if you use options twice.

 

Two hints:

- The File Explorer Plugin has a move feature included, acts as root and works even if the browser window has been closed. But I'm not sure if Hardlinks are preserved 🤷

Downside: It checks for each file if there is access, which makes it a little bit slower than rsync alone. But I'm not sure r

- You can execute rsync as follows to be able to close the Terminal:

rsync -aH src/ dst & disown

Downside: You need to use kill <pid> to stop rsync.

 

 

Thanks, everything makes sense now haha!

 

Thank you for the tips too!  I do use the file manager plugin for smaller transfers - but using rsync in the Krusader docker's terminal feels way more reliable for larger transfers.  Now I've got the root access issue sorted hopefully will be my go to!  Though will defo keep your "disown" trick in mind for future!

Thank you!

[Jensus11]

Hello,
I have the problem that the Docker containers do not start again after the backup has been performed. What could this be?

[ChatNoir]

11 minutes ago, Jensus11 said:

Hello,
I have the problem that the Docker containers do not start again after the backup has been performed. What could this be?

If I am not mistaking, there is a log create with the backup. Have you checked the content ? It could point you in the right direction.

[Jensus11]

I looked at the log file. Unfortunately I can't find any information about stopping or starting.

[Jensus11]

This morning the same thing happened again. After the backup all containers were shut down and I had to start them again manually.

[mgutt]

Everything is logged: Stopping, Snapshot Creation and Starting the containers. So check the logs.