[Support] binhex - UrBackup

[Masterwishx]

Using binhex-urbackup for a while and all fine , but on clients i dont have a local time,

in the server its OK, how to fix it ,should i add (-v /etc/localtime:/etc/localtime:ro \) in container for it ?

 

Server:

 

Client

[bigsub]

On 9/12/2022 at 6:21 PM, KluthR said:

Basically you remove UMASK, PUID and PGID variables and change the the variable for the backup to point to "/backups" (container path) and the appdata to "/var/urbackup" (container path).

 

Change "Repository" to: uroni/urbackup-server:latest

Change "Docker Hub URL" to: https://hub.docker.com/r/uroni/urbackup-server

If you want you can use the Icon @ 

https://forums.urbackup.org/uploads/default/original/2X/1/1051fd74d1dcbc3ad4220b43007fcab5287272b0.png

 

Add a new variable, named "TZ" and enter your right Timezone (e.g. "Europe/Berlin").

 

The template for the docker container by uroni:

 

<?xml version="1.0"?>
<Container version="2">
  <Name>UrBackup</Name>
  <Repository>uroni/urbackup-server:latest</Repository>
  <Registry>https://hub.docker.com/r/uroni/urbackup-server</Registry>
  <Network>host</Network>
  <MyIP/>
  <Shell>bash</Shell>
  <Privileged>false</Privileged>
  <Support/>
  <Project/>
  <Overview/>
  <Category/>
  <WebUI>http://[IP]:[PORT:55414]/</WebUI>
  <TemplateURL/>
  <Icon>https://forums.urbackup.org/uploads/default/original/2X/1/1051fd74d1dcbc3ad4220b43007fcab5287272b0.png</Icon>
  <ExtraParams/>
  <PostArgs/>
  <CPUset/>
  <DateInstalled>1662522898</DateInstalled>
  <DonateText/>
  <DonateLink/>
  <Requires/>
  <Config Name="TimeZone" Target="TZ" Default="" Mode="" Description="" Type="Variable" Display="always" Required="false" Mask="false">Europe/Berlin</Config>
  <Config Name="Backup Folder" Target="/backups" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/BackupsV2</Config>
  <Config Name="Database Folder" Target="/var/urbackup" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/cache/appdata/urbackup</Config>
</Container>

 

Hello, with these changes could I migrate all configurations and data from binhex to uroni, without lossing anything?

[KluthR]

Yes. I did it this way.

 

Make a backup if smth. goes wrong.

[DeadMode]

Just a quick one that I wonder if anyone can help with. Loving the UrBackup container, thought it was going to solve all my backup needs.

 

My main issue really is that I using it to backup a few Windows machines in my network, and the backups initially go to my cache in Unraid. Mover doesn't seem to be able to cope with the volume of files being backed up, as it's very slow to shift anything off the cache following a backup. I have a 512GB NVMe SSD for the cache (Samsung 980 pro) and the main array is just an 8TB 7200rpm Sata disk. No other issues with the setup, but mover is incredibly slow shifting the backups over to the array.

 

Obviously I could have UrBackup write to it directly, but kinda defeats the point of having the cache in the first place really. Just trying to move one full backup of my main PC seems to bring it to it's knees (over 100,000 files). It is moving them, just very slowly.

 

Perhaps I'm not being patient enough, I'm going to try stopping other services today like UrBackup and my Windows vm which syncs the array up to the cloud using Backblaze.

 

I guess once it's finished the full backup, the following incremental backups in theory should be much smaller.

[Kilrah]

You should really have Cache:No on the urbackup share, cache will be extremely slow as you can see and potentially break stuff given urbackup uses hardlinks.

It's perfectly fine just direct to array.

Edited December 6, 2022 by Kilrah

[DeadMode]

27 minutes ago, Kilrah said:

You should really have Cache:No on the urbackup share, cache will be extremely slow as you can see and potentially break stuff given urbackup uses hardlinks.

It's perfectly fine just direct to array.

 

Thanks, seems a shame not to be able to use the cache, but I'll be hopefully be putting an extra SSD in the system soon and may just backup directly to that, no moving about etc..

 

edit: Just to add, mover has been running for about 30mins now with my Vm shutoff and anything else that would touch the cache, and it's speeded up a great deal and shifted about 60,000 files in that time. So I guess the SATA drive was slowing things down with being accessed for other things when mover was running previously.

 

If the advice is not to cache backups though, I'll stick with that as I don't want to risk file corruption or a backup that I can't use in the future.

Edited December 6, 2022 by DeadMode

[binhex]

On 12/6/2022 at 9:40 AM, DeadMode said:

If the advice is not to cache backups though

i wouldn't let's put it that way, too many small files, plus urbackup makes use of symlinks quite heavily, those two combinations make me nervous about a bash script moving files around, but the choice is yours :-).

[Nepherim]

On 11/29/2020 at 2:08 PM, binhex said:

by look of it:- /usr/local/bin/urbackupclientctl

as its installed to ram (keep in mind everything is in ram in unraid, other than cache, array disks and flash drive), so you will probably(i dont know yet), have to re-run the script i mentioned above, or perhaps just copy the urbackupclient that gets built and shove that on your flash drive and then copy that to /usr/local/bin/ and execute that as part of the go script

Is anyone able to share exactly how to do this? Which directory on the flash-drive would it be appropriate to store a copy of the client directory? I can do the copy from flash to /usr/local/bin/urbackupclientctl, but not sure how to actually have the client start up.

[binhex]

Is anyone able to share exactly how to do this? Which directory on the flash-drive would it be appropriate to store a copy of the client directory? I can do the copy from flash to /usr/local/bin/urbackupclientctl, but not sure how to actually have the client start up.

You can put the urbackup client wherever you want on the flash drive, create a new root folder if you wish and copy it to there, then In the 'go' file copy from flash to /usr/local/bin/ and chmod the file.

Sent from my 22021211RG using Tapatalk

[Nepherim]

Here's where I'm at in terms of using urBackup as a means to backup Unraid shares.

 

You need to backup a bunch of files and directories, permissions/execute flags are important. After installing urbackup client on your server, use the tar command below to make a copy of the urbackup client. (Use the server to configure paths and other client settings.) This will create a directory on your flash drive /boot/urbackup with the tar file inside:

tar -cvf /boot/urbackup/urbackup.tar \
    /usr/local/var/urbackup \
    /usr/local/bin/urbackupclientctl \
    /usr/local/sbin/uninstall_urbackupclient \
    /usr/local/sbin/urbackupclient_dmsnaptool \
    /usr/local/sbin/urbackupclientbackend \
    /etc/default/urbackupclient

 

Then update the /boot/config/go file:

nano /boot/config/go

 

and add the following to ensure urBackup is reinstalled and restarted on your Unraid server after reboot.
 

tar -xvf /boot/urbackup/urbackup.tar -C /
/usr/local/sbin/urbackupclientbackend -d

 

 

What I'm now stumped on is how to actually have the client run. Anyone able to help identify where/how to run the client? My local client uses systemd, which isn't what unraid uses, so kind of stumped at this point.

 

Edit 1: updated to add:

Here's what the systemd service runs, which (removing the ExecStart= part) fails since there is no /etc/sysconfig on Unraid, nor is there a urbackupclient anywhere.

ExecStart=/usr/local/sbin/urbackupclientbackend --config /etc/sysconfig/urbackupclient --no-consoletime

 

Presumably  urbackup has an init file somewhere, but not able to find it.

 

Edit 2: found the daemon

Updated process above to run the daemon:

/usr/local/sbin/urbackupclientbackend -d

 

Edited December 15, 2022 by Nepherim

[Nepherim]

Having moved backup files to a different disk using Unablance plugin and the -H parameter, I'm now trying to clean-up symbolic links with remove-unknown:

sh-5.1# urbackupsrv remove-unknown
2022-12-12 22:28:39: Going to remove all unknown files and directories in the urbackup storage directory. Waiting 20 seconds...
2022-12-12 22:28:59: Shutting down all database instances...
2022-12-12 22:28:59: Opening urbackup server database...
2022-12-12 22:28:59: WARNING: SQLite: cannot open file at line 39363 of [3bfa9cc97d] errorcode: 14
2022-12-12 22:28:59: WARNING: SQLite: os_unix.c:39363: (13) open(/var/urbackup/backup_server_files.db) -  errorcode: 14
2022-12-12 22:28:59: Could not open db [urbackup/backup_server_files.db]
2022-12-12 22:28:59: ERROR: Database "urbackup/backup_server_files.db" couldn't be opened
2022-12-12 22:28:59: ERROR: Couldn't open backup server database. Exiting. Expecting database at "/var/urbackup/backup_server_files.db"

Any ideas?

 

Edit 1: Solution

Solution is to ensure the command is running as root user:

urbackupsrv remove-unknown -u root

 

Edited December 15, 2022 by Nepherim

[Masterwishx]

Wanted to ask about Cache disk for urBackup , im using it for while and no problems at all .

but today founded that Backup share is unprotected becose of urBackup directory in it. and some Dir/files is still on cache.

Using cache Yes for Backup share, so when used mover manualy  the dirs was not found . 

On cache was some dir,links but cant be moved from cache. Tryed to move them manualy in mc,krusader but some was exist on array so decided to remove dirs on cache at end ...

The question is if some one had this situation and fixed it somehow? 

 

[Nepherim]

Moving backups from one volume to another needs to be done carefully as urBackup uses hard links extensively. I just moved my backups from one unraid disk to another using Unbalance mod, and the -H parameter, and saw a large spike in incremental backup szes, I suspect due to messed up links. Not sure I'd go the route of backing up to cache.

[Masterwishx]

2 hours ago, Nepherim said:

Moving backups from one volume to another needs to be done carefully as urBackup uses hard links

 

using cache yes but then mover move all files to array, the strange thing all backup befor today its on array but today some linked folders not moved . and mover manually cant find folders ...

also found the mover take about 2 hours to move 7GB (40000 files) from cache to array.

unBalance app move only from/to Disks in array i cant see my cache pools in it.

 

[Masterwishx]

On 12/7/2022 at 11:48 AM, binhex said:

i wouldn't let's put it that way, too many small files, plus urbackup makes use of symlinks quite heavily, those two combinations make me nervous about a bash script moving files around, but the choice is yours :-).

 

Your advice is to use cache pool or not ?

[Kilrah]

I'm not him but I really would recommend against it. Performance is good enough straight to array in the first place, my incrementals usually take <10mins.

 

 

 

[Masterwishx]

 

is this right of container making folder/files (775)?

the mask still 000 but i think it was befor (777) 

[Masterwishx]

12 hours ago, Kilrah said:

I really would recommend against it

 

Thanks i will try without cache ,in my case the last backups not all files moved from cache becose linked folders maybe ,also worried about long time of moved files from cash.

snapshots working fine

Edited December 20, 2022 by Masterwishx

[bigsub]

On 12/4/2022 at 9:28 AM, KluthR said:

Yes. I did it this way.

 

Make a backup if smth. goes wrong.

Hello, I've just update to uroni urbackup. Urbackup server started but empty, without any clients and previous backups..where I was wrong?
When urbackup started, it upgrade database version from old version to version 65.

[bigsub]

Just another doubt, removing PUID and PGID from template file, where I must to specify the user and what permission I must to assign to backup folder?

[Evolze]

Hi everyone,

 

I'm fairly new to Unraid still and definitely still trying to get the hang of Docker too. I just recently got this container installed and everything appears to be running fine. However, there seem to be some frightening error messages and this is making me hold off on beginning to use this fully. When looking at Unraid's syslog, the container starts up fine. However, when looking at the container logs, I get the following:

 

2022-12-28 04:00:53,971 DEBG 'urbackup' stderr output:
Raising maximum file descriptor to 65535 failed. This may cause problems with many clients. (errno=1)
Raising nice-ceiling to 35 failed. (errno=1)

2022-12-28 04:00:53,983 DEBG 'urbackup' stdout output:
Backupfolder not set

2022-12-28 04:00:53,987 DEBG 'urbackup' stdout output:
Backupfolder not set

 

In addition, here is what I see in the Unraid Syslog when turning the container off via the Web GUI. It seems very concerning.

Is this expected behavior for anyone else?

 

Dec 28 04:16:08 Tower kernel: urbackupsrv[4176]: segfault at 0 ip 0000000000000000 sp 00007ffdb1ce84b8 error 14
Dec 28 04:16:08 Tower kernel: Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6.

 

Here is a screenshot of my Docker configuration.

 

 

 

 

Any assistance or ideas into fixing these?

Edited December 28, 2022 by Evolze

[Evolze]

An edit / addition to my post above from yesterday. After clearing out everything (appdata), I re-downloaded the container again. I believe I found out why it says:

 

2022-12-28 04:00:53,987 DEBG 'urbackup' stdout output:
Backupfolder not set

 

From within the container, I was looking at the start.sh script within the root user directory (/root/start.sh) and found something interesting. It has this listed, which I believe is not set correctly:

 

# set default location for backup storage to /media
  echo "/media" > /var/urbackup/backupfolder

 

I checked in /var/urbackup/backupfolder and it is not a directory @binhex It simply creates a file called backupfolder that has "/media" printed within it. I'm not sure if this is supposed to behave this way but it does not create a directory. There might need to be an update to the docker container itself to fix this? I'm still fairly new to Linux, but maybe a Symlink instead?

 

Anyways, hope I'm not the only one getting this 'Backupfolder not set' message. I'm just wanting everything sorted out before I start using it full-time. 😀

 

Especially this message as it appears to be the most concerning when turning off the container.

Dec 28 04:16:08 Tower kernel: urbackupsrv[4176]: segfault at 0 ip 0000000000000000 sp 00007ffdb1ce84b8 error 14
Dec 28 04:16:08 Tower kernel: Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6.

 

Edited December 29, 2022 by Evolze

[primeval_god]

6 hours ago, Evolze said:

# set default location for backup storage to /media
  echo "/media" > /var/urbackup/backupfolder

 

I checked in /var/urbackup/backupfolder and it is not a directory @binhex It simply creates a file called backupfolder that has "/media" printed within it. I'm not sure if this is supposed to behave this way but it does not create a directory. There might need to be an update to the docker container itself to fix this? I'm still fairly new to Linux, but maybe a Symlink instead?

This does in fact need to be a text file named "backupfolder" that has the path of the backup folder in it. The thing that may be wrong is the location. From what i understand it needs to be "/etc/urbackup/backupfolder". Bind mounting a file containing /media to that location fixed the Backup Folder not set messages  for me. I cant speak to the others.

[primeval_god]

Has anyone managed to setup internet urBackup clients through a reverse proxy (specifically traefik but i dont think the proxy is the issue) using websockets and basic auth? I have been following the relatively sparse information from the urbackup site and forums but I find myself stuck. Nearest i can tell despite embedding the basic auth credentials in the url the client doesnt send them with its websocket connect request. Anyone encountered something similar?

[Evolze]

22 hours ago, primeval_god said:

This does in fact need to be a text file named "backupfolder" that has the path of the backup folder in it. The thing that may be wrong is the location. From what i understand it needs to be "/etc/urbackup/backupfolder". Bind mounting a file containing /media to that location fixed the Backup Folder not set messages  for me. I cant speak to the others.

 

Could you please tell me how you did this from within the container's filesystem itself? Did you just make a directory in /etc/urbackup/backupfolder? Or did you create a symlink to it at all? I tried just about everything and cannot seem to get rid of this error. It shouldn't nag me this much but it does 😅