KluthR

Small change here: If a start fails (for whatever reason), the script now tries 3 times to start the container with a delay of 5 secs. If it starts within this time, all is good. If not, an info is sent at the end. With that, I try to work around the observed behaviors. I test it further on my system for some days. @CS01-HSdid you observed not starting containers again after your initial post?

Are those actions not logged via docker.log? I remember, that I got something out of docker, using docker run: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: time="2022-11-09T17:46:39+01:00" level=fatal msg="failed to set IPv6 gateway while updating gateway: file exists": unknown Error: failed to start containers: mysql I dont know if this is a docker issue only here. However, does the API (used by DockerClient) return more info here, which could be added to the return variable?

Hi, I use the https://github.com/limetech/webgui/blob/master/plugins/dynamix.docker.manager/include/DockerClient.php to stop/start some containers. It returns `$code` which is normally "1". But sometimes I get "Server error" back. But I dont see any messages inside the system log. What could cause "Server error" and how could I debug it? Thanks in advance! nas-diagnostics-20221129-1738.zip Unraid 6.11.5

Hmm weird. How many files are backed up? Maybe millions of small files could decrease backup performance? it is normal that you see only one core being used - tar does not need more cores. The limit is the disk i/o. for compression: gz is not multicore cabable. pigz would be. But disabling compression does not making a difference for you is interesting.

Did you adjusted the source in settings??

Could you be more specific? I dont really get what you mean.

I just removed any PR for my fixes and improvements due to some disorder of my changes built on top of pending changes. I merged now all together and will create a new PR with ALL changes. New changes as of today: Use dynamix DockerClient methods for stopping/starting containers and get rid of the shell_execs. This handles some extra things, which are done inside docker manager Fixed BackupPage: Missing end tags for bold text, which lead to make all following pages bold Copy logfile for backup run to the output directory as well I created a new plg with all new changes for testing. If anyone want to test, PM me. Ill test it myself a few days and open a new PR.

Exactly. It should be starting with 0 4, not * 4.

That seems incorrect - could you post your schedule settings from inside the plugin settings page? The current setting says "At every minute past hour 4 on Monday, Wednesday, and Friday."

Interesting, the backup seem to start 2 times. Could you post the output of "crontab -l" inside an unraid terminal?

We‘ll see. I dont know the status. All changes are ready 2 review. The plugin also could do encryption. I also would do some refactoring of the whole code if I continue developing.

Not at all, yet. 😅

Please clarify - you want to select the single archives which to restore and which not? I want this merged, of course! As soon as Andrew has time to review everything (https://github.com/Squidly271/ca.backup2/pulls). I already made changes on top of my changes which prevents me currently of creating more PRs but I sort that out later. Could you show a screeshot? Any last log entries?

I packed all things to an experimental/unofficial plg, which is simply an update. I dont know if its the best option to publish it here - so: If anyone wants to test ALL my changes, please PM me. So far: Fixed tar error detection during backup Improve backuplog format Option to create separate backup archives Option to hide Docker warning message box Check docker start result and wait 2 seconds between each start/stop operation (because of the: "failed to set IPv6 gateway" issue - which still needs some interperation of a docker guru)

I added a 2 second wait between docker starts and a check, if it was successful. This night went without any issues. I think, I will leave it this way. Back to separate archives: The restore is working as well now.

Got a bit further (regarding: some containers sometimes doesnt start). This time, it happened to mysql: [09.11.2022 17:46:38] Starting mysql [09.11.2022 17:46:39] Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: time="2022-11-09T17:46:39+01:00" level=fatal msg="failed to set IPv6 gateway while updating gateway: file exists": unknown Error: failed to start containers: mysql (the important error seems to be: "failed to set IPv6 gateway while updating gateway: file exists") And I have no clue what that mean. Maybe a 1 sec wait between docker start's do the trick? This issue only occurs occasionally! If anyone, affected by this, want to get some more output while ca.backup is starting containers (maybe you get same/other errors?): Open the file: /usr/local/emhttp/plugins/ca.backup2/scripts/backup.php Scroll to line 246 Replace "shell_exec("docker start ".$docker);" with: backupLog("Starting docker container info: ".shell_exec("docker start ".$docker." 2>&1")); You should now see a line starting with "Starting docker container info: " for each starting container, inside the log in the WebUI. And maybe some more output, if it silently fails. I will add a check, if the docker start returned any error and let the user know. One more improvement

My containers do not havy any requirements. npm could start without any containers. Thats the point. The order is irrelevant but sometimes it just dotn work. I'll capture whats going on this night and let all know.

Small update: Had this issue tonight with NPM: It was stopped successfully but not started again although it was correctly started via ca.backup. I add some more checks to ca.backup and test if I can get some more context when it happens. In the meanwhile, I wonder why Andrew does not respond to any of my questions/issues. I guess its due to lack of time? If so, at least the tar exit code check thing would be important since this issue breaks reliability.

I did not tracked down the "issue" that docker containers sometimes dont start but I worked on other features: This is now working: [08.11.2022 18:21:02] Backup of appData starting. This may take awhile [08.11.2022 18:21:02] Stopping deconz... done (took 1 seconds) [08.11.2022 18:21:03] Stopping jdownloader... done (took 0 seconds) [08.11.2022 18:21:03] Backing Up appData from /mnt/cache/appdata/ to /mnt/disks/92E6P00LT/appdata/[email protected] [08.11.2022 18:21:03] Separate archives enabled! [08.11.2022 18:21:03] Backing up: deconz (cd '/mnt/cache/appdata/deconz' && /usr/bin/tar -caf '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_deconz.tar' . >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/backupInProgress && wait $!) [08.11.2022 18:21:03] Verifying Backup deconz - cd '/mnt/cache/appdata/deconz' && /usr/bin/tar --diff -C '/mnt/cache/appdata/deconz' -af '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_deconz.tar' >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/verifyInProgress && wait $! [08.11.2022 18:21:04] Backing up: discourse (cd '/mnt/cache/appdata/discourse' && /usr/bin/tar -caf '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_discourse.tar' . >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/backupInProgress && wait $!) [08.11.2022 18:21:04] Verifying Backup discourse - cd '/mnt/cache/appdata/discourse' && /usr/bin/tar --diff -C '/mnt/cache/appdata/discourse' -af '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_discourse.tar' >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/verifyInProgress && wait $! [08.11.2022 18:21:04] Backing up: discourse_data (cd '/mnt/cache/appdata/discourse_data' && /usr/bin/tar -caf '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_discourse_data.tar' . >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/backupInProgress && wait $!) [08.11.2022 18:21:04] Verifying Backup discourse_data - cd '/mnt/cache/appdata/discourse_data' && /usr/bin/tar --diff -C '/mnt/cache/appdata/discourse_data' -af '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_discourse_data.tar' >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/verifyInProgress && wait $! [08.11.2022 18:21:05] Backing up: jdownloader (cd '/mnt/cache/appdata/jdownloader' && /usr/bin/tar -caf '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_jdownloader.tar' . >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/backupInProgress && wait $!) [08.11.2022 18:21:05] Verifying Backup jdownloader - cd '/mnt/cache/appdata/jdownloader' && /usr/bin/tar --diff -C '/mnt/cache/appdata/jdownloader' -af '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_jdownloader.tar' >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/verifyInProgress && wait $! [08.11.2022 18:21:05] Backing up: mysql (cd '/mnt/cache/appdata/mysql' && /usr/bin/tar -caf '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_mysql.tar' . >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/backupInProgress && wait $!) [08.11.2022 18:21:05] Verifying Backup mysql - cd '/mnt/cache/appdata/mysql' && /usr/bin/tar --diff -C '/mnt/cache/appdata/mysql' -af '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_mysql.tar' >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/verifyInProgress && wait $! [08.11.2022 18:21:06] Backing up: npm (cd '/mnt/cache/appdata/npm' && /usr/bin/tar -caf '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_npm.tar' . >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/backupInProgress && wait $!) [08.11.2022 18:21:06] Verifying Backup npm - cd '/mnt/cache/appdata/npm' && /usr/bin/tar --diff -C '/mnt/cache/appdata/npm' -af '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_npm.tar' >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/verifyInProgress && wait $! [08.11.2022 18:21:06] Backing up: urbackup (cd '/mnt/cache/appdata/urbackup' && /usr/bin/tar -caf '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_urbackup.tar' . >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/backupInProgress && wait $!) [08.11.2022 18:21:12] Verifying Backup urbackup - cd '/mnt/cache/appdata/urbackup' && /usr/bin/tar --diff -C '/mnt/cache/appdata/urbackup' -af '/mnt/disks/92E6P00LT/appdata/[email protected]/CA_backup_urbackup.tar' >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/verifyInProgress && wait $! [08.11.2022 18:21:15] done [08.11.2022 18:21:15] Starting deconz [08.11.2022 18:21:16] Starting jdownloader [08.11.2022 18:21:17] Backup/Restore Complete. tar Return Value: 0 [08.11.2022 18:21:17] Backup / Restore Completed and turns out into: I still need to adapt restore. I also added an option to disable the docker warning (https://github.com/Squidly271/ca.backup2/issues/9).

Same issue, but not 6.11.2 related, as Iam on 6.11.1. Sometimes two containers are not started, sometimes only 1. If I have more time, I check, whats exactly is going on anbd report back

@Squid Anything new on this?

Hi, I converted my vdi to qcow2 and want to ask, if its possible to start the VM with same internal UUIDs or wahtever to keep windows license etc. Is that possible? I did not found a solution. If there is one, it would be great, if the wiki gets updated Thanks in advance!

Hello all @SquidI created some PRs (https://github.com/Squidly271/ca.backup2/pulls) and sent you a dm (https://forums.unraid.net/messenger/47467/?tab=comments#comment-89845) regarding those, but got no answer, yet. Could you check those? Thanks

Basically you remove UMASK, PUID and PGID variables and change the the variable for the backup to point to "/backups" (container path) and the appdata to "/var/urbackup" (container path). Change "Repository" to: uroni/urbackup-server:latest Change "Docker Hub URL" to: https://hub.docker.com/r/uroni/urbackup-server If you want you can use the Icon @ https://forums.urbackup.org/uploads/default/original/2X/1/1051fd74d1dcbc3ad4220b43007fcab5287272b0.png Add a new variable, named "TZ" and enter your right Timezone (e.g. "Europe/Berlin"). The template for the docker container by uroni: <?xml version="1.0"?> <Container version="2"> <Name>UrBackup</Name> <Repository>uroni/urbackup-server:latest</Repository> <Registry>https://hub.docker.com/r/uroni/urbackup-server</Registry> <Network>host</Network> <MyIP/> <Shell>bash</Shell> <Privileged>false</Privileged> <Support/> <Project/> <Overview/> <Category/> <WebUI>http://[IP]:[PORT:55414]/</WebUI> <TemplateURL/> <Icon>https://forums.urbackup.org/uploads/default/original/2X/1/1051fd74d1dcbc3ad4220b43007fcab5287272b0.png</Icon> <ExtraParams/> <PostArgs/> <CPUset/> <DateInstalled>1662522898</DateInstalled> <DonateText/> <DonateLink/> <Requires/> <Config Name="TimeZone" Target="TZ" Default="" Mode="" Description="" Type="Variable" Display="always" Required="false" Mask="false">Europe/Berlin</Config> <Config Name="Backup Folder" Target="/backups" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/user/BackupsV2</Config> <Config Name="Database Folder" Target="/var/urbackup" Default="" Mode="rw" Description="" Type="Path" Display="always" Required="false" Mask="false">/mnt/cache/appdata/urbackup</Config> </Container>

I just switched to the official urBackup container. It needs some paths adaptations but it works. Since they offer an official one, I dont see any need fir the binhex anymore. => https://hub.docker.com/r/uroni/urbackup-server