[Support] Bacula-Server

[VRx]

On 3/12/2023 at 4:34 PM, murkus said:

I believe it would be safe to integrate vchanger with the latest container now.

I'll try to merge it before the weekend (when jenkins is building new versions)

[VRx]

@murkus You can try to switch Your's installation to latest image with vchanger build in.

Could You please write some instruction how You configure it? Or maybe by PM.

[murkus]

On 3/28/2023 at 11:57 PM, VRx said:

@murkus You can try to switch Your's installation to latest image with vchanger build in.

Could You please write some instruction how You configure it? Or maybe by PM.

I have noticed and I am running the latets container now, and vchanger seems to work. Let me run it for some more days, and I will give you a notice whether it is running solidly.

 

As for instructions: the vChanger documentation is pretty good and it explains well what to do, and there is a mailing list where the author helps out. I made a short tutorial how to configure it with USB drives:

 

## Docker

- map USB mass storage folders to the following pathes in the bacula container:
    - `/mnt/usb0`
    - `/mnt/usb1`
    - `/mnt/usb2`
    - etc.

## vChanger

```
#/opt/bacula/etc/vchanger1.conf
#
bconsole = "/usr/bin/bconsole"
bconsole config = "/opt/bacula/etc/bconsole.conf"
User = "bacula"
Group = "tape"
#
Storage Resource = "usb-storage"
Log level = 7
Logfile = "/opt/bacula/log/vchanger/vchanger1.log"
Work Dir = "/opt/bacula/working/vchanger/vchanger1"
Default Pool = "Scratch"
Magazine = "/mnt/usb0"
Magazine = "/mnt/usb1"
Magazine = "/mnt/usb2"
#
#eof
```

## Dir

```
Storage {
  Name = "usb-storage"
  Description = "USB storage"
  Device = "vchanger1"
  MediaType = "usb-file"
  Autochanger = "usb-storage"
}
```

## SD

```
Autochanger {
  Name = "vchanger1"
  Description = "vChanger for USB mass storage devices"
  Device = "vchgr1-mag0"
  Device = "vchgr1-mag1"
  Device = "vchgr1-mag2"
  ChangerDevice = "/opt/bacula/etc/vchanger1.conf"
  ChangerCommand = "/usr/local/bin/vchanger %c %o %S %a %d"
}
Device {
  Name = "vchgr1-mag0"
  Description = "USB device FS 1"
  MediaType = "usb-file"
  DeviceType = "File"
  ArchiveDevice = "/opt/bacula/working/vchanger/vchanger1/0"
  RemovableMedia = no
  RandomAccess = yes
  LabelMedia = yes
  Autochanger = yes
  DriveIndex = 0
}
Device {
  Name = "vchgr1-mag1"
  Description = "USB device FS 2"
  MediaType = "usb-file"
  DeviceType = "File"
  ArchiveDevice = "/opt/bacula/working/vchanger/vchanger1/1"
  RemovableMedia = no
  RandomAccess = yes
  LabelMedia = yes
  Autochanger = yes
  DriveIndex = 1
}
Device {
  Name = "vchgr1-mag2"
  Description = "USB device FS 3"
  MediaType = "usb-file"
  DeviceType = "File"
  ArchiveDevice = "/opt/bacula/working/vchanger/vchanger1/2"
  RemovableMedia = no
  RandomAccess = yes
  LabelMedia = yes
  Autochanger = yes
  DriveIndex = 2
}
```

# Operation

- vChanger CANNOT create volumes automatically, you MUST create them manually
- compute how many volumes may fit on a USB filesystem depending on the maximum volume size defined -> `<num-vols>`
- create an according number of volumes on the USB filesystem (e.g. `/mnt/usb0` is disk 0, -> `<disk> is 0`), run the `vchanger` command on the console of the container
    - `vchanger -u bacula -g tape /opt/bacula/etc/vchanger1.conf createvols <disk> <num-vols>`
- if the currently used disk is not full, create more new empty volumes
- if the currently used disk is full, create volumes on the next empty disk

[murkus]

vchanger in latest is working fine. Thanks for the effort.

 

[murkus]

today I had the occurrence that the director process died. no information on that in the bacula.log or system log. so I restarted the container. naturally a lot of the processes that were waiting were displayed to be in error.

 

So I clicked the restart button on them in the history list in relatively quick succession. The director died again. I can reproduce this, if I restart - say 5 or more - jobs from the history without waiting for each restart button spinner to finish, the director will die and some bconsole processes will complain.

 

If I wait for the spinner to finish before I restart the next job, the director will stay alive.

 

I would be interested whether anybody else can reproduce this behavior.

 

[bubbadk]

did all from post 1.

using postgresql 15

 

[VRx]

On 4/17/2023 at 4:29 PM, murkus said:

I would be interested whether anybody else can reproduce this behavior.

 

I will try to recreate this behavior and try to find the cause

 

 

On 5/4/2023 at 10:01 AM, bubbadk said:

did all from post 1.

using postgresql 15

 

There is a clear info, You have some configuration in bacula config files about database, but database looks epmpty (no tables).

Have you used this image before?

[bubbadk]

4 hours ago, VRx said:

I will try to recreate this behavior and try to find the cause

 

 

There is a clear info, You have some configuration in bacula config files about database, but database looks epmpty (no tables).

Have you used this image before?

nope. i just made it at the time. image..do you mean bacula

[VRx]

Yes I mean bacula.

[murkus]

@VRx

Has vchanger been put into the "normal" production images or is it still only part of the beta images?

[VRx]

@murkus

 

On 4/1/2023 at 12:40 PM, murkus said:

I have noticed and I am running the latets container now, and vchanger seems to work

 

[dharcus]

I keep getting this error.

 

Error code: 1000

Message: Internal error. Invalid table/view name 'job', or that table/view 'job' contains no accessible column/field definitions.

 

I have checked permissions and followed the guide to no avail, I have even reinstalled with the same outcome.

[VRx]

This is Your first time with this image or maybe You are using Bacula some time?

[murkus]

@VRx What is your strategy regarding providing images for available major versions? Are you waiting for the second next major version to appear before you work on the next major version? You currently work with 11. 13 is available some time and 15 betas are being relased. Will you look into 13 when 15 has been relased (as non-beta)?

[VRx]

My strategy is to find free time.
I work full-time for my employer who pays me, and this is a non-profit project in my free time.
There is a link to github in the first post, you can fork it and try to work on the new version.

[murkus]

If someone wants to use a more recent version of Postgres than 13, Baculum 11 (as provided in this container) will throw an error. Baculum 13 contains a fix:

 

https://gitlab.bacula.org/bacula-community-edition/bacula-community/-/commit/e1389d3caf89875c0009930237ba59a1133f6cd6

 

This fix also works with Baculum 11. You may manually edit the file in the container.

 

More ideal would be if the fix could be incorporated by @VRx for updates of the container image.

 

Just my 2 cents.

 

[VRx]

Version 13 available.
Two container configuration changes required:
1. Bacula WEB Port: container port 9095 should be changed to 9097

2. Baculum config: container path should be updated /etc/baculum -> /etc/bacularis
(If You create any additional users for Baculum WEB UI, after update You need to create them again)

3*. If You prefer to change both container and mapped port 9095:9095 to 9097:9097 WebUI: http://[IP]:[PORT:9095]/ at the Advanced View should be changed to http://[IP]:[PORT:9097]/

Read Update at the first post.

[murkus]

Good work, got it up and running.

Thanks for the effort!

 

[murkus]

@VRx

is there a chance that the pg_dump binary gets upgraded to 16.1 in the bacula 13 postgres containers?

 

currently the catalog backup fails for everyone using Postgres version higher than 14 with bacula 13.

 

07-Jan 23:45 bacula-dir JobId 18092: BeforeJob: pg_dump: error: server version: 16.1 (Debian 16.1-1.pgdg120+1); pg_dump version: 14.10 (Debian 14.10-1.pgdg110+1) 07-Jan 23:45 bacula-dir JobId 18092: BeforeJob: pg_dump: error: aborting because of server version mismatch 07-Jan 23:45 bacula-dir JobId 18092: Error: Runscript: BeforeJob returned non-zero status=1. ERR=Child exited with code 1

Edited January 7 by murkus

[VRx]

Application: Bacula - https://www.bacula.org/
                  Baculum App - https://www.baculum.app/ / Bacularis - https://bacularis.app/
Docker Hub: https://hub.docker.com/r/pwa666/bacula-server
GitHub: https://github.com/vrx-666/bacula-server

Bacula is a client / server backup application.
Baculum/Bacularis is a web application that allows you to easily manage the bacula application.
This container combines these applications to form a backup tool with a graphical interface.
This container can be run in 5 different versions.

Bacula server consists of several components:

• storage daemon - responsible for saving backups to disk or removable media (tape)
• bacula director - responsible for the entire backup process, contacts agents on client devices, indicates what resources should be backed up, how and where to send them (storage daemon)
• file daemon - is a bacula client available for many operating systems, responds to requests from bacula director.

A relational database is necessary, where the bacula stores all information related to the performed backups.
During installation, you can choose Bacula version 9.6 or 11.
Version 9.6 allows you to use the mysql or postgresql database as well as sqlite3.
Version 11 allows you to use the postgresql or sqlite3 database.
The sqlite3-enabled version is not recommended for larger environments, but allows you to run this application without running mysql / postgresql as a separate container or server. If you already have postgresql or mysql running as your VM or docker application, I recommend choosing this version of the application.

If you plan to use the PostgreSQL or MySQL version you should first create an empty database named "bacula", create a password user for that database and grant full privileges to this database for the created user.For mySQL:

CREATE USER 'USERNAME' @ '%' IDENTIFIED BY 'PASSWORD';
CREATE DATABASE bacula;
GRANT ALL PRIVILEGES ON `bacula`. * TO 'USERNAME' @ '%';
flush privileges;

For postgreSQL:

create user USERNAME with password 'PASSWORD';
create database bacula encoding 'SQL_ASCII' owner "USERNAME" template = template0;

When running container there is some mandatory environment variables.
Most important is to set up:

• SD_Host - should define the IP address at which the Bacula Server will be reachable for computers.
  If you are using a bridge network this parameter should be the ip of your UnRAID.
  If you set custom bridge, You shuld type in assigned IP.
  This variable is passed to the Bacula configuration and later during the backup it is passed to the clients so they can communicate with the Bacula server.

The following are necessary when using an external database (postgreSQL, mySQL)

• DB_Host - database ip
• DB_Port - database port (could be omitted if default)
• DB_User - database user
• DB_Password

In addition, there are two important ports that must be exposed:

• 9095 - Bacula Web
• 9103 - Bacula Storage

The first allows access to the webUI Baculum App.
The second is used to receive backups by the server from clients / computers.
It can be changed, but you have to change the configuration of Storage Daemon and Bacula Director e.g. via webUI.
I recommend that you leave these ports unchanged.

There are also necessary paths to folders where configuration files, logs and backups will be saved, these variables are transparent in the UnRaid's docker form, I think that they do not need to be described here.

If you have any questions, please ask below.

[ Update 2022.05.30 ]
End of development for Bacula 9.6 images versions!
Older versions of the client work well with the newest server version. The last update of the bacula9.6 code was in December 2020.

[ Update 2022.09.15 ]

Email notification added.
If You want Your's bacula could send emails, first of all You should set container hostname!!! Some of email providers could reject email, if sender will not introduce himself. Container hostname should be Your public reverse dns.

Below You have environment variables that must be set to send bacula messages:
EMAIL_Recipient - email address where You want bacula send notifications
SMTP_Host - address of Your email provider's smtp server
SMTP_Port - Your email provider smtp port (check provider documentation)
SMTP_User - Email sender login (could be: user@domain or user, check Your email provider documentation)
SMTP_Password - Password for email sender account SMTP_Port - smtp port for Your email provider smtp server (check provider documentation) - Yes this is ENV Variable not Docker Port
Additionally, it is possible to set email log path pointing to container path /var/log/exim4

 

[ Update 2022.10.06 ]

In the next image update, it will contain postgres client v14.5 (it gives possibility to use a newer version of the postgres server)

 

[ Update 2024.01.02 ]

Version 13 available.
New Web UI interface

Upgrade from version 11 requires change two params:

1. "Bacula WEB port" 9095 -> 9097 (container port)
2. "Baculum config" - path for persistent web configuration:
there was /etc/baculum inside container at v11, in version 13 there is /etc/bacularis
Be careful, configurations vary! You cannot map the same host folder for /etc/bacularis that was used for /etc/baculum.
At the same time, this change forces to create any custom users if you created any in version 11.
Unraid template will be updated soon, but it not update if You run it before.

[ Update 2024.01.08 ]

Postgresql Client v16
Bacularis/Baculum logs redirected to container (docker) logs - no need to bind path for apache (bacularis/baculum) logs

[murkus]

Thanks for including pg_dump 16!

[VRx]

There is an additional change, currently there is no need to define the "Apache log path".
Web interface logs currently go to the container (docker) logs.

[murkus]

On 1/10/2024 at 8:33 PM, VRx said:

There is an additional change, currently there is no need to define the "Apache log path".
Web interface logs currently go to the container (docker) logs.

 

Yes, I realized that. I have redirected all my docker logs on all hosts to go to a central syslog. The web UI logs are not really of interest to me in the central log repo and I would need to suppress those. You probably have a reason why you think it is better to have those in the docker logs. I personally liked it the way it was before, but that's just me.

[murkus]

I am planning to add offsite backup to my backup strategy and to use the Bacula S3 driver to implement it - actually using minIO as S3 server on a remote NAS.

Does this container already include the S3 driver for Bacula SD?

[murkus]

I configured a copy job for the catalog to be copied to my S3 minIO, but it seems this container does not contain the S3 cloud driver:
 

Fatal error: init_dev.c:505 [SF0020] dlopen of SD driver=cloud at /opt/bacula/plugins/bacula-sd-cloud-driver-13.0.3.so failed: ERR=/opt/bacula/plugins/bacula-sd-cloud-driver-13.0.3.so: cannot open shared object file: No such file or directory

 

As follows I would suggest to include this driver in the container, if this is possible. AFAIK it is supported for the Community Edition.