uberchuckie Posted September 24, 2020 Author Share Posted September 24, 2020 (edited) @chip The "aria_log_control" should be created in the /config/database volume. /etc/service/mariadb/run is the runit script to start MariaDB in the container. It should be starting MariaDB with Quote /usr/bin/mysqld_safe --skip-syslog --datadir='/config/databases' I'm not sure why it's using /var/lib/mysql in your case. EDIT: Ah, you were running it from the command line. Can you start from scratch and post up the error in the MariaDB log file? The log file should be '/config/databases/8ffb5f7097ea.err' Edited September 24, 2020 by uberchuckie Quote Link to comment
chip Posted September 24, 2020 Share Posted September 24, 2020 (edited) That log file never gets created. /config/databases# dir aria_log.00000001 ib_buffer_pool ibdata1 mysql performance_schema aria_log_control ib_logfile0 multi-master.info observium test If I run your command above - /etc/service/mariadb/run Database exists. Starting MariaDB... 200924 06:07:37 mysqld_safe Logging to '/config/databases/2e96f60615f2.err'. 200924 06:07:37 mysqld_safe Starting mariadbd daemon with databases from /config/databases service mariadb status * MariaDB is stopped. Also I am running Version: 6.8.3 of Unraid. Edited September 24, 2020 by chip Quote Link to comment
seanwalter Posted September 28, 2020 Share Posted September 28, 2020 I may have new news, but it's all a little weird... Because there was some discussion that these issues have to do with mariadb not updating within the docker container, on Friday I went ahead on a whim and ran an apt update and upgrade to all the packages from the observium:latest docker console. Several updates were pulled down -- not sure if that's normal for docker containers, as I generally don't get this far into it Following the updates, I started actually getting errors logged to my observium.err file per the reporting in the container log file. The error was: 2020-09-25 20:10:00 0 [ERROR] mysqld: File '/var/log/mysql/mariadb-bin.index' not found (Errcode: 13 "Permission denied") 2020-09-25 20:10:00 0 [ERROR] Aborting The file in question did exist. Permissions were 660. With permissions changed to 666 or 777 the error persisted, which seemed a little weird. The file in the log seems simply to reference another file in the same directory, mariadb-bin.000001, with the same 660 permissions. I also tried changing those permissions, but it didn't seem to help. I subsequently reverted back to 2.0.2. In writing this post now (~2 days later) I am seeing that in the 2.0.2 container, the files are owned by nobody:users, which is different from the owners that were showing up under the :latest container. In that container, the group was "adm" but I don't remember the user name. Next-level weirdness...I tried to repeat this entire process just now to get the user name again, but I could not reproduce the results -- after running the apt upgrades, things remained in the state where nothing was being logged to observium.err. Worse(?), the mariadb-bin files were not even being created in the container's /var/log/mysql directory. Finally, next-next-level weirdness, I decided I should create a second docker container to mess around with, using observium:latest. So basically I did a new install of the container, with a new container name, into a new appdata folder. When I had reverted back to 2.0.2 above, I had not deleted my apt upgraded :latest image file, so docker used that file to create the new container. And...this one works. Only problem is, all my data is in the 2.0.2 container. I can try to copy my 2.0.2 databases over, but I'd love any recommendations on whether this is a reasonable thing to do, and/or what exactly needs to be copied, and/or how to force a database conversion if one is necessary. With both containers now run-able, I am also happy to run other tests if helpful. Side note, my unRAID Docker page reports that the version is "not available" when I pull the container from uberchuckie/observium or uberchuckie/observium:latest. This is not an issue with uberchuckie/observium:2.0.2. I am slightly nervous there's an updated source on dockerhub and I missed the memo... Quote Link to comment
uberchuckie Posted September 29, 2020 Author Share Posted September 29, 2020 @seanwalter MariaDB shouldn't be using /var/log/mysql for its log files or /var/lib/mysql for its data files. It's in the container's ephemeral disk and does not persist between container instances. It's supposed to use the /config/ volume mount. MariaDB is started with /usr/bin/mysqld_safe --skip-syslog --datadir='/config/databases' from the runit service, MariaDB should be using the volume mount for its data. It's weird that a brand new container instance works for you but not @chip. This has got to be something related to file permission issues or something along the line. It's hard for me to debug this without access to a broken system. Quote Link to comment
uberchuckie Posted September 29, 2020 Author Share Posted September 29, 2020 Another thing to look into is what Squid had mentioned. Do you have a cache drive? If so, is the volume mount on /mnt/user/appdata or /mnt/cache/appdata? Quote Link to comment
uberchuckie Posted September 29, 2020 Author Share Posted September 29, 2020 (edited) @seanwalter I've just pushed the tag for 3.0_beta. This includes Observium CE 20.9.10731, MariaDB 10.5.5 and the latest apt packages. It shouldn't change how the container works on your system but you can always give it a try. It works fine for me both creating a database from scratch and updating from the 19.8.1 schema. Edited September 29, 2020 by uberchuckie Quote Link to comment
chip Posted September 29, 2020 Share Posted September 29, 2020 (edited) I have my settings to /mnt/cache/appdata/observium for all 3 in the container. So with the build as is it can't write to the /config/databases folder so the .err file never shows up. So reading a page back here I ran - chown -R nobody:users /config/databases chmod -R 755 /config/databases Then Reran - /usr/bin/mysqld_safe --skip-syslog --datadir='/config/databases' Now I have a .err file in the databases folder. Here is some of that error. 2020-09-29 10:32:04 0 [Note] InnoDB: Using Linux native AIO 2020-09-29 10:32:04 0 [Note] InnoDB: Uses event mutexes 2020-09-29 10:32:04 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 2020-09-29 10:32:04 0 [Note] InnoDB: Number of pools: 1 2020-09-29 10:32:04 0 [Note] InnoDB: Using SSE4.2 crc32 instructions 2020-09-29 10:32:04 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts) 2020-09-29 10:32:04 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728 2020-09-29 10:32:04 0 [Note] InnoDB: Completed initialization of buffer pool 2020-09-29 10:32:04 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2020-09-29 10:32:04 0 [Note] InnoDB: 128 rollback segments are active. 2020-09-29 10:32:04 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2020-09-29 10:32:04 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2020-09-29 10:32:04 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2020-09-29 10:32:04 0 [Note] InnoDB: 10.5.4 started; log sequence number 37755; transaction id 9 2020-09-29 10:32:04 0 [Note] InnoDB: Loading buffer pool(s) from /config/databases/ib_buffer_pool 2020-09-29 10:32:04 0 [Note] Plugin 'FEEDBACK' is disabled. 2020-09-29 10:32:04 0 [Note] InnoDB: Buffer pool(s) load completed at 200929 10:32:04 2020-09-29 10:32:04 0 [Note] Zerofilling moved table: './mysql/plugin' 2020-09-29 10:32:04 0 [Warning] Checking table: './mysql/plugin' 2020-09-29 10:32:04 0 [ERROR] mysql.plugin: Table is from another system and must be zerofilled or repaired to be usable on this system 2020-09-29 10:32:04 0 [Warning] Recovering table: './mysql/plugin' 2020-09-29 10:32:04 0 [ERROR] mysql.plugin: 9 when fixing table 2020-09-29 10:32:04 0 [ERROR] Couldn't repair table: mysql.plugin 2020-09-29 10:32:04 0 [ERROR] Could not open mysql.plugin table: "Index for table 'plugin' is corrupt; try to repair it". Some plugins may be not loaded 2020-09-29 10:32:04 0 [Note] Zerofilling moved table: './mysql/servers' 2020-09-29 10:32:04 0 [Warning] Checking table: './mysql/servers' 2020-09-29 10:32:04 0 [ERROR] mysql.servers: Table is from another system and must be zerofilled or repaired to be usable on this system Any Ideas? Edited September 29, 2020 by chip Quote Link to comment
uberchuckie Posted September 30, 2020 Author Share Posted September 30, 2020 (edited) @chip Can you delete everything and try the 3.0_beta image? I'm not sure why the table would be corrupted right after being created. Edit the container definition and change the Repository from uberchuckie/observium to uberchuckie/observium:3.0_beta. The image contains Observium 20.9, MariaDB 10.5.5 and a change to the run script to always update the file permissions. Edited September 30, 2020 by uberchuckie Quote Link to comment
chip Posted October 1, 2020 Share Posted October 1, 2020 (edited) Remove the Docker and the folder. Added it and then changed it to uberchuckie/observium:3.0_beta. I also change the other two settings to /mnt/cache/appdata Thought it was good for a moment as it loaded the log in page. But again DB Error 2002: No such file or directory. Nope looks to be in the same messed up state on my end. Still won't write the log err file to config/databases either so still must be a rights issue again as well. This is odd as all my other dockers work fine so I believe I don't have an issue with my cache drive that I use for dockers. Edited October 1, 2020 by chip Quote Link to comment
chip Posted October 1, 2020 Share Posted October 1, 2020 I also ran the following commands after it was still broken - chown -R nobody:users /config/databases chmod -R 755 /config/databases No go/changes and log file still doesn't write to /config/databases So I did the following chmod -R 777 /config/databases Noticed the err log file showed up in /config/databases Skipped looking at the err file and just ran /usr/bin/mysqld_safe --skip-syslog --datadir='/config/databases' and it is working. I am far from a linux guy just know the basics and know the above commands are about rights/etc but something still seems to be broken in my mind. Again not a linux expert but something isn't working properly. Quote Link to comment
uberchuckie Posted October 1, 2020 Author Share Posted October 1, 2020 OK, I think I know what's going on. When MariaDB first starts with the script to bootstrap itself, it's using the mysql:mysql user/group (104:106). The script changes the ownership to nobody:users (99:100) and causes the file permission issues. This has always been the behaviour and I don't know why this doesn't fail for others but fail for some. It makes sense that changing the permissions to 777 works for you. My host doesn't have user 104 and group 106 defined. Maybe this is why it's working. I'll need to think about how to fix this without breaking it for others. Quote Link to comment
uberchuckie Posted October 1, 2020 Author Share Posted October 1, 2020 The problem is related to the MariaDB 10.4 -> 10.5 upgrade. The some of the configuration that is to be rewritten in the my.cnf has moved. I'll look into fixing it when I have time. Quote Link to comment
uberchuckie Posted October 1, 2020 Author Share Posted October 1, 2020 (edited) @chip @seanwalter I think I've fixed it. I have updated the 3.0_beta tag with the fix. If everything goes well, I'll push all the updates to the latest tag. @seanwalter Please backup your data before you try it. The 3.0_beta tag contains Observium 20.9 and would perform an irreversible schema update. Thanks for your patience to help me track down this bug. The pull request for the fix is: https://github.com/charlescng/docker-containers/pull/8 Edited October 1, 2020 by uberchuckie Quote Link to comment
chip Posted October 1, 2020 Share Posted October 1, 2020 let me know when I can retry it. Thanks. Quote Link to comment
uberchuckie Posted October 1, 2020 Author Share Posted October 1, 2020 3 minutes ago, chip said: let me know when I can retry it. Thanks. @chip You can retry now with the 3.0_beta tag. Just click on "Check for Updates" to pick up the new container image. Quote Link to comment
chip Posted October 1, 2020 Share Posted October 1, 2020 (edited) Thanks man. To be safe I removed the docker, deleted the cache folder and started from scratch. Seems to be OK now. Edited October 1, 2020 by chip Quote Link to comment
uberchuckie Posted October 1, 2020 Author Share Posted October 1, 2020 52 minutes ago, chip said: To be safe I removed the docker, deleted the cache folder and started from scratch. Seems to be OK now. Good to hear! Thanks for helping me diagnosing this problem. Quote Link to comment
uberchuckie Posted October 1, 2020 Author Share Posted October 1, 2020 The new version 3.0 image is working for me both upgrading from the previous image and from scratch. I've updated the latest tag. Please switch back to the latest tag if you've tested the other tags. Thanks. Quote Link to comment
Fiala06 Posted October 1, 2020 Share Posted October 1, 2020 (edited) Just reinstalled and its working again. Glad to know it wasn't just me lol Edit: Anyone got this to work with Nginx Proxy Manager? Edited October 1, 2020 by Fiala06 Nginx Quote Link to comment
seanwalter Posted October 3, 2020 Share Posted October 3, 2020 @uberchuckie good news...updating my 2.0.2 container to 3.0_beta worked -- my data history is intact, I can connect via the web GUI using my user name and password, etc... It looks like others are having success as well, though to close the loop, I am using /mnt/cache/appdata for my observium config and other files. Thank you for sticking with us on this, and working through the issues! Quote Link to comment
uberchuckie Posted October 3, 2020 Author Share Posted October 3, 2020 16 hours ago, seanwalter said: @uberchuckie good news...updating my 2.0.2 container to 3.0_beta worked -- my data history is intact, I can connect via the web GUI using my user name and password, etc... It looks like others are having success as well, though to close the loop, I am using /mnt/cache/appdata for my observium config and other files. Thank you for sticking with us on this, and working through the issues! @seanwalter Thanks for helping to debug. You can switch back to the latest tag as all the fixes have been applied to it. Quote Link to comment
Silvan-0172 Posted January 26, 2021 Share Posted January 26, 2021 Hi uberchuckie, thank you for your time and effort in maintaining this container. I was wondering if the issue with the timezone did get fixed ? Yesterday i installed the container again and set the TZ in the properties but its still showing in UTC. I searched the thread but didnt find a anwser. Kind regards. Quote Link to comment
uberchuckie Posted February 7, 2021 Author Share Posted February 7, 2021 On 1/26/2021 at 8:23 AM, Silvan-0172 said: Hi uberchuckie, thank you for your time and effort in maintaining this container. I was wondering if the issue with the timezone did get fixed ? Yesterday i installed the container again and set the TZ in the properties but its still showing in UTC. I searched the thread but didnt find a anwser. Kind regards. Yeah, I don't think anyone has figured it out yet. Quote Link to comment
uberchuckie Posted February 16, 2021 Author Share Posted February 16, 2021 I just checked again, all the timezones are set correctly in my container. I'm still seeing graphs in UTC. Quote Link to comment
uberchuckie Posted February 16, 2021 Author Share Posted February 16, 2021 (edited) I created a test image yesterday based on phusion's alpha Ubuntu 20.04 based baseimage. It seems to work from my short testing. The biggest change is upgrading Python to 3.8 from 2.7, the latter of which has reached End-Of-Life awhile back. I've tagged the image as focal-alpha if people want to try it out. Observium and MariaDB doesn't change in this image so there shouldn't be any differences other than upgraded libraries. Again, this is just for testing or if you were curious, I wouldn't upgrade right now if you depend on it working correctly. Edited February 16, 2021 by uberchuckie Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.