repomanz Posted January 4, 2021 Share Posted January 4, 2021 This is my favorite unraid docker container and I don't mind rebuilding it but would like to prevent others from likely corruption as it's not obvious. Is there anything in your build process with the new template to prevent people from stumbling into this issue? Meaning they need to set the previous shm value into the new template before the upgrade will even start? Quote Link to comment
dlandon Posted January 4, 2021 Author Share Posted January 4, 2021 1 minute ago, repomanz said: Meaning they need to set the previous shm value into the new template before the upgrade will even start? That would be best. Quote Link to comment
repomanz Posted January 4, 2021 Share Posted January 4, 2021 1 minute ago, dlandon said: That would be best. Through unraid ui how does that happen given the template change? I saw the update available in the Unraid UI and updated the container. That's how easy it is to stumble into this issue (assuming improper shm size in template led to db corruption). Quote Link to comment
hbrob Posted January 4, 2021 Share Posted January 4, 2021 1 hour ago, dlandon said: Those of you with database issues, it may be due to you restarting the docker and updating Zoneminder in the process. Zoneminder will get updated whenever the docker is restarted if there is a new version of Zoneminder. Part of the update to Zoeminder is an update to the database. If that failed because of the issues with shared memory, the database may have gotten corrupted. Yes, this is what I figured happened to me too. I got a clean install up and running, but notifications are broken. Eager to hear if others are successful or not getting these going. Quote Link to comment
dlandon Posted January 4, 2021 Author Share Posted January 4, 2021 Just now, hbrob said: Yes, this is what I figured happened to me too. I got a clean install up and running, but notifications are broken. Eager to hear if others are successful or not getting these going. ES has been updated to 6.1.5. You probably need to make some changes for the newer version. Quote Link to comment
hbrob Posted January 4, 2021 Share Posted January 4, 2021 15 minutes ago, dlandon said: ES has been updated to 6.1.5. You probably need to make some changes for the newer version. Thanks. Project for another night. Appreciate the work putting this together. Quote Link to comment
dlandon Posted January 4, 2021 Author Share Posted January 4, 2021 2 minutes ago, hbrob said: Thanks. Project for another night. Appreciate the work putting this together. You don't have a backup to recover? Quote Link to comment
hbrob Posted January 4, 2021 Share Posted January 4, 2021 (edited) 1 hour ago, dlandon said: You don't have a backup to recover? I do, I tried restoring my settings to the new install and removing the DB which was corrupted (this was one of my first actions). ES hooks did not work. I may try this again tomorrow. Edited January 4, 2021 by hbrob Quote Link to comment
joace Posted January 4, 2021 Share Posted January 4, 2021 Just found ES updated after restarted container, and notification stopped working. Realized some changes made in the new ES to use ml_sequence, so updated the objectconfig.ini according to the doc, but still couldn't get it working, from the log console, below is the message I got, no idea where it was generated. And even reverted back to the legacy detection method (use_sequence = no), still had the same error, anybody seeing the same issue? 01/03/21 22:12:49 zmesdetect_m4[3648] FAT zm_detect.py:534 [Unrecoverable error:'ml_sequence' Traceback:Traceback (most recent call last): File "/var/lib/zmeventnotification/bin/zm_detect.py", line 531, in <module> main_handler() File "/var/lib/zmeventnotification/bin/zm_detect.py", line 295, in main_handler if g.config['ml_sequence'] and g.config['use_sequence'] == 'yes': KeyError: 'ml_sequence' ] Quote Link to comment
nxtiak Posted January 4, 2021 Share Posted January 4, 2021 Okay, so default shm size is set to 5G. I have 16GB ram in my unRAID server, and Zoneminder /dev/shm shows 15%. If I set shm size to 8G, dev/shm shows 10%. What should I do? I'm a total noob and I just started using Zoneminder a few days ago. Quote Link to comment
sillywalks Posted January 4, 2021 Share Posted January 4, 2021 (edited) Also pressed on "update" last night thinking...what could happen. This morning nothing was running Once again a lesson for me to check changes before updating... I also have issues with the included mysql database not working. See attached logs at the bottom for further datapoints. Also does opencv need to be compiled every restart of the container? I have a pretty poor cpu and no gpu and the compile takes ages. I fear having to tweak the new mem settings when each restart with a config change takes 40+ minutes... Compiling opencv - this will take a while... Jan 4 07:57:14 17b93c0e5743 EventServer: Compiling opencv without GPU Support Jan 4 07:57:35 17b93c0e5743 EventServer: Compiling opencv... Jan 4 08:42:17 17b93c0e5743 EventServer: Installing opencv... Jan 4 08:42:17 17b93c0e5743 EventServer: Installing opencv... Jan 4 08:42:21 17b93c0e5743 EventServer: Cleaning up... Error Logs: Jan 4 08:42:21 17b93c0e5743 EventServer: Cleaning up... Starting services... * Starting Apache httpd web server apache2 Jan 4 08:42:23 17b93c0e5743 web_php[19213]: FAT [Failed db connection to ] Jan 4 08:42:23 17b93c0e5743 web_php[19213]: ERR [Unable to authenticate user. error decoding JWT token:Signature verification failed] Jan 4 08:42:23 17b93c0e5743 web_php[19213]: ERR [Unable to authenticate user. error decoding JWT token:Signature verification failed] Jan 4 08:42:23 17b93c0e5743 web_php[19214]: FAT [Failed db connection to ] Jan 4 08:42:23 17b93c0e5743 web_php[19215]: FAT [Failed db connection to ] Jan 4 08:42:23 17b93c0e5743 web_php[19216]: FAT [Failed db connection to ] Jan 4 08:42:23 17b93c0e5743 web_php[19217]: FAT [Failed db connection to ] Jan 4 08:42:23 17b93c0e5743 web_php[19217]: ERR [Unable to authenticate user. error decoding JWT token:Signature verification failed] Jan 4 08:42:23 17b93c0e5743 web_php[19217]: ERR [Unable to authenticate user. error decoding JWT token:Signature verification failed] Jan 4 08:42:23 17b93c0e5743 web_php[19213]: FAT [Failed db connection to ] Jan 4 08:42:23 17b93c0e5743 web_php[19214]: FAT [Failed db connection to ] Jan 4 08:42:23 17b93c0e5743 web_php[19215]: FAT [Failed db connection to ] * * Starting MariaDB database server mysqld Jan 4 08:42:40 17b93c0e5743 web_php[19216]: FAT [Failed db connection to ] Jan 4 08:42:40 17b93c0e5743 web_php[19216]: ERR [Unable to authenticate user. error decoding JWT token:Signature verification failed] Jan 4 08:42:40 17b93c0e5743 web_php[19216]: ERR [Unable to authenticate user. error decoding JWT token:Signature verification failed] Jan 4 08:42:40 17b93c0e5743 web_php[19217]: FAT [Failed db connection to ] Jan 4 08:42:40 17b93c0e5743 web_php[19213]: FAT [Failed db connection to ] Jan 4 08:42:40 17b93c0e5743 web_php[19214]: FAT [Failed db connection to ] # # This happens quite a few times # Redacted for better readability Jan 4 08:42:53 17b93c0e5743 web_php[19215]: FAT [Failed db connection to ] Jan 4 08:42:53 17b93c0e5743 web_php[19216]: FAT [Failed db connection to ] ...fail! DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. Jan 4 08:42:54 17b93c0e5743 zmupdate[19654]: ERR [Error reconnecting to db: errstr:Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) error val:] DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. Can't call method "prepare_cached" on an undefined value at /usr/share/perl5/ZoneMinder/Config.pm line 96. BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder/Config.pm line 147. Compilation failed in require at /usr/bin/zmupdate.pl line 73. BEGIN failed--compilation aborted at /usr/bin/zmupdate.pl line 73. Jan 4 08:42:54 17b93c0e5743 zmupdate[19654]: ERR [Error reconnecting to db: errstr:Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) error val:] DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. Jan 4 08:42:54 17b93c0e5743 zmupdate[19655]: ERR [Error reconnecting to db: errstr:Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) error val:] DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. Can't call method "prepare_cached" on an undefined value at /usr/share/perl5/ZoneMinder/Config.pm line 96. BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder/Config.pm line 147. Compilation failed in require at /usr/bin/zmupdate.pl line 73. BEGIN failed--compilation aborted at /usr/bin/zmupdate.pl line 73. Jan 4 08:42:54 17b93c0e5743 zmupdate[19655]: ERR [Error reconnecting to db: errstr:Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) error val:] Starting ZoneMinder: DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. Jan 4 08:42:54 17b93c0e5743 zmpkg[19665]: ERR [Error reconnecting to db: errstr:Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) error val:] DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. Can't call method "prepare_cached" on an undefined value at /usr/share/perl5/ZoneMinder/Config.pm line 96. BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder/Config.pm line 147. Compilation failed in require at /usr/share/perl5/ZoneMinder.pm line 33. BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder.pm line 33. Compilation failed in require at /usr/bin/zmpkg.pl line 34. BEGIN failed--compilation aborted at /usr/bin/zmpkg.pl line 34. Jan 4 08:42:54 17b93c0e5743 zmpkg[19665]: ERR [Error reconnecting to db: errstr:Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) error val:] ZoneMinder failed to start *** /etc/my_init.d/40_firstrun.sh failed with status 255 Edited January 4, 2021 by sillywalks Quote Link to comment
Jokerigno Posted January 4, 2021 Share Posted January 4, 2021 Hi, I had issue with db so I ended up restarting from scratch. Now I wanted to make things better so I tried to change DB from int to ext ( I have a mariadb container). So I followed this according to @dlandon : Quote Change Databases: You can change to an external database by overwriting the default zm.config database settings. Change the following settings for your particular needs and put them in the file "99-mysql.conf' in the /appdata/Zoneminder/conf/ folder. Your settings will be installed when the docker is started. I changed only ZM_DB_HOST to my db istance 192.168.2.100:3306. I don't find any "99-mysql.conf' in the /appdata/Zoneminder/conf/ folder. Something has changed? Quote Link to comment
dlandon Posted January 4, 2021 Author Share Posted January 4, 2021 7 hours ago, Jokerigno said: Hi, I had issue with db so I ended up restarting from scratch. Now I wanted to make things better so I tried to change DB from int to ext ( I have a mariadb container). So I followed this according to @dlandon : I changed only ZM_DB_HOST to my db istance 192.168.2.100:3306. I don't find any "99-mysql.conf' in the /appdata/Zoneminder/conf/ folder. Something has changed? I think this got broken over the last few changes. I'll take a look and update the docker if necessary. Quote Link to comment
dlandon Posted January 4, 2021 Author Share Posted January 4, 2021 7 hours ago, Jokerigno said: Hi, I had issue with db so I ended up restarting from scratch. Now I wanted to make things better so I tried to change DB from int to ext ( I have a mariadb container). So I followed this according to @dlandon : I changed only ZM_DB_HOST to my db istance 192.168.2.100:3306. I don't find any "99-mysql.conf' in the /appdata/Zoneminder/conf/ folder. Something has changed? Building a new docker with a fix. You create the 99-mysql.conf from the zm.default file in the appdata/conf/ folder. The file actually gets copied to /etc/zm/conf.d/ inside the container. Quote Link to comment
repomanz Posted January 4, 2021 Share Posted January 4, 2021 I'm trying to figure this one out after rebuilding the container from scratch. Anyone here know what this may be about? |----> FORK:garage-low-res (1), eid:96 Not sending event end alarm, as we did not send a start alarm for this, or start hook processing failed Quote Link to comment
dlandon Posted January 5, 2021 Author Share Posted January 5, 2021 55 minutes ago, repomanz said: I'm trying to figure this one out after rebuilding the container from scratch. Anyone here know what this may be about? |----> FORK:garage-low-res (1), eid:96 Not sending event end alarm, as we did not send a start alarm for this, or start hook processing failed That's a message from hook processing. I normally turn this off in the zmeventnotification.ini file, but it was left on and causes that message: # send event end notifications (default: no) # Note that if you are using hooks for end notifications, they may change # the final decision. This needs to be yes if you want end notifications with # or without hooks send_event_end_notification = no Be sure it's set to no. For some reason pliablepixels sets it to yes in the ES release. Quote Link to comment
repomanz Posted January 5, 2021 Share Posted January 5, 2021 39 minutes ago, dlandon said: That's a message from hook processing. I normally turn this off in the zmeventnotification.ini file, but it was left on and causes that message: # send event end notifications (default: no) # Note that if you are using hooks for end notifications, they may change # the final decision. This needs to be yes if you want end notifications with # or without hooks send_event_end_notification = no Be sure it's set to no. For some reason pliablepixels sets it to yes in the ES release. I think something else is going on. For example, my monitor picked up motion and recorded it but I don't see anything in the log ui for zmdetect. Any pointers here? Quote Link to comment
ThreeFN Posted January 5, 2021 Share Posted January 5, 2021 Just an FYI I noticed that the log file limitations that were in place have gone away from the template. Extra Parameters: --log-opt max-file=1 --log-out max-size=50m I know it probably junked up/confused the field that users would be adjusting the shared mem, but zoneminder will probably make some huge log files given the chance. Also a PSA, I recommend users add a cpu usage limit to the container, specifically when using object detect because of the opencv compile, which can lock up your server when it uses 100% CPU to compile. I'd recommend N-2 at most, where N is your cpu's threads so you leave 2 threads for all your other stuff to run on while ZM compiles. The number can be a float so fraction of threads is allowed. Extra Parameters: --cpus=X.X Quote Link to comment
repomanz Posted January 5, 2021 Share Posted January 5, 2021 3 minutes ago, ThreeFN said: Just an FYI I noticed that the log file limitations that were in place have gone away from the template. Extra Parameters: --log-opt max-file=1 --log-out max-size=50m I know it probably junked up/confused the field that users would be adjusting the shared mem, but zoneminder will probably make some huge log files given the chance. Also a PSA, I recommend users add a cpu usage limit to the container, specifically when using object detect because of the opencv compile, which can lock up your server when it uses 100% CPU to compile. I'd recommend N-2 at most, where N is your cpu's threads so you leave 2 threads for all your other stuff to run on while ZM compiles. The number can be a float so fraction of threads is allowed. Extra Parameters: --cpus=X.X I use those myself. Are you saying the switches are no longer applicable given the new template? Quote Link to comment
ThreeFN Posted January 5, 2021 Share Posted January 5, 2021 All of them are still applicable, they're docker parameters (you can have a google about them if you like). Unless I completely forgot that I added them myself, I thought that they were originally in the template but had fallen off for whatever reason. Quote Link to comment
dlandon Posted January 5, 2021 Author Share Posted January 5, 2021 Log stuff is no longer needed. Go to Docker settings, stop dockers and you'll see: Quote Link to comment
ThreeFN Posted January 5, 2021 Share Posted January 5, 2021 2 minutes ago, dlandon said: Log stuff is no longer needed. Go to Docker settings, stop dockers and you'll see: Ah, good to know, thanks for that, completely missed it. Then yes it's probably good it's gone as it frees up space a reduces confusion in the extra parameters field. Quote Link to comment
repomanz Posted January 5, 2021 Share Posted January 5, 2021 going to restore a zoneminder folder from a previous backup, this time using shm 16g. wish me luck Quote Link to comment
dlandon Posted January 5, 2021 Author Share Posted January 5, 2021 For anyone having issues with ES, this documentation may help: https://zmeventnotification.readthedocs.io/en/latest/guides/hooks.html#understanding-detection-configuration 1 Quote Link to comment
repomanz Posted January 5, 2021 Share Posted January 5, 2021 (edited) 1 hour ago, repomanz said: going to restore a zoneminder folder from a previous backup, this time using shm 16g. wish me luck quick update. To test out the shm issue corrupting db I restored an older zoneminder folder and installed again. The upgrade did work. What isn't working post upgrade is object detection / event server related stuff. This was working per the previous zoneminder version (pre template change) Is this what others have experienced? Do you know what the issue is? Edited January 5, 2021 by repomanz Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.