sillywalks Posted January 5, 2021 Share Posted January 5, 2021 (edited) So I just updated from the broken release to the latest version and still have the database corrupted issues. The container can't even start. Do I have to start from scratch? That would be quite annoying as I had most of my settings nicely tweaked What can I do to get this running again without having to start at zero? Starting services... * Starting Apache httpd web server apache2 * * Starting MariaDB database server mysqld Jan 5 08:23:43 e490f9ecde0f web_php[19211]: FAT [Failed db connection to ] Jan 5 08:23:43 e490f9ecde0f web_php[19211]: ERR [Unable to authenticate user. error decoding JWT token:Signature verification failed] Jan 5 08:23:43 e490f9ecde0f web_php[19211]: ERR [Unable to authenticate user. error decoding JWT token:Signature verification failed] Jan 5 08:23:43 e490f9ecde0f web_php[19212]: FAT [Failed db connection to ] Jan 5 08:23:43 e490f9ecde0f web_php[19213]: FAT [Failed db connection to ] Jan 5 08:23:43 e490f9ecde0f web_php[19214]: FAT [Failed db connection to ] Jan 5 08:23:43 e490f9ecde0f web_php[19215]: FAT [Failed db connection to ] Jan 5 08:23:43 e490f9ecde0f web_php[19215]: ERR [Unable to authenticate user. error decoding JWT token:Signature verification failed] Jan 5 08:23:43 e490f9ecde0f web_php[19215]: ERR [Unable to authenticate user. error decoding JWT token:Signature verification failed] ... # Some hundred more failed attempts in a few seconds ... Jan 5 08:23:55 e490f9ecde0f web_php[19214]: FAT [Failed db connection to ] Jan 5 08:23:55 e490f9ecde0f web_php[19215]: FAT [Failed db connection to ] Jan 5 08:23:55 e490f9ecde0f web_php[19211]: FAT [Failed db connection to ] ...fail! DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. Jan 5 08:24:03 e490f9ecde0f zmupdate[19652]: ERR [Error reconnecting to db: errstr:Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) error val:] DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. Can't call method "prepare_cached" on an undefined value at /usr/share/perl5/ZoneMinder/Config.pm line 96. BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder/Config.pm line 147. Compilation failed in require at /usr/bin/zmupdate.pl line 73. BEGIN failed--compilation aborted at /usr/bin/zmupdate.pl line 73. Jan 5 08:24:03 e490f9ecde0f zmupdate[19652]: ERR [Error reconnecting to db: errstr:Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) error val:] DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. Jan 5 08:24:03 e490f9ecde0f zmupdate[19653]: ERR [Error reconnecting to db: errstr:Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) error val:] DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. Can't call method "prepare_cached" on an undefined value at /usr/share/perl5/ZoneMinder/Config.pm line 96. BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder/Config.pm line 147. Compilation failed in require at /usr/bin/zmupdate.pl line 73. BEGIN failed--compilation aborted at /usr/bin/zmupdate.pl line 73. Jan 5 08:24:03 e490f9ecde0f zmupdate[19653]: ERR [Error reconnecting to db: errstr:Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) error val:] Starting ZoneMinder: DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. Jan 5 08:24:03 e490f9ecde0f zmpkg[19663]: ERR [Error reconnecting to db: errstr:Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) error val:] DBI connect('database=zm;host=localhost','zmuser',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/perl5/ZoneMinder/Database.pm line 110. Can't call method "prepare_cached" on an undefined value at /usr/share/perl5/ZoneMinder/Config.pm line 96. BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder/Config.pm line 147. Compilation failed in require at /usr/share/perl5/ZoneMinder.pm line 33. BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder.pm line 33. Compilation failed in require at /usr/bin/zmpkg.pl line 34. BEGIN failed--compilation aborted at /usr/bin/zmpkg.pl line 34. Jan 5 08:24:03 e490f9ecde0f zmpkg[19663]: ERR [Error reconnecting to db: errstr:Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) error val:] ZoneMinder failed to start *** /etc/my_init.d/40_firstrun.sh failed with status 255 *** Killing all processes... Edited January 5, 2021 by sillywalks Quote Link to comment
Borbosch Posted January 5, 2021 Share Posted January 5, 2021 Had the same problem today with the update and the idea of having a corrupted database. After some researching how to start the zoneminder-docker without automatically starting mysql and zoneminder (Create an environment variable: NO_START_ZM="1") i could connect to the console of the docker. There i could run mysqld in safe mode root@b02f8d0fc773:/# mysqld_safe That just helped me finding the error-log (mysqld_safe Logging to '/var/log/mysqld.log'.). In the error log i got the error "InnoDB: Upgrade after a crash is not supported." After some more searching i found this: https://discuss.erpnext.com/t/solved-mariadb-10-1-to-mariadb-10-2-upgrade-crash/35048 Then i just deleted the /var/lib/mysql/ib_logfile* and startet the Zoneminder-Docker without the NO_START_ZM="1" and everything works now. Quote Link to comment
dlandon Posted January 5, 2021 Author Share Posted January 5, 2021 6 hours ago, repomanz said: quick update. To test out the shm issue corrupting db I restored an older zoneminder folder and installed again. The upgrade did work. What isn't working post upgrade is object detection / event server related stuff. This was working per the previous zoneminder version (pre template change) Is this what others have experienced? Do you know what the issue is? ES has been updated to 6.1.5. You'll probably need to update your configuration. See the documentation mentioned above for help. Quote Link to comment
dlandon Posted January 5, 2021 Author Share Posted January 5, 2021 4 hours ago, sillywalks said: Do I have to start from scratch? That would be quite annoying as I had most of my settings nicely tweaked You can always do a restore from your backup. Quote Link to comment
repomanz Posted January 5, 2021 Share Posted January 5, 2021 6 hours ago, Borbosch said: Had the same problem today with the update and the idea of having a corrupted database. After some researching how to start the zoneminder-docker without automatically starting mysql and zoneminder (Create an environment variable: NO_START_ZM="1") i could connect to the console of the docker. There i could run mysqld in safe mode root@b02f8d0fc773:/# mysqld_safe That just helped me finding the error-log (mysqld_safe Logging to '/var/log/mysqld.log'.). In the error log i got the error "InnoDB: Upgrade after a crash is not supported." After some more searching i found this: https://discuss.erpnext.com/t/solved-mariadb-10-1-to-mariadb-10-2-upgrade-crash/35048 Then i just deleted the /var/lib/mysql/ib_logfile* and startet the Zoneminder-Docker without the NO_START_ZM="1" and everything works now. good stuff here, possible should be added in first post when people run into the shm / db corrupt issue. Quote Link to comment
Jokerigno Posted January 5, 2021 Share Posted January 5, 2021 17 hours ago, dlandon said: Building a new docker with a fix. You create the 99-mysql.conf from the zm.default file in the appdata/conf/ folder. The file actually gets copied to /etc/zm/conf.d/ inside the container. Sorry but I think I'm not understanding. I've created a file 99-mysql.conf and put it in appdata/conf/ folder. The content of this file it's just this Quote ZM_DB_HOST=192.168.2.100:3306 # ZoneMinder database name ZM_DB_NAME=zm # ZoneMinder database user ZM_DB_USER=zmuser # ZoneMinder database password ZM_DB_PASS=zmpass I leaved the default values in zm.default (but I've tried also commenting them) At boot it doesn't work: This is the error log: Quote * Starting Apache httpd web server apache2 * * Starting MariaDB database server mysqld ...done. Jan 5 15:59:05 c0caad8c37d8 /etc/mysql/debian-start[812]: Upgrading MySQL tables if necessary. Jan 5 15:59:05 c0caad8c37d8 /etc/mysql/debian-start[816]: Looking for 'mysql' as: /usr/bin/mysql Jan 5 15:59:05 c0caad8c37d8 /etc/mysql/debian-start[816]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck Jan 5 15:59:05 c0caad8c37d8 /etc/mysql/debian-start[816]: This installation of MySQL is already upgraded to 10.3.25-MariaDB, use --force if you still need to run mysql_upgrade Jan 5 15:59:05 c0caad8c37d8 /etc/mysql/debian-start[823]: Checking for insecure root accounts. Jan 5 15:59:05 c0caad8c37d8 /etc/mysql/debian-start[827]: Triggering myisam-recover for all MyISAM tables and aria-recover for all Aria tables DBD::mysql::st execute failed: Table 'zm.Config' doesn't exist at /usr/share/perl5/ZoneMinder/Config.pm line 97. Can't execute: Table 'zm.Config' doesn't exist at /usr/share/perl5/ZoneMinder/Config.pm line 147. BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder/Config.pm line 147. Compilation failed in require at /usr/bin/zmupdate.pl line 73. BEGIN failed--compilation aborted at /usr/bin/zmupdate.pl line 73. DBD::mysql::st execute failed: Table 'zm.Config' doesn't exist at /usr/share/perl5/ZoneMinder/Config.pm line 97. Can't execute: Table 'zm.Config' doesn't exist at /usr/share/perl5/ZoneMinder/Config.pm line 147. BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder/Config.pm line 147. Compilation failed in require at /usr/bin/zmupdate.pl line 73. BEGIN failed--compilation aborted at /usr/bin/zmupdate.pl line 73. Starting ZoneMinder: DBD::mysql::st execute failed: Table 'zm.Config' doesn't exist at /usr/share/perl5/ZoneMinder/Config.pm line 97. Can't execute: Table 'zm.Config' doesn't exist at /usr/share/perl5/ZoneMinder/Config.pm line 147. BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder/Config.pm line 147. Compilation failed in require at /usr/share/perl5/ZoneMinder.pm line 33. BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder.pm line 33. Compilation failed in require at /usr/bin/zmpkg.pl line 34. BEGIN failed--compilation aborted at /usr/bin/zmpkg.pl line 34. ZoneMinder failed to start *** /etc/my_init.d/40_firstrun.sh failed with status 255 *** Killing all processes... Jan 5 15:59:05 c0caad8c37d8 syslog-ng[38]: syslog-ng shutting down; version='3.25.1' The only solution is commenting 99-mysql.conf. BTW I have another feedback. I tried to enable MQTT so I changed this in zmeventnotification.ini # Use MQTT for messaging (default: no) [mqtt] enable = yes # Allow you to set a custom MQTT topic name # default: zoneminder #topic = my topic name # MQTT server (default: 127.0.0.1) server = 192.168.2.100 # Authenticate to MQTT server as user # username = !MQTT_USERNAME # Password # password = !MQTT_PASSWORD # Set retain flag on MQTT messages (default: no) retain = no and changed values for !MQTT_USERNAME and !MQTT_PASSWORD in secret.ini MQTT_USERNAME=mysupersecretuser MQTT_PASSWORD=mysupersecretpassword At boot nothing changes. I checked with MQTT Explorer for topic zoneminder (who was ther with previous version) but its not. Any hint? Quote Link to comment
joselito Posted January 5, 2021 Share Posted January 5, 2021 (edited) How to update ES?? added new config for ES in objectconfig.ini: [ml] use_sequence = yes disable_locks= no stream_sequence = { 'frame_strategy': 'most_models', 'frame_set': 'snapshot,alarm', 'contig_frames_before_error': 5, 'max_attempts': 3, 'sleep_between_attempts': 4, 'resize':800 } ml_sequence= { 'general': { 'model_sequence': 'object' }, 'object': { 'general':{ 'pattern':'(car|truck|bicycle|motorbike|bus|person|dog|cat)', 'same_model_sequence_strategy': 'first' # also 'most', 'most_unique's }, 'sequence': [{ # YoloV4 on GPU if TPU fails (because sequence strategy is 'first') 'object_config':'{{base_data_path}}/models/yolov4/yolov4.cfg', 'object_weights':'{{base_data_path}}/models/yolov4/yolov4.weights', 'object_labels': '{{base_data_path}}/models/yolov4/coco.names', 'object_min_confidence': 0.3, 'object_framework':'opencv', 'object_processor': 'gpu' }] } Also tried to disable use_sequence: no, but dont work. Edited January 5, 2021 by joselito Quote Link to comment
Jurykov Posted January 5, 2021 Share Posted January 5, 2021 12 hours ago, Borbosch said: That just helped me finding the error-log (mysqld_safe Logging to '/var/log/mysqld.log'.). In the error log i got the error "InnoDB: Upgrade after a crash is not supported." After some more searching i found this: https://discuss.erpnext.com/t/solved-mariadb-10-1-to-mariadb-10-2-upgrade-crash/35048 Then i just deleted the /var/lib/mysql/ib_logfile* and startet the Zoneminder-Docker without the NO_START_ZM="1" and everything works now. Thanks for this! Also fixed my 'corrupted' database issues. Saved me setting up again from scratch. Quote Link to comment
dlandon Posted January 5, 2021 Author Share Posted January 5, 2021 17 hours ago, repomanz said: quick update. To test out the shm issue corrupting db I restored an older zoneminder folder and installed again. The upgrade did work. What isn't working post upgrade is object detection / event server related stuff. This was working per the previous zoneminder version (pre template change) Is this what others have experienced? Do you know what the issue is? ES has been upgraded to 6.1.5. You probably need to make some changes. Several posts up is a link to the documentation. Quote Link to comment
dlandon Posted January 5, 2021 Author Share Posted January 5, 2021 7 hours ago, Jokerigno said: Sorry but I think I'm not understanding. I've created a file 99-mysql.conf and put it in appdata/conf/ folder. The content of this file it's just this I leaved the default values in zm.default (but I've tried also commenting them) At boot it doesn't work: This is the error log: The only solution is commenting 99-mysql.conf. BTW I have another feedback. I tried to enable MQTT so I changed this in zmeventnotification.ini # Use MQTT for messaging (default: no) [mqtt] enable = yes # Allow you to set a custom MQTT topic name # default: zoneminder #topic = my topic name # MQTT server (default: 127.0.0.1) server = 192.168.2.100 # Authenticate to MQTT server as user # username = !MQTT_USERNAME # Password # password = !MQTT_PASSWORD # Set retain flag on MQTT messages (default: no) retain = no and changed values for !MQTT_USERNAME and !MQTT_PASSWORD in secret.ini MQTT_USERNAME=mysupersecretuser MQTT_PASSWORD=mysupersecretpassword At boot nothing changes. I checked with MQTT Explorer for topic zoneminder (who was ther with previous version) but its not. Any hint? Go to the Zoneminder forum for help with the database issues. Go to the link posted earlier about ES 6.1.5. Quote Link to comment
dlandon Posted January 5, 2021 Author Share Posted January 5, 2021 5 hours ago, joselito said: How to update ES?? added new config for ES in objectconfig.ini: [ml] use_sequence = yes disable_locks= no stream_sequence = { 'frame_strategy': 'most_models', 'frame_set': 'snapshot,alarm', 'contig_frames_before_error': 5, 'max_attempts': 3, 'sleep_between_attempts': 4, 'resize':800 } ml_sequence= { 'general': { 'model_sequence': 'object' }, 'object': { 'general':{ 'pattern':'(car|truck|bicycle|motorbike|bus|person|dog|cat)', 'same_model_sequence_strategy': 'first' # also 'most', 'most_unique's }, 'sequence': [{ # YoloV4 on GPU if TPU fails (because sequence strategy is 'first') 'object_config':'{{base_data_path}}/models/yolov4/yolov4.cfg', 'object_weights':'{{base_data_path}}/models/yolov4/yolov4.weights', 'object_labels': '{{base_data_path}}/models/yolov4/coco.names', 'object_min_confidence': 0.3, 'object_framework':'opencv', 'object_processor': 'gpu' }] } Also tried to disable use_sequence: no, but dont work. ES is updated whenever the Docker is restarted. Go to the 6.1.5 documentation link posted earlier for help with ES. Quote Link to comment
repomanz Posted January 5, 2021 Share Posted January 5, 2021 (edited) Hey everyone - for those doing ES/ML I have a question. I have cuda / gpu enabled zoneminder container; should I be seeing some other process related to el/ml hanging on to the graphics card? the 3 zmc processes you see below are my high res monitors (nodetect) and use gpu to convert the 5mp stream down in size. Edited January 5, 2021 by repomanz Quote Link to comment
Nickglott Posted January 5, 2021 Share Posted January 5, 2021 1 hour ago, Jurykov said: Thanks for this! Also fixed my 'corrupted' database issues. Saved me setting up again from scratch. This is not working for me, it did allow ZM docker to start but none of my cams are capturing and at the top it says "/dev/shm: 100%". Log file spits all kinds of errors... Any one else getting this? Quote Link to comment
dlandon Posted January 5, 2021 Author Share Posted January 5, 2021 1 minute ago, Nickglott said: This is not working for me, it did allow ZM docker to start but none of my cams are capturing and at the top it says "/dev/shm: 100%". Log file spits all kinds of errors... Any one else getting this? Add more shared memory. It needs to be much lower. Set it to half your installed memory. Quote Link to comment
Nickglott Posted January 5, 2021 Share Posted January 5, 2021 22 minutes ago, dlandon said: Add more shared memory. It needs to be much lower. Set it to half your installed memory. That worked. Thanks added FYI for others I added --shm-size="16G" Quote Link to comment
repomanz Posted January 6, 2021 Share Posted January 6, 2021 (edited) @dlandon Does your container contain python cv2 module? Getting this error with ML side of event server. root@43f7cbf04308:/# sudo -u www-data /var/lib/zmeventnotification/bin/zm_detect.py --config /etc/zm/objectconfig.ini --eventid 152 --monitorid 1 --debug Traceback (most recent call last): File "/var/lib/zmeventnotification/bin/zm_detect.py", line 15, in <module> import imutils File "/usr/local/lib/python3.8/dist-packages/imutils/__init__.py", line 8, in <module> from .convenience import translate File "/usr/local/lib/python3.8/dist-packages/imutils/convenience.py", line 6, in <module> import cv2 oot@43f7cbf04308:/# python3 Python 3.8.5 (default, Jul 28 2020, 12:59:40) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import cv2 Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'cv2' This is based on the information here: https://zmeventnotification.readthedocs.io/en/latest/guides/hooks.html?highlight=cv2#make-sure-opencv-works Quote Important After you install opencv, make sure it works. Start python3 and inside the interpreter, do a import cv2. If it seg faults, you have a problem with the package you installed. Like I said, I’ve never had issues after building from source. Note that if you get an error saying cv2 not found that means you did not install it in a place python3 can find it (you might have installed it for python2 by mistake) This is brand new installation. edit: I think I solved this by going into container and running this command: sudo pip3 install opencv-python Shouldn't this be included in your build process for the container? I also noticed that the pushover py file was missing from /var/lib/zmeventnotification/bin After adding pushover py script and manually adding the opencv-python to the container my complete solution (zm/es/ml) is now working. Edited January 6, 2021 by repomanz Quote Link to comment
ThreeFN Posted January 6, 2021 Share Posted January 6, 2021 22 hours ago, dlandon said: ES is updated whenever the Docker is restarted. Go to the 6.1.5 documentation link posted earlier for help with ES. I think I may have figured it out for some (most?) people having these problems, maybe? A) dlandon is right, you really need to go into the configs (specifically objectconfig.ini) and understand (at least somewhat) what is going on in there. Everyone's config is a little different (Yolo v3 or v4? tiny or not? cpu, gpu, or tpu?, etc) and will need to be setup for what hardware/software you're using. B) the default [object] section looks like this: [object] # If you are using legacy format (use_sequence=no) then these parameters will # be used during ML inferencing object_detection_pattern=(person|car|motorbike|bus|truck|boat) object_min_confidence=0.3 object_framework=coral_edgetpu object_processor=tpu object_weights={{base_data_path}}/models/coral_edgetpu/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite object_labels={{base_data_path}}/models/coral_edgetpu/coco_indexed.names # If you are using the new ml_sequence format (use_sequence=yes) then # you can fiddle with these parameters and look at ml_sequence later # Note that these can be named anything. You can add custom variables, ad-infinitum # Google Coral tpu_object_weights={{base_data_path}}/models/coral_edgetpu/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite tpu_object_labels={{base_data_path}}/models/coral_edgetpu/coco_indexed.names tpu_object_framework=coral_edgetpu tpu_object_processor=tpu tpu_min_confidence=0.6 # Yolo v4 on GPU (falls back to CPU if no GPU) yolo4_object_weights={{base_data_path}}/models/yolov4/yolov4.weights yolo4_object_labels={{base_data_path}}/models/yolov4/coco.names yolo4_object_config={{base_data_path}}/models/yolov4/yolov4.cfg yolo4_object_framework=opencv yolo4_object_processor=gpu # Yolo v3 on GPU (falls back to CPU if no GPU) yolo3_object_weights={{base_data_path}}/models/yolov3/yolov3.weights yolo3_object_labels={{base_data_path}}/models/yolov3/coco.names yolo3_object_config={{base_data_path}}/models/yolov3/yolov3.cfg yolo3_object_framework=opencv yolo3_object_processor=gpu # Tiny Yolo V4 on GPU (falls back to CPU if no GPU) tinyyolo_object_config={{base_data_path}}/models/tinyyolov4/yolov4-tiny.cfg tinyyolo_object_weights={{base_data_path}}/models/tinyyolov4/yolov4-tiny.weights tinyyolo_object_labels={{base_data_path}}/models/tinyyolov4/coco.names tinyyolo_object_framework=opencv tinyyolo_object_processor=gpu these are basically configs for each of the different setups you might have with Yolo, cpu, gpu, tpu, etc. And looking at the default ML_SEQUENCE: ml_sequence= { 'general': { 'model_sequence': 'object,face,alpr', 'disable_locks': '{{disable_locks}}', }, 'object': { 'general':{ 'pattern':'{{object_detection_pattern}}', 'same_model_sequence_strategy': 'first' # also 'most', 'most_unique's }, 'sequence': [{ #First run on TPU with higher confidence 'object_weights':'{{tpu_object_weights}}', 'object_labels': '{{tpu_object_labels}}', 'object_min_confidence': {{tpu_min_confidence}}, 'object_framework':'{{tpu_object_framework}}', 'tpu_max_processes': {{tpu_max_processes}}, 'tpu_max_lock_wait': {{tpu_max_lock_wait}}, 'max_detection_size':'{{max_detection_size}}' }, { # YoloV4 on GPU if TPU fails (because sequence strategy is 'first') 'object_config':'{{yolo4_object_config}}', 'object_weights':'{{yolo4_object_weights}}', 'object_labels': '{{yolo4_object_labels}}', 'object_min_confidence': {{object_min_confidence}}, 'object_framework':'{{yolo4_object_framework}}', 'object_processor': '{{yolo4_object_processor}}', 'gpu_max_processes': {{gpu_max_processes}}, 'gpu_max_lock_wait': {{gpu_max_lock_wait}}, 'cpu_max_processes': {{cpu_max_processes}}, 'cpu_max_lock_wait': {{cpu_max_lock_wait}}, 'max_detection_size':'{{max_detection_size}}' }] }, 'face': { 'general':{ 'pattern': '{{face_detection_pattern}}', 'same_model_sequence_strategy': 'first' }, 'sequence': [{ 'save_unknown_faces':'{{save_unknown_faces}}', 'save_unknown_faces_leeway_pixels':{{save_unknown_faces_leeway_pixels}}, 'face_detection_framework': '{{face_detection_framework}}', 'known_images_path': '{{known_images_path}}', 'unknown_images_path': '{{unknown_images_path}}', 'face_model': '{{face_model}}', 'face_train_model': '{{face_train_model}}', 'face_recog_dist_threshold': '{{face_recog_dist_threshold}}', 'face_num_jitters': '{{face_num_jitters}}', 'face_upsample_times':'{{face_upsample_times}}', 'gpu_max_processes': {{gpu_max_processes}}, 'gpu_max_lock_wait': {{gpu_max_lock_wait}}, 'cpu_max_processes': {{cpu_max_processes}}, 'cpu_max_lock_wait': {{cpu_max_lock_wait}}, 'max_size':800 }] }, 'alpr': { 'general':{ 'same_model_sequence_strategy': 'first', 'pre_existing_labels':['car', 'motorbike', 'bus', 'truck', 'boat'], 'pattern': '{{alpr_detection_pattern}}' }, 'sequence': [{ 'alpr_api_type': '{{alpr_api_type}}', 'alpr_service': '{{alpr_service}}', 'alpr_key': '{{alpr_key}}', 'platrec_stats': '{{platerec_stats}}', 'platerec_min_dscore': {{platerec_min_dscore}}, 'platerec_min_score': {{platerec_min_score}}, 'max_size':1600 }] } } you see that the sequence sets things up to use the TPU first, then to try the yolo v4 on gpu. If TPU fails (it will in many cases cause I don't see a lot about people using Google Coral) it tries the next in the sequence which is yolo v4, so on and so forth. I assume I know why pliablepixels is doing it this way, he's trying to make the configs more robust so they work without modification; eg imagine a config that is 'try whatever processor the user may have in order of processing speed' (TPU > GPU > CPU) and one will work and successfully detect of what hardware the user is running. It also adds growth for more dectection methods other than object, face, alpr, etc in the future. This extensibility and multiple dectors may be conflicting with the container, which certainly in my case, I have setup to only compile one model (yolo v4 full) for CPU (waiting on 6.9 stable before I try GPU). Keep in mind ES doc etc is written for a OS install and not a container. I think that when the config says: # Yolo v4 on GPU (falls back to CPU if no GPU) It isn't strictly true, either opencv doesn't compile code to allow this to happen or something similar maybe. I changed: yolo4_object_processor=cpu And simplified ml_sequence to ml_sequence= { 'general': { #'model_sequence': 'object,face,alpr', 'model_sequence': 'object', 'disable_locks': '{{disable_locks}}', }, 'object':{ 'general':{ 'pattern':'{{object_detection_pattern}}', 'same_model_sequence_strategy': 'first' # also 'most', 'most_unique's }, 'sequence': [ { # YoloV4 on CPU only 'object_config':'{{yolo4_object_config}}', 'object_weights':'{{yolo4_object_weights}}', 'object_labels': '{{yolo4_object_labels}}', 'object_min_confidence': {{object_min_confidence}}, 'object_framework':'{{yolo4_object_framework}}', 'object_processor': '{{yolo4_object_processor}}', 'gpu_max_processes': {{gpu_max_processes}}, 'gpu_max_lock_wait': {{gpu_max_lock_wait}}, 'cpu_max_processes': {{cpu_max_processes}}, 'cpu_max_lock_wait': {{cpu_max_lock_wait}}, 'max_detection_size':'{{max_detection_size}}' }] } } And things started behaving. Yes technically the gpu_max could probably go as well but I was being small and deliberate in changes to make sure I didn't goof. Anyways these are my findings so far. Quote Link to comment
dlandon Posted January 6, 2021 Author Share Posted January 6, 2021 4 hours ago, repomanz said: edit: I think I solved this by going into container and running this command: sudo pip3 install opencv-python Shouldn't this be included in your build process for the container? I also noticed that the pushover py file was missing from /var/lib/zmeventnotification/bin Opencv (cv2), is compiled when the docker starts. It needs to be compiled for your computer configuration. I've added the pushapi_pushover.py file to the ES bundle. Next time you restart the docker, it will be updated. Quote Link to comment
repomanz Posted January 7, 2021 Share Posted January 7, 2021 (edited) 5 hours ago, dlandon said: Opencv (cv2), is compiled when the docker starts. It needs to be compiled for your computer configuration. I've added the pushapi_pushover.py file to the ES bundle. Next time you restart the docker, it will be updated. Hi @dlandon Understood. That's what I'm pointing out. I watched the logs during install and opencv took about 20 minutes to compile. I had to add that package specifically into the container in order to fix cv2. Maybe there is a dependency issue with the libraries being called for compile? thx for adding pushover. Edited January 7, 2021 by repomanz Quote Link to comment
BigChris Posted January 7, 2021 Share Posted January 7, 2021 (edited) After the update today I get the following error message when calling zoneminder: Unable to connect to ZM db.SQLSTATE[HY000] [2002] No such file or directory The container is automatically terminated. What can I do? Update I fix it wirth this: Because of the change in how shared memory is set up, you will need to use the new Zoneminder template:;, Do this procedure: - Remove the docker image. All your data and settings will be kept. - Click on add a container and delete the Zoneminder template. - Go to he apps and get the new template. Read about adjusting shared memory and be sure to increase until you get to less than 50% on the Zoneminder console. Edited January 7, 2021 by BigChris Quote Link to comment
joselito Posted January 7, 2021 Share Posted January 7, 2021 It is something else, not the memory error. reinstalled a least 10x in the night: ``` sudo docker run -d --name="Zoneminder" --net="bridge" --privileged="false" --shm-size="30G" -p 888:80/tcp -p 9001:9001/tcp -p 6802:6802/tcp -e TZ="Europe/Berlin" -e PUID="1000" -e PGID="1000" -e INSTALL_HOOK="1" -e INSTALL_FACE="1" -e INSTALL_TINY_YOLOV3="1" -e INSTALL_YOLOV3="1" -e INSTALL_TINY_YOLOV4="1" -e INSTALL_YOLOV4="1" -e MULTI_PORT_START="0" -e MULTI_PORT_END="0" --device /dev/dri:/dev/dri -v "/mnt/Zoneminder":"/config":rw -v "/mnt/Zoneminder/data":"/var/cache/zoneminder":rw -v "/media/zm":"/mnt/zm":rw dlandon/zoneminder ``` or: ``` version: '3.1' services: zoneminder: container_name: Zoneminder image: dlandon/zoneminder:latest restart: unless-stopped ports: - 888:80/tcp - 9001:9001/tcp - 6802:6802/tcp network_mode: "bridge" privileged: false shm_size: 32G environment: - TZ=Europe/Berlin - PUID=1000 - PGID=1000 - INSTALL_HOOK=1 - INSTALL_FACE=0 - INSTALL_TINY_YOLOV3=0 - INSTALL_YOLOV3=0 - INSTALL_TINY_YOLOV4=0 - INSTALL_YOLOV4=1 - MULTI_PORT_START=0 - MULTI_PORT_END=0 volumes: - /mnt/Zoneminder:/config:rw - /mnt/Zoneminder/data:/var/cache/zoneminder:rw - /media/zm:/mnt/zm:rw devices: - /dev/dri:/dev/dri ``` I get always if I install INSTALL_HOOK=1: ``` Unrecoverable error:'ml_sequence' Traceback:Traceback (most recent call last): File "/var/lib/zmeventnotification/bin/zm_detect.py", line 531, in <module> main_handler() File "/var/lib/zmeventnotification/bin/zm_detect.py", line 295, in main_handler if g.config['ml_sequence'] and g.config['use_sequence'] == 'yes':KeyError: 'ml_sequence' ``` If i install with INSTALL_HOOK=0, then the error dissapered but no zm_detect in logs Quote Link to comment
dlandon Posted January 7, 2021 Author Share Posted January 7, 2021 4 hours ago, repomanz said: Hi @dlandon Understood. That's what I'm pointing out. I watched the logs during install and opencv took about 20 minutes to compile. I had to add that package specifically into the container in order to fix cv2. Maybe there is a dependency issue with the libraries being called for compile? thx for adding pushover. No one else is reporting this issue with cv2. Quote Link to comment
dlandon Posted January 7, 2021 Author Share Posted January 7, 2021 45 minutes ago, joselito said: It is something else, not the memory error. reinstalled a least 10x in the night: ``` sudo docker run -d --name="Zoneminder" --net="bridge" --privileged="false" --shm-size="30G" -p 888:80/tcp -p 9001:9001/tcp -p 6802:6802/tcp -e TZ="Europe/Berlin" -e PUID="1000" -e PGID="1000" -e INSTALL_HOOK="1" -e INSTALL_FACE="1" -e INSTALL_TINY_YOLOV3="1" -e INSTALL_YOLOV3="1" -e INSTALL_TINY_YOLOV4="1" -e INSTALL_YOLOV4="1" -e MULTI_PORT_START="0" -e MULTI_PORT_END="0" --device /dev/dri:/dev/dri -v "/mnt/Zoneminder":"/config":rw -v "/mnt/Zoneminder/data":"/var/cache/zoneminder":rw -v "/media/zm":"/mnt/zm":rw dlandon/zoneminder ``` or: ``` version: '3.1' services: zoneminder: container_name: Zoneminder image: dlandon/zoneminder:latest restart: unless-stopped ports: - 888:80/tcp - 9001:9001/tcp - 6802:6802/tcp network_mode: "bridge" privileged: false shm_size: 32G environment: - TZ=Europe/Berlin - PUID=1000 - PGID=1000 - INSTALL_HOOK=1 - INSTALL_FACE=0 - INSTALL_TINY_YOLOV3=0 - INSTALL_YOLOV3=0 - INSTALL_TINY_YOLOV4=0 - INSTALL_YOLOV4=1 - MULTI_PORT_START=0 - MULTI_PORT_END=0 volumes: - /mnt/Zoneminder:/config:rw - /mnt/Zoneminder/data:/var/cache/zoneminder:rw - /media/zm:/mnt/zm:rw devices: - /dev/dri:/dev/dri ``` I get always if I install INSTALL_HOOK=1: ``` Unrecoverable error:'ml_sequence' Traceback:Traceback (most recent call last): File "/var/lib/zmeventnotification/bin/zm_detect.py", line 531, in <module> main_handler() File "/var/lib/zmeventnotification/bin/zm_detect.py", line 295, in main_handler if g.config['ml_sequence'] and g.config['use_sequence'] == 'yes':KeyError: 'ml_sequence' ``` If i install with INSTALL_HOOK=0, then the error dissapered but no zm_detect in logs Why aren't you using the Docker template? Quote Link to comment
joselito Posted January 7, 2021 Share Posted January 7, 2021 sorry, but which? I use it from your site on github Quote Link to comment
dlandon Posted January 7, 2021 Author Share Posted January 7, 2021 1 minute ago, joselito said: sorry, but which? I use it from your site on github This is a forum for Unraid support. Are you using this Docker on Unraid? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.