[support] dlandon - Zoneminder 1.36


Recommended Posts

I just had an issue with ZoneMinder after an update. 
It started complaining about MariaDb/SQL and refusing to start. 

 

After reading the last few pages, I removed the log files, and it's seemingly all good again, Figured a message to layout what I did in simple terms (i.e. terms I can follow) might help others, and check I've not goofed in any way. 

 

I went to appdata, and removed the following files (There were 2) : /mysql/ib_logfile*

Link to comment

@dlandon FYI in the latest build for GPU, as the file is mv not cp the .pub is not there when opencv_ok has 'yes' present. So fails on rebuilding

 

Jan 7 07:47:39 3aa4a63b1d0a EventServer: Compiling opencv with GPU Support
Jan 7 07:47:39 3aa4a63b1d0a EventServer: Installing cuda toolkit...
Warning: apt-key output should not be parsed (stdout is not a terminal)
gpg: can't open '/var/cuda-repo-10-2-local-10.2.89-440.33.01/7fa2af80.pub': No such file or directory
Warning: apt-key output should not be parsed (stdout is not a terminal)
gpg: can't open '/var/cuda-repo-10-2-local-10.2.89-440.33.01/7fa2af80.pub': No such file or directory
W: GPG error: file:/var/cuda-repo-ubuntu1804-11-1-local Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY F60F4B3D7FA2AF80

EDIT: Also need to update to 20.04 Nvidia components

Edited by Jaburges
Link to comment

The time has come for me to make some changes to the Zoneminder Docker.  It has become a real support burden for me and I just don't have the time for all the support issues.  For those of you using ES machine learning (hook processing), you will need to move to building a VM for Zoneminder.  This Docker is acting more like a VM than a Docker and that is not how Dockers are supposed to work.

 

There will be no more updates to this Docker.  I will be building a new Zoneminder Docker that will have basic ES notifications of events and that is all.

Link to comment
1 hour ago, dlandon said:

The time has come for me to make some changes to the Zoneminder Docker.  It has become a real support burden for me and I just don't have the time for all the support issues.  For those of you using ES machine learning (hook processing), you will need to move to building a VM for Zoneminder.  This Docker is acting more like a VM than a Docker and that is not how Dockers are supposed to work.

 

There will be no more updates to this Docker.  I will be building a new Zoneminder Docker that will have basic ES notifications of events and that is all.

Sorry it’s been a burden, it is a huge complicated docker, and you’ve put a lot of great work into creating this. We and Zoneminder owe you a lot of thanks!

 

in terms of the best way to approach Zoneminder moving forward - could a micro services approach work? It will add a little more overhead but would it make sense to have 

Zoneminder + SQL + Eventserver (hooks)

and then could be docker composed together or upgraded individually?

thinking a no-hooks ES, and a hooks ES? I could be over simplifying?

 

my knowledge isn’t super deep, but If I can help LMK

Link to comment
2 hours ago, Jaburges said:

Sorry it’s been a burden, it is a huge complicated docker, and you’ve put a lot of great work into creating this. We and Zoneminder owe you a lot of thanks!

 

in terms of the best way to approach Zoneminder moving forward - could a micro services approach work? It will add a little more overhead but would it make sense to have 

Zoneminder + SQL + Eventserver (hooks)

and then could be docker composed together or upgraded individually?

thinking a no-hooks ES, and a hooks ES? I could be over simplifying?

 

my knowledge isn’t super deep, but If I can help LMK

Zoneminder , mysql, and simple ES notifications.  No hooks processing.

Link to comment
2 hours ago, Jaburges said:

Sorry it’s been a burden, it is a huge complicated docker, and you’ve put a lot of great work into creating this. We and Zoneminder owe you a lot of thanks!

 

in terms of the best way to approach Zoneminder moving forward - could a micro services approach work? It will add a little more overhead but would it make sense to have 

Zoneminder + SQL + Eventserver (hooks)

and then could be docker composed together or upgraded individually?

thinking a no-hooks ES, and a hooks ES? I could be over simplifying?

 

my knowledge isn’t super deep, but If I can help LMK

ES does support remote ML server as described here.

 

Which if you go down the rabit hole, you get pliablepixels mlapi software, which does have a containerized version someone has made (and may have GPU support?).  It may be possible even now to glue this all together.

 

Obviously experimentation must ensue.

 

The more I dig through stuff the more I tend to agree with dlandon that this container is doing a lot of 'init install' stuff that is more OS/VM behavior than docker pull/start behavior and I don't fault wanting to kick that to the curb.

 

Having said that ZM is 'almost exclusively made useful' by ES's ML stuff for false positive rejection, so no ML = no ZM for me.  So at the moment it looks like the options are spin up a VM and get a well supported installation (pliablepixels documentation), or investigate the aforementioned remote ML docker.  My preference is for the latter because at the moment, containerization is about the only way in unraid to spread a GPU amongst multiple work loads (eg plex & ML) unless/until vGPU/SRIOV/etc support is added and VMs & docker can share.

 

I guess the other solution would be to move ML processing to a Google Coral device, and give that device to the ZM VM.  Or even go the route of a TPU & mlapi VM remoted to the ZM docker.  The benchmarks seem to indicate the TPU is good but maybe hamstrung a bit by it's usb 2.0 (or slower) interface to get the data.  Not sure if the M.2 versions would be any faster, if TPU can't saturate 2.0 that seem like the chip is the bottleneck and not it's interface...

 

Hell of a device for the cost, size, and power draw though...

 

Dlandon, I'm guessing you'll be dropping GPU support entirely from the docker?  Like even for ffmpeg use directly by ZM and not for ML (h.264 decode etc)?  Or is that something that doesn't require your direct support (give container gpu, ffmpeg sorts it out) and will work on its own?

Link to comment
8 minutes ago, ThreeFN said:

Dlandon, I'm guessing you'll be dropping GPU support entirely from the docker?  Like even for ffmpeg use directly by ZM and not for ML (h.264 decode etc)?  Or is that something that doesn't require your direct support (give container gpu, ffmpeg sorts it out) and will work on its own?

With the upgrade to Ubuntu 20.04 and the built in Zoneminder support for GPU, it should work fine.  The fancy GPU stuff (opencv) was for hook processing.

Link to comment

I just rebuilt my Docker and template file for Zoneminder. In addition I applied the recommended setting for memory. My question is each time I try to limit the amount of storage space that I want to allow Zoneminder to use. The program never accurately reflects this. For example lets say I just want to give it 10G for video storage. However, when I go into the user interface it shows my entire storage array amount of a couple to terabytes. How do I limit the amount of available storage for the docker container

Edited by ralieghnet
Link to comment
3 hours ago, dlandon said:

I've built a new Docker (zoneminder.unraid) that will be used when the template is used from CA.  Your current template will continue to work, but no updates will be applied to that Docker.

So also external mysql support are gone too?

Edited by Jokerigno
Link to comment
46 minutes ago, ralieghnet said:

I just rebuilt my Docker and template file for Zoneminder. In addition I applied the recommended setting for memory. My question is each time I try to limit the amount of storage space that I want to allow Zoneminder to use. The program never accurately reflects this. For example lets say I just want to give it 10G for video storage. However, when I go into the user interface it shows my entire storage array amount of a couple to terabytes. How do I limit the amount of available storage for the docker container

Are you trying to limit the RAM the container needs or the storage of video files on your array?

Link to comment
23 minutes ago, BlueRyuu said:

Hello.

Is it normal that the docker container image is growing too fast?

Yesterday it was 3.31GB and today it is  6.62GB.

image.png.64a0ad2e793b22220d5e372a08bfdb77.png

 

I'm saving the videos to an Unassigned device (Data Path).

At this rate it will fill up my docker image in less than a week :(

Check your data path mapping.  You probably have it inadvertently in the Docker container.

Link to comment
54 minutes ago, ralieghnet said:

I've already limited the RAM based on the number of cameras. I now want to limit the amount of storage array zoneminder has access to.

 

32 minutes ago, trurl said:

I don't use this particular container, but in general, image shouldn't grow. Post your docker run command as explained in this very first link in the Docker FAQ:

 

 

14 minutes ago, dlandon said:

Check your data path mapping.  You probably have it inadvertently in the Docker container.

This is all the same problem.  /EDIT (this sentence didn't leave my head) your ZM storage location needs to be corrected. /EDIT. /appdata/zoneminder/data is the default storage location for video/images that zoneminder stores.  ZM will delete when (default 95%?) full, go to the filters tab in ZM and you can adjust.

 

I highly recommend setting up a quota system to limit ZM's storage area, and recommend even more setting up an unassigned drive to be your video only storage.  Unraid doesn't do quota's in the normal sense (this share shall be XXXGiB in size) so you'll have to do your own research on how to accomplish this.  I use a separate unassigned drive for ZM storage (technically on another server).

 

Keep in mind the 'until full' filter in ZM is just based on free space on your drive.  If you add other stuff to the drive it will eat from your ZM storage.  Again I don't recommend storing ZM footage on you main array, give it it's own drive/partition/etc to itself.

Edited by ThreeFN
Link to comment
10 minutes ago, dlandon said:

Check your data path mapping.  You probably have it inadvertently in the Docker container.

I have it set in the docker to the Unassigned drive. Just opened the edit page of the docker just in case I missed something but it's set to that. I can even ssh to the unraid server, go to the unassigned device mounted path and the videos are there (480GB of files, 3 cams 24/7 recording).

image.thumb.png.3face66fac7e69288a4380094780ad71.png

 

This is the run command as requested by trurl:

image.thumb.png.1d180591ccff0faa357dfaf4a76e2a63.png

 

Link to comment
1 hour ago, BlueRyuu said:

Hello.

Is it normal that the docker container image is growing too fast?

Yesterday it was 3.31GB and today it is  6.62GB.

image.png.64a0ad2e793b22220d5e372a08bfdb77.png

 

I'm saving the videos to an Unassigned device (Data Path).

At this rate it will fill up my docker image in less than a week :(

Remind me again what's giving you this image/readout?  I'm struggling today.

 

EDIT: Oh FFS, 2 seconds after I post of course it comes to me, (Container Sizes) button on Docker page...(grumble grumble)

Edited by ThreeFN
Link to comment
46 minutes ago, BlueRyuu said:

I have it set in the docker to the Unassigned drive. Just opened the edit page of the docker just in case I missed something but it's set to that. I can even ssh to the unraid server, go to the unassigned device mounted path and the videos are there (480GB of files, 3 cams 24/7 recording).

image.thumb.png.3face66fac7e69288a4380094780ad71.png

 

This is the run command as requested by trurl:

image.thumb.png.1d180591ccff0faa357dfaf4a76e2a63.png

 

If it's still growing, My next best guess is that ZM config may have an error in it.  Have you changed this?

32548086_Screenshot2021-01-08154947.png.23d5c94bcdee8d718a37c56d52ba1e4a.png

 

Does the rate of growth of the container match your expected rate of recorded video?

 

Your config and run command looks correct to me. 

 

/mnt/disks/cctv/zoneminder has files in it, but does it have the LATEST files and ALL the files?  If the cctv disk wasn't mounted at one point when the docker started some files might go into the image instead, I'm pretty sure I goofed and broke things that way one time.

 

 

 

Edited by ThreeFN
Link to comment
  • dlandon changed the title to [support] dlandon - Zoneminder 1.36

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.