[support] Corgans Docker Repository - double-take, facebox, tagbox, videobox, compreface

[corgan]

[Application List]

 

1. machinebox/Facebox - Facebox detects and identifies faces in photos. You can teach facebox with as little as one sample image.

Project Page : https://machinebox.io/docs/facebox

Dockerhub Page : https://hub.docker.com/r/machinebox/facebox/

GitHub Page : https://github.com/corgan2222/unraid-templates

Latest Version : January 2021

 

2. double-take - Unified UI and API for processing and training images for facial recognition.

Project Page : https://github.com/jakowenko/double-take

Dockerhub Page : https://hub.docker.com/r/jakowenko/double-take

GitHub Page : https://github.com/jakowenko/double-take

Latest Version : Release 1.3.0 - 04.10.2021

 

3. machinebox/tagbox Tagbox lets you identify the content of images by getting a list of ordered tags.

Project Page: https://machinebox.io/docs/tagbox

Dockerhub Page: https://hub.docker.com/r/machinebox/tagbox

 

4. machinebox/videobox Videobox extracts frames from videos and passes them to other Machine Box boxes for processing.

Project Page: https://machinebox.io/docs/videobox

Dockerhub Page: https://hub.docker.com/r/machinebox/videobox

 

5. prometheus-phoenixstats This is a containerised version of phoenixstats for Prometheus.

Project Page: https://github.com/corgan2222/Prometheus-phoenixstats-Exporter

Dockerhub Page: https://hub.docker.com/repository/docker/stefanknaak/prometheus-phoenixstats

 

6. compreface - CompreFace is a leading free and open-source face recognition system

Project Page: https://github.com/exadel-inc/CompreFace

Dockerhub Page: https://hub.docker.com/r/exadel/compreface/

Github Issue: https://github.com/exadel-inc/CompreFace/issues/651

Latest Version : Release 14.12.2021

 

6. compreface GPU - CompreFace is a leading free and open-source face recognition system

Project Page: https://github.com/exadel-inc/CompreFace

Dockerhub Page: https://hub.docker.com/r/exadel/compreface/

Github Issue: https://github.com/exadel-inc/CompreFace/issues/651

Latest Version : Release 14.12.2021

Edited December 14, 2021 by corgan
compreface GPU

[Jako]

Hey there, I created Double Take and just wanted to thank you for creating the Unraid template. I've had some requests for one, so I'll direct people your way 🍻

[corgan]

6 hours ago, Jako said:

Hey there, I created Double Take and just wanted to thank you for creating the Unraid template. I've had some requests for one, so I'll direct people your way 🍻

 

Nice one! Thanks for this great app!

No problem! But I have one question, I could not find in your docs. Are the face information stored in your data container, or is everything stored in facebox/compreface? So which container data should be backuped. Yours, or the others?
 

[Jako]

2 hours ago, corgan said:

 

Nice one! Thanks for this great app!

No problem! But I have one question, I could not find in your docs. Are the face information stored in your data container, or is everything stored in facebox/compreface? So which container data should be backuped. Yours, or the others?
 

 

The detectors store the actual face data, Double Take just stores the results of the recognition / training from these detectors. If you lost the Double Take data and not the detectors, then it would just be out of sync and would need to be retrained. Probably best to back them all up if you want to be able to easily restore.

[corgan]

On 10/15/2021 at 4:56 PM, Jako said:

 

The detectors store the actual face data, Double Take just stores the results of the recognition / training from these detectors. If you lost the Double Take data and not the detectors, then it would just be out of sync and would need to be retrained. Probably best to back them all up if you want to be able to easily restore.

 

Thanks for the info. I just changed the double-take xml to make the /.storage folder persistent for easy backup in unraid.

[corgan]

added

 

6. compreface - CompreFace is a leading free and open-source face recognition system

Project Page: https://github.com/exadel-inc/CompreFace

Dockerhub Page: https://hub.docker.com/r/exadel/compreface/

Github Issue: https://github.com/exadel-inc/CompreFace/issues/651

[MarbleComa]

Thanks for making Compreface available for Unraid! Does the template need some additional fields added? I would like to try out the GPU version, which I guess at least requires the NVIDIA_VISIBLE_DEVICES and NVIDIA_DRIVER_CAPABILITIES variables to be provided. I also see there's no AppData mapping, but I would guess that's also required. Many thanks!

[Iceman24]

Was there anything else that needed to be done to CompreFace for it to all work correctly? I know there were some issues that were going to be taken to the unRAID devs.

[corgan]

On 11/27/2021 at 8:03 PM, Iceman24 said:

Was there anything else that needed to be done to CompreFace for it to all work correctly? I know there were some issues that were going to be taken to the unRAID devs.

 

We had some Problems with the Data Folder to get persistent.

This should now be fixed. Additionally, there is the possibility to use an external Postgres Server.

The new Template Version is pushed to Github. We just have to wait till it's available in the CA Store.

Edited December 14, 2021 by corgan

[bigbangus]

@corgan big thanks for getting these containers going with the compreface dev on github.

 

However, I am still unable to use either the regular or gpu version as is from the appstore.

 

This is what the GUI looks like in either case:

 

The error I'm getting boil down to (see log attached):

 

2021-12-14 06:51:54,200 INFO exited: compreface-postgres-db (exit status 1; not expected)
2021-12-14 06:51:55.097 ERROR 484 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Exception during pool initialization.

org.postgresql.util.PSQLException: The connection attempt failed.

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.

2021-12-14 06:51:55.115 ERROR 484 --- [ main] o.s.boot.SpringApplication : Application run failed

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [org/springframework/boot/autoconfigure/liquibase/LiquibaseAutoConfiguration$LiquibaseConfiguration.class]: Invocation of init method failed; nested exception is liquibase.exception.DatabaseException: org.postgresql.util.PSQLException: The connection attempt failed.

Caused by: liquibase.exception.DatabaseException: org.postgresql.util.PSQLException: The connection attempt failed.

Caused by: org.postgresql.util.PSQLException: The connection attempt failed.

compreface-log.txt

[corgan]

FATAL: "/var/lib/postgresql/data" is not a valid data directory

 

The Data Folder is wrong. Can you show me your configuration ?

[bigbangus]

Here is the docker command:

[email protected]:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='CompreFace' --net='bridge' -e TZ="America/New_York" -e HOST_OS="Unraid" -e HOST_HOSTNAME="UnraidNAS" -e HOST_CONTAINERNAME="CompreFace" -e 'EXTERNAL_DB'='false' -e 'POSTGRES_URL'='jdbc:postgresql://192.168.2.254:5432/compreface' -e 'POSTGRES_PASSWORD'='corgan80982' -e 'POSTGRES_USER'='compreface' -l net.unraid.docker.managed=dockerman -l net.unraid.docker.webui='http://[IP]:[PORT:80]' -l net.unraid.docker.icon='https://raw.githubusercontent.com/corgan2222/unraid-templates/main/img/compreface.png' -p '8800:80/tcp' -v '/mnt/user/appdata/compreface':'/var/lib/postgresql/data':'rw' 'exadel/compreface:0.6.1'
0b36eec7b5d84133acc6634470f94350ecb4a9291ee83aa7626a96415f13a955

 

Also on your github, I believe the data path is the same:

<Data>
    <Volume>
      <HostDir>/mnt/user/appdata/compreface</HostDir>
      <ContainerDir>/var/lib/postgresql/data</ContainerDir>
      <Mode>rw</Mode>
    </Volume>
  </Data>

 

[corgan]

What I'm wondering, the webui part is wrong. I had the port:80 Part wrong in an old Version. But this was fixed.

https://github.com/corgan2222/unraid-templates/blame/0c12ba77b8ea5df95510ac5c41dddf61842c2e2f/CompreFace-GPU/CompreFace-GPU.xml#L37

 

Maybe you still have an old cached version.

 

Please delete the compreface apps.
If its still exist, delete /mnt/user/appdata/compreface

Reinstall the app and then check if /mnt/user/appdata/compreface exists and if there are data inside.

 

[corgan]

ähm, are you doing the installation from the command line?

[bigbangus]

No I'm pulling it from the docker template in the app store... but I do have the original docker compose compreface group of 5 containers still running... is there some kind of conflict there? Should I delete all the original stuff

 

[corgan]

ahhh, ok, that's the Problem. As i can see in your screen, the compose DB is using the same folder. So there is a conflict.

If you don't need the compose app/db data anymore, then it makes sense to delete it. I did this today also.

[bigbangus]

OK but still after deleting all my templates and all my docker compose and all my appdata/compreface... I still have the same problem.

 

See log attached. However, it does work in external db mode w/postgres 11

 

 

 

compreface-gpu.txt

[corgan]

In the logs you find:

com.impossibl.postgres.jdbc.PGSQLSimpleException: Connection Error: connection timed out: /192.168.2.254:5432

 

That means, that the container trys to connect to the external DB. Can you please delete the IP from the external_DB field and try again?

[MarbleComa]

I tried removing the IP from the field, as well as leaving the field completely blank, but neither of those worked for me. So far I have only been able to get into the web ui by using an external DB. 

 

However, whenever I click on an Application, I just see a loading icon in the Services section. Meanwhile, the log is throwing errors, as you can see below. Ultimately, I get kicked back to the homepage of the web ui with the error message "Error during synchronization between servers: Unexpected end of file from server executing GET http://localhost:3000/status."

 

I'm using the Arcface model, and I attached the full log file below. 

 

compreface-gpu_error_log.txt

[corgan]

in your logs i found

clock source: unix
detected number of CPU cores: 24
current working directory: /app/ml

 

sure that you use the gpu version? Have you tried the normal version, without gpu?

[bigbangus]

So I'm all set using the external db and I think it's probably the way most users should go. For the record I'm on postgres13 with no issues. I'm using pgadmin4 to manage it (had to chmod 5050:5050 my pgadmin4 appdata on first start to get it to work).

 

@corgan I've kind of given up on the internal db but if you need me to continue testing for you I will.

[gshlomi]

Hi,

Just installed the CompreFace template, it seems the parameter of ExternalDB is being ignored so the application is crashing while trying to connect to a PostgreDB specified in the DB ip address variable...

[bigbangus]

5 hours ago, gshlomi said:

Hi,

Just installed the CompreFace template, it seems the parameter of ExternalDB is being ignored so the application is crashing while trying to connect to a PostgreDB specified in the DB ip address variable...

 

I see the same thing. From what I've tested with all versions (standard, and both gpu options) it will only work with an external db specified. On github the developper said it's compatible with postgres 11 or 13.

[bigbangus]

Is it normal to get this warning with the gpu version after processing your first image?

 

2021-12-20 11:37:09.749 WARN 127 --- [nio-8080-exec-3] o.n.l.cpu.nativecpu.CpuNDArrayFactory : *********************************** CPU Feature Check Warning ***********************************

2021-12-20 11:37:09.749 WARN 127 --- [nio-8080-exec-3] o.n.l.cpu.nativecpu.CpuNDArrayFactory : Warning: Initializing ND4J with Generic x86 binary on a CPU with AVX/AVX2 support

2021-12-20 11:37:09.749 WARN 127 --- [nio-8080-exec-3] o.n.l.cpu.nativecpu.CpuNDArrayFactory : Using ND4J with AVX/AVX2 will improve performance. See deeplearning4j.org/cpu for more details
2021-12-20 11:37:09.749 WARN 127 --- [nio-8080-exec-3] o.n.l.cpu.nativecpu.CpuNDArrayFactory : Or set environment variable ND4J_IGNORE_AVX=true to suppress this warning

2021-12-20 11:37:09.749 WARN 127 --- [nio-8080-exec-3] o.n.l.cpu.nativecpu.CpuNDArrayFactory : *************************************************************************************************

 

[Sublivion]

On 12/20/2021 at 10:48 AM, gshlomi said:

Hi,

Just installed the CompreFace template, it seems the parameter of ExternalDB is being ignored so the application is crashing while trying to connect to a PostgreDB specified in the DB ip address variable...

I'm seeing this same thing. Had to delete all those extra params to get the container to start again.