[Support] Crocs - Tube Archivist


Recommended Posts

I have other Redis and Elasticsearch dockers on my machine and it seems like they are connecting but Im getting an Internal Server Error when trying to use the web gui. Posting logs soon. Just not sure if I should exclude anything from the logs, should I?

Link to comment
6 hours ago, Thorsten said:

 Hi,

what template did you use for elasicsearch and reddist. Are these templates also available? That would simplify the installtion ...

Regards

Thorsten

For anyone else that is having trouble, they're inside of the install instructions on CA

thdiJAZ.png

LjsfhKL.png

Link to comment
3 hours ago, Ezrael said:

I just have one question (right now): What is the elasticsearch repo? It's longer than the text box and I can't copy it.
I can't see that repo on Github/Docker either...

 

Thank you!

Sorry about that, its "docker.elastic.co/elasticsearch/elasticsearch:7.14.1"

Link to comment
1 hour ago, muppie said:

I have other Redis and Elasticsearch dockers on my machine and it seems like they are connecting but Im getting an Internal Server Error when trying to use the web gui. Posting logs soon. Just not sure if I should exclude anything from the logs, should I?

Make sure you have the "http://" in front of your IP for the ElasticSearch variable in the template.

 

It will throw errors if you don't.

Link to comment

I have http in front but I’m still getting a Internal Server Error.

 

Do I have the wrong redis version? I don’t have any special for redis json.

Logs here:

 

Quote

ErrorWarningSystemArrayLogin


redis.exceptions.ResponseError: unknown command `JSON.GET`, with args beginning with: `config`,
[uWSGI] getting INI configuration from uwsgi.ini
*** Starting uWSGI 2.0.19.1 (64bit) on [Fri Sep 24 22:13:05 2021] ***
compiled with version: 10.2.1 20210110 on 08 September 2021 09:13:02
os: Linux-5.10.28-Unraid #1 SMP Wed Apr 7 08:23:18 PDT 2021
nodename: 4742eae369e2
machine: x86_64
clock source: unix
detected number of CPU cores: 8
current working directory: /app
writing pidfile to /tmp/project-master.pid
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
your processes number limit is 191529
your memory page size is 4096 bytes
detected max file descriptor number: 40960
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address :8080 fd 3
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
Python version: 3.9.7 (default, Sep 3 2021, 02:02:37) [GCC 10.2.1 20210110]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x557c139c7800
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145808 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
Traceback (most recent call last):
File "/app/./config/wsgi.py", line 16, in <module>
application = get_wsgi_application()
File "/usr/local/lib/python3.9/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
django.setup(set_prefix=False)
File "/usr/local/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 91, in populate
app_config = AppConfig.create(entry)
File "/usr/local/lib/python3.9/site-packages/django/apps/config.py", line 212, in create
mod = import_module(mod_path)
File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/app/./home/__init__.py", line 7, in <module>
from home.src.index_management import index_check
File "/app/./home/src/index_management.py", line 159, in <module>
class ElasticIndex:
File "/app/./home/src/index_management.py", line 164, in ElasticIndex
CONFIG = AppConfig().config
File "/app/./home/src/config.py", line 18, in __init__
self.config = self.get_config()
File "/app/./home/src/config.py", line 22, in get_config
config = self.get_config_redis()
File "/app/./home/src/config.py", line 54, in get_config_redis
config = get_message("config")
File "/app/./home/src/helper.py", line 80, in get_message
reply = redis_connection.execute_command("JSON.GET", key)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 901, in execute_command
return self.parse_response(conn, command_name, **options)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 915, in parse_response
response = connection.read_response()
File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 756, in read_response
raise response
redis.exceptions.ResponseError: unknown command `JSON.GET`, with args beginning with: `config`,
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 13)
spawned uWSGI worker 1 (pid: 23, cores: 1)
Usage: celery [OPTIONS] COMMAND [ARGS]...

Error: Invalid value for '-A' / '--app':
Unable to load celery application.
While trying to load the module home.tasks the following error occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/celery/bin/celery.py", line 53, in convert
return find_app(value)
File "/usr/local/lib/python3.9/site-packages/celery/app/utils.py", line 384, in find_app
sym = symbol_by_name(app, imp=imp)
File "/usr/local/lib/python3.9/site-packages/kombu/utils/imports.py", line 56, in symbol_by_name
module = imp(module_name, package=package, **kwargs)
File "/usr/local/lib/python3.9/site-packages/celery/utils/imports.py", line 100, in import_from_cwd
return imp(module, package=package)
File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/app/home/__init__.py", line 7, in <module>
from home.src.index_management import index_check
File "/app/home/src/index_management.py", line 159, in <module>
class ElasticIndex:
File "/app/home/src/index_management.py", line 164, in ElasticIndex
CONFIG = AppConfig().config
File "/app/home/src/config.py", line 18, in __init__
self.config = self.get_config()
File "/app/home/src/config.py", line 22, in get_config
config = self.get_config_redis()
File "/app/home/src/config.py", line 54, in get_config_redis
config = get_message("config")
File "/app/home/src/helper.py", line 80, in get_message
reply = redis_connection.execute_command("JSON.GET", key)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 901, in execute_command
return self.parse_response(conn, command_name, **options)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 915, in parse_response
response = connection.read_response()
File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 756, in read_response
raise response
redis.exceptions.ResponseError: unknown command `JSON.GET`, with args beginning with: `config`,

--- no python application found, check your startup logs for errors ---
[pid: 23|app: -1|req: -1/1] 10.0.8.2 () {40 vars in 650 bytes} [Fri Sep 24 20:13:09 2021] GET / => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
--- no python application found, check your startup logs for errors ---
[pid: 23|app: -1|req: -1/2] 10.0.8.2 () {40 vars in 650 bytes} [Fri Sep 24 20:13:11 2021] GET / => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
{
"name" : "87c285cb6eaf",
"cluster_name" : "docker-cluster",
"cluster_uuid" : "iaxRVLu4RTyqKTB-TN6A4Q",
"version" : {
"number" : "7.10.2",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "747e1cc71def077253878a59143c1f785afa92b9",
"build_date" : "2021-01-13T00:42:12.435326Z",
"build_snapshot" : false,
"lucene_version" : "8.7.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
Traceback (most recent call last):
File "/app/manage.py", line 23, in <module>
main()
File "/app/manage.py", line 19, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 395, in execute
django.setup()
File "/usr/local/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 91, in populate
app_config = AppConfig.create(entry)
File "/usr/local/lib/python3.9/site-packages/django/apps/config.py", line 212, in create
mod = import_module(mod_path)
File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/app/home/__init__.py", line 7, in <module>
from home.src.index_management import index_check
File "/app/home/src/index_management.py", line 159, in <module>
class ElasticIndex:
File "/app/home/src/index_management.py", line 164, in ElasticIndex
CONFIG = AppConfig().config
File "/app/home/src/config.py", line 18, in __init__
self.config = self.get_config()
File "/app/home/src/config.py", line 22, in get_config
config = self.get_config_redis()
File "/app/home/src/config.py", line 54, in get_config_redis
config = get_message("config")
File "/app/home/src/helper.py", line 80, in get_message
reply = redis_connection.execute_command("JSON.GET", key)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 901, in execute_command
return self.parse_response(conn, command_name, **options)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 915, in parse_response
response = connection.read_response()
File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 756, in read_response
raise response
redis.exceptions.ResponseError: unknown command `JSON.GET`, with args beginning with: `config`,
Traceback (most recent call last):
File "/app/manage.py", line 23, in <module>
main()
File "/app/manage.py", line 19, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 395, in execute
django.setup()
File "/usr/local/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 91, in populate
app_config = AppConfig.create(entry)
File "/usr/local/lib/python3.9/site-packages/django/apps/config.py", line 212, in create
mod = import_module(mod_path)
File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/app/home/__init__.py", line 7, in <module>
from home.src.index_management import index_check
File "/app/home/src/index_management.py", line 159, in <module>
class ElasticIndex:
File "/app/home/src/index_management.py", line 164, in ElasticIndex
CONFIG = AppConfig().config
File "/app/home/src/config.py", line 18, in __init__
self.config = self.get_config()
File "/app/home/src/config.py", line 22, in get_config
config = self.get_config_redis()
File "/app/home/src/config.py", line 54, in get_config_redis
config = get_message("config")
File "/app/home/src/helper.py", line 80, in get_message
reply = redis_connection.execute_command("JSON.GET", key)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 901, in execute_command
return self.parse_response(conn, command_name, **options)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 915, in parse_response
response = connection.read_response()
File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 756, in read_response
raise response
redis.exceptions.ResponseError: unknown command `JSON.GET`, with args beginning with: `config`,
[uWSGI] getting INI configuration from uwsgi.ini
*** Starting uWSGI 2.0.19.1 (64bit) on [Sat Sep 25 19:34:29 2021] ***
compiled with version: 10.2.1 20210110 on 08 September 2021 09:13:02
os: Linux-5.10.28-Unraid #1 SMP Wed Apr 7 08:23:18 PDT 2021
nodename: 4742eae369e2
machine: x86_64
clock source: unix
detected number of CPU cores: 8
current working directory: /app
writing pidfile to /tmp/project-master.pid
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
your processes number limit is 191529
your memory page size is 4096 bytes
detected max file descriptor number: 40960
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address :8080 fd 3
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
Python version: 3.9.7 (default, Sep 3 2021, 02:02:37) [GCC 10.2.1 20210110]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x5615eec8f800
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145808 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
Traceback (most recent call last):
File "/app/./config/wsgi.py", line 16, in <module>
application = get_wsgi_application()
File "/usr/local/lib/python3.9/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
django.setup(set_prefix=False)
File "/usr/local/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 91, in populate
app_config = AppConfig.create(entry)
File "/usr/local/lib/python3.9/site-packages/django/apps/config.py", line 212, in create
mod = import_module(mod_path)
File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/app/./home/__init__.py", line 7, in <module>
from home.src.index_management import index_check
File "/app/./home/src/index_management.py", line 159, in <module>
class ElasticIndex:
File "/app/./home/src/index_management.py", line 164, in ElasticIndex
CONFIG = AppConfig().config
File "/app/./home/src/config.py", line 18, in __init__
self.config = self.get_config()
File "/app/./home/src/config.py", line 22, in get_config
config = self.get_config_redis()
File "/app/./home/src/config.py", line 54, in get_config_redis
config = get_message("config")
File "/app/./home/src/helper.py", line 80, in get_message
reply = redis_connection.execute_command("JSON.GET", key)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 901, in execute_command
return self.parse_response(conn, command_name, **options)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 915, in parse_response
response = connection.read_response()
File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 756, in read_response
raise response
redis.exceptions.ResponseError: unknown command `JSON.GET`, with args beginning with: `config`,
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 12)
spawned uWSGI worker 1 (pid: 22, cores: 1)
Usage: celery [OPTIONS] COMMAND [ARGS]...

Error: Invalid value for '-A' / '--app':
Unable to load celery application.
While trying to load the module home.tasks the following error occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/celery/bin/celery.py", line 53, in convert
return find_app(value)
File "/usr/local/lib/python3.9/site-packages/celery/app/utils.py", line 384, in find_app
sym = symbol_by_name(app, imp=imp)
File "/usr/local/lib/python3.9/site-packages/kombu/utils/imports.py", line 56, in symbol_by_name
module = imp(module_name, package=package, **kwargs)
File "/usr/local/lib/python3.9/site-packages/celery/utils/imports.py", line 100, in import_from_cwd
return imp(module, package=package)
File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/app/home/__init__.py", line 7, in <module>
from home.src.index_management import index_check
File "/app/home/src/index_management.py", line 159, in <module>
class ElasticIndex:
File "/app/home/src/index_management.py", line 164, in ElasticIndex
CONFIG = AppConfig().config
File "/app/home/src/config.py", line 18, in __init__
self.config = self.get_config()
File "/app/home/src/config.py", line 22, in get_config
config = self.get_config_redis()
File "/app/home/src/config.py", line 54, in get_config_redis
config = get_message("config")
File "/app/home/src/helper.py", line 80, in get_message
reply = redis_connection.execute_command("JSON.GET", key)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 901, in execute_command
return self.parse_response(conn, command_name, **options)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 915, in parse_response
response = connection.read_response()
File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 756, in read_response
raise response
redis.exceptions.ResponseError: unknown command `JSON.GET`, with args beginning with: `config`,

 

Link to comment
1 hour ago, muppie said:

I have http in front but I’m still getting a Internal Server Error.

 

Do I have the wrong redis version? I don’t have any special for redis json.

Logs here:

 

 

Check the screenshots above, do you have Redis and ES mapped to the correct volumes inside of the TubeArchivsit directory?

Link to comment
11 hours ago, Crocs said:

Check the screenshots above, do you have Redis and ES mapped to the correct volumes inside of the TubeArchivsit directory?

I’ve filled out the redis IP and elasticsearch IP in the template and I’ve tried to review the screenshots again and to me they look the same. Attaching it here:

 

I cannot spot any errors. If I add port to the redis IP it is duplicated in the logs (as IP:6379:6379) so I removed it.

DCA796EC-E861-4C44-A1A2-D9266780E86B.jpeg

Edited by muppie
Link to comment
20 hours ago, muppie said:

I’ve filled out the redis IP and elasticsearch IP in the template and I’ve tried to review the screenshots again and to me they look the same. Attaching it here:

 

I cannot spot any errors. If I add port to the redis IP it is duplicated in the logs (as IP:6379:6379) so I removed it.

DCA796EC-E861-4C44-A1A2-D9266780E86B.jpeg

I'm not seeing anything obvious either. The only thing that I can think is that you're using different IP addresses for each container. Not being the dev of the project, I can only assume that this might be your issue.

Link to comment

Anyone got subtiltes working? It accepts my parameters, but alas, no subs for me.
I'm using

bestvideo[height<=1080]+bestaudio/best[height<=1080] --sub-lan en --write-auto-sub --convert-subs=srt 

currently. That does not work though.

 

Cheers

Link to comment

I already have Elasticsearch and Redis on my server for other reasons. I see no easy way to point Tubearchivist to where they are at. I tried symlinking but that isn't working. It would be nice if I could set the redis and ES directories as environment variables within the container.

Link to comment

I posted mu question and logs in the github for the application and it appears that you need a special redis version that support json. As far as I can see, there is no template for the correct redis so we need to install it manuallly

Link to comment
12 hours ago, muppie said:

I posted mu question and logs in the github for the application and it appears that you need a special redis version that support json. As far as I can see, there is no template for the correct redis so we need to install it manuallly

I posted screenshots above for the correct templates needed.

Link to comment
  • 1 month later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.