[Support] Crocs - Tube Archivist


Crocs

Recommended Posts

I am unable to view logs... I reinstalled the docker as a fresh install... and watched the logs...

 

The last thing that I saw was that it was confirming access to ES and then right after that there are two new lines that flash and the log window closes... and after the docker is stopped... I can not view the logs even after it's stopped... 

 

I browsed to the appdata path and there is a single file there... db.sqlite3 file... feels like there should be more files/data there after a fresh install... 

 

Feels like this docker is half of what it should be... like it was taken down... 

Link to comment
17 hours ago, rainfactor said:

The channels are being displayed under subscription and totally have videos

 

Can you be a bit clearer here? Do you mean that in TA channels, you see the subscribed channels and there are videos listed there? If you click on one can you watch it in TA?

 

That would mean the TA is working fine.

 

If that's not what you mean, I would take a look at the Scheduler Setup (in TA not the Docker template settings) and check what the rescan and start download frequency are set to.

 

image.png.e1be93896abe36b8e2b1ba22d5c82344.png

Link to comment
1 hour ago, aglyons said:

 

Can you be a bit clearer here? Do you mean that in TA channels, you see the subscribed channels and there are videos listed there? If you click on one can you watch it in TA?

 

That would mean the TA is working fine.

 

If that's not what you mean, I would take a look at the Scheduler Setup (in TA not the Docker template settings) and check what the rescan and start download frequency are set to.

 

image.png.e1be93896abe36b8e2b1ba22d5c82344.png

Apologize for the lack of info. Essentially after I added 2 subscriptions i have tesdrove it and went to the downloads tab and hit rescan subscriptions. I was under the impression it would be scanning the 2 channels I have added and presented me with a list of of videos it found (regardless of the scheduler).  The rescan spins for a while but it stops and never finds anything.  I might have done something wrong. I have restarted the container(s) a couple of times and made sure there was no error in the logs

 

Thank you

Link to comment

Can you make a screen recording of what you are doing?

 

When you subscribe to a channel, the entire channel will be queued up to download. The queue will run on the scheduler settings, but it is limited to not connecting to YT to download more than once per hour. 

 

If you click on the Channels page, it should list all your subscribed channels. If you click on one of the channels, it should bring you to a page listing all the downloaded videos.

 

Are you saying that this is not happening?

 

The other thing that I should mention in the event that his makes a difference. There is a Chrome plugin for TA that will allow you to subscribe to channels and add individual videos to the download queue. This plugin also syncs the YT cookie with TA. It is this cookie sync that may be the difference and what might be stopping your install from downloading properly.

Link to comment
On 2/26/2023 at 11:28 AM, aglyons said:

Can you make a screen recording of what you are doing?

 

When you subscribe to a channel, the entire channel will be queued up to download. The queue will run on the scheduler settings, but it is limited to not connecting to YT to download more than once per hour. 

 

If you click on the Channels page, it should list all your subscribed channels. If you click on one of the channels, it should bring you to a page listing all the downloaded videos.

 

Are you saying that this is not happening?

 

The other thing that I should mention in the event that his makes a difference. There is a Chrome plugin for TA that will allow you to subscribe to channels and add individual videos to the download queue. This plugin also syncs the YT cookie with TA. It is this cookie sync that may be the difference and what might be stopping your install from downloading properly.

I added 2 random channels and if I click on any of them it says there are no videos but I tried with a lot of other channels.

Rescan spins and spins but never finds anything.

 

 

SCR-20230227-ghc.png

SCR-20230227-gi5.png

SCR-20230227-gh2.png

Link to comment

Let's get the immediate things out of the way first.

 

1. You installed TubeArchivist-ES

2. You've installed TubeArchivist-Redis (using redis/redis-stack-server:latest image)

3. You've added the 'path.repo' variable to the Redis template and pointed it to the right user share for ES snapshots

4. You've configured the right paths in the TA template such as ES, Redis and where it's going to store DL's

 

Did you install the Chrome Add-on and check off 'sync cookies'

image.png.0f1efbae9e6e93bb48c0e7760aa1099e.png

 

Link to comment

I moved TA to my Docker network with a dedicated NIC. I also moved ES and Redis to the same network each with dedicated IP's. I updated the TA config so it can connect to those services and then changed the port in TA from 8000 to 80.

 

But it won't take. It still shows port 8000.

 

I don't see any error messages in logs referring to that.

 

Everything was working fine prior and still does aside form the port issue.

 

image.thumb.png.ccea71b77320e1b69e00f4248cf37e6d.png

 

image.png.2c33926cb45c60bf13e66654ad61d8fa.png

Edited by aglyons
Link to comment
  • 1 month later...
On 12/8/2022 at 10:33 PM, msherman83 said:

Hello,

Same issue as the others above on a fresh install. It seems for some reason you need to change the ./es folder to 777 to get the es container to write to it.

 

Access the Unraid CLI and enter:

chmod -R 777 /mnt/user/appdata/TubeArchivist/es

 

I noticed a large java file permissions error in the ES container logs the first few times i tried to start it. It would die right away, so you couldn't login to it.

 

 

This fixed my issue of both the TubeArchivist and TubeArchivist-ES containers from not starting. THANK YOU SO MUCH.

Link to comment

periodically, using the file manager plugin, select the appdata share and reset the ownership to 'nobody' and the permissions to 'read/write' for all three levels.

 

This is the default for Unraids built in permissions system to function. Some Docker containers either don't have explicit settings for GUID and PUID as well as UMASK support. As such, they write files with internal container permissions and it messes with Unraid on the host side.

 

I wish there was an automation plugin that could watch for this happenening and correect it in real-time. All my searches have led to people telling me to script it myself. There is a plugin that is specifically for this purpose, "Docker Safe New Perms" in the tools section, top row. But it is a manual process and it hits all shares across the whole server. Probably smart to do now and then as well.

Edited by aglyons
Link to comment

So, being that I am not a scripter, and that ChatGPT is so popular and awesome. I had ChatGPT write a bash script!

 

Perhaps someone with some more coding experience can review it and make sure this would work.

 

#!/bin/bash

# Function to check ownership and permissions of files and directories
check_obj() {
    # Check ownership
    owner=$(stat -c '%U' "$1")
    if [[ "$owner" != "nobody" ]]; then
        # Set ownership to nobody
        chown nobody "$1"
    fi

    # Check permissions
    if [[ -d "$1" ]]; then
        # Directory
        perms=$(stat -c '%a' "$1")
        if [[ "$perms" != "777" ]]; then
            # Set permissions to 'drwxrwxrwx'
            chmod 777 "$1"
        fi
    else
        # File
        perms=$(stat -c '%a' "$1")
        if [[ "$perms" != "666" ]]; then
            # Set permissions to '-rw-rw-rw-'
            chmod 666 "$1"
        fi
    fi
}

# Watch for changes to files and directories
inotifywait -m -e -r create,delete,modify,move,attrib /path/to/watch |
while read path action file; do
    # Check ownership and permissions of changed object
    check_obj "$path/$file"
done

 

With the following instructions:

 

Quote

 

Save this script to a file, e.g. watcher.sh, and make it executable with chmod +x watcher.sh.

Replace /path/to/watch with the path to the directory you want to watch. When you run the script, it will monitor the directory for any changes and check the ownership and permissions of any changed files or directories. If any ownership or permission issues are found, the script will set them appropriately.

 

 

Edited by aglyons
Link to comment
  • 2 weeks later...
On 2/21/2023 at 5:57 AM, mathomas3 said:

for anyone everyone else... there is a new ES setting that needs to be manually done... here it is

image.png.4e508b8162a0fe9fe248d5dd1f75a3f8.png

This isn't working for me, the docker just crashes. I am not sure /usr/share is an actual unraid share, maybe I need to move it to appdata?

 

Hmm - just tried to update the path and the CA apps is a variable, not a path. I guess I will wait to see what people say. 

 

 

FIXED - had to run new permissions on es appdata folder

 

 

Screenshot 2023-05-02 084522.jpg

Edited by kri kri
Fixed
Link to comment
  • 2 weeks later...
On 7/31/2022 at 1:28 AM, Vyressi said:

So in order to get this to work I needed to:

 

  1. manually change the permissions on /mnt/user/appdata/TubeArchivist/es to get elastisearch to properly load
  2. Changing the permissions wasn't enough so I also needed to add PID and GID environment variables to the elastisearch container.
  3. install their version of redis as it would not connect my already installed and functioning version of redis
  4. add a TA_HOST environment variable to the main archive, which I only discovered I needed to do after manually probing the log files from the terminal, as the log would close instantly from the unraid docker interface
  5. Container loaded fine, and would not let me log in
  6. manually specify a non-default TA_USER and TA_PASSWORD, still wouldn't let me log in.
  7. go into the container terminal once it stopped closing immediately and hunt down a python script in /app called manage.py
  8. execute 
    python manage.py createsuperuser
    python manage.py drf_create_token %myusername%

     

  9. finally able to log in.

Several of these things are because the compose that's rolled into the CA repository is either missing documentation, missing variables, missing options, or does not handle bad startups gracefully. 

Easily the most helpful post in this forum so far.

Edited by ikyn
  • Like 1
Link to comment
31 minutes ago, mathomas3 said:

at this point... I think its fair to say that the docker version is dead... I am now two version old vs the new release and docker still shows that there isnt an update... 

 

I suggest perhaps rolling a VM to replace the docker... I plan on doing this soon

Sorry, but what? 

 

The container updates fine. It's up to date

 The "docker version" is the official version.

 

It sounds to me that perhaps you haven't applied the docker update patch that came out at the server level and you're not seeing "update unavailable" instead? 

 

Ps. You could still hit force update. 

Link to comment
39 minutes ago, CorneliousJD said:

Sorry, but what? 

 

The container updates fine. It's up to date

 The "docker version" is the official version.

 

It sounds to me that perhaps you haven't applied the docker update patch that came out at the server level and you're not seeing "update unavailable" instead? 

 

Ps. You could still hit force update. 

server level update? I am on the latest unraid version... the current status of this docker has been the same for 4-6 months now... 

 

image.png.87d363776af85b0bf4e71750b2a72e68.png

 

I think ES was in the same status until recently... This docker has been finicky recently thus I didnt want to change things... now that I am 2 versions behind I am willing to make changes... also I have 100+ unfinished downloads... so... -_-

  • Confused 1
Link to comment
8 hours ago, mathomas3 said:

server level update? I am on the latest unraid version... the current status of this docker has been the same for 4-6 months now... 

 

image.png.87d363776af85b0bf4e71750b2a72e68.png

 

I think ES was in the same status until recently... This docker has been finicky recently thus I didnt want to change things... now that I am 2 versions behind I am willing to make changes... also I have 100+ unfinished downloads... so... -_-

I'm mobile but check the apps tab and type on "docker patch" 

 

There's a patch to fix that orange "not available" update. It's some issue with the way unRAID checks for docker updates. 

 

That "not available" means it can't check if there's an update, not that one *isn't* available. 

 

You probably have a lot of updates you are behind on, but install that patch, check for updates and let them ride. 

Link to comment
On 5/2/2023 at 9:45 AM, kri kri said:

This isn't working for me, the docker just crashes. I am not sure /usr/share is an actual unraid share, maybe I need to move it to appdata?

 

Hmm - just tried to update the path and the CA apps is a variable, not a path. I guess I will wait to see what people say. 

 

 

FIXED - had to run new permissions on es appdata folder

 

 

Screenshot 2023-05-02 084522.jpg

 

That's interesting. How long have you had your TA installed for?

 

I installed mine about 3 months ago and that was already part of the community template, if I recall correctly. I already had that variable in place since installing. Maybe that's why I haven't been seeing these errors crop up. I also installed the Docker patch a while ago when I noticed the Unknown Version issue pop up.

 

Has there been any movement on a docker image that is self contained? I read somewhere a while back that this was something that was being looked into to simplify the TA setup process. One container to rule them all!

Link to comment
  • 4 weeks later...
  • 2 weeks later...
On 6/10/2023 at 10:11 AM, zolointo said:

I'm stuck on path.repo permissions issues and am totally clueless as to what to do.

 

I've chown'd everything to nobody:users, chmod'd to 777 across the board, but it still generates a permissions issue as seen in logs.png.

 

What the heck am I doing wrong?

1_elasticsearch.png

2_data.png

3_snapshots.png

logs.png

What are the GID and PID both container is running under? 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.