Official Repos:

https://hub.docker.com/r/archiveteam/warrior-dockerfile/

https://github.com/ArchiveTeam/warrior-dockerfile

Hi, Thanks for that docker

 

However, no projects works :

 

2021-12-17 14:37:10,494 - seesaw.warrior - DEBUG - Update warrior hq.
2021-12-17 14:37:10,494 - seesaw.warrior - DEBUG - Warrior ID ''.
2021-12-17 14:37:10,595 - seesaw.warrior - DEBUG - Select project reddit
2021-12-17 14:37:10,595 - seesaw.warrior - DEBUG - Start selected project reddit (reinstall=False)
2021-12-17 14:37:10,595 - seesaw.warrior - DEBUG - Install project reddit
2021-12-17 14:37:10,601 - seesaw.warrior - DEBUG - git pull from https://github.com/ArchiveTeam/reddit-grab
2021-12-17 14:37:10,888 - seesaw.warrior - DEBUG - git operation: Already up to date.

2021-12-17 14:37:10,889 - seesaw.warrior - DEBUG - Install complete Already up to date.

2021-12-17 14:37:10,889 - seesaw.warrior - DEBUG - Result of the install process: True
2021-12-17 14:37:10,889 - seesaw.warrior - DEBUG - Clone project reddit /home/warrior/projects/reddit
2021-12-17 14:37:10,894 - seesaw.warrior - DEBUG - Cloning version 8a3f8cd
2021-12-17 14:37:10,895 - seesaw.warrior - DEBUG - Load pipeline /home/warrior/data/projects/reddit-8a3f8cd/pipeline.py
2021-12-17 14:37:10,895 - seesaw.warrior - DEBUG - Pipeline has been read. Begin ConfigValue collection
2021-12-17 14:37:10,895 - seesaw.warrior - DEBUG - Executing pipeline
2021-12-17 14:37:10,955 - seesaw.warrior - ERROR - Error loading pipeline
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 736, in start_selected_project
    (project, pipeline, config_values) = self.load_pipeline(
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 686, in load_pipeline
    exec(pipeline_str, local_context, global_context)
  File "<string>", line 54, in <module>
Exception: No usable Wget+At found.
2021-12-17 14:37:10,956 - seesaw.warrior - WARNING - Project reddit did not install correctly and we're ignoring this problem.
2021-12-17 14:38:12,219 - seesaw.warrior - DEBUG - Select project urls
2021-12-17 14:38:12,220 - seesaw.warrior - DEBUG - Start selected project urls (reinstall=False)
2021-12-17 14:38:12,220 - seesaw.warrior - DEBUG - Install project urls
2021-12-17 14:38:12,220 - seesaw.warrior - DEBUG - git clone
2021-12-17 14:38:13,077 - seesaw.warrior - DEBUG - git operation: Cloning into '/home/warrior/projects/urls'...

2021-12-17 14:38:13,077 - seesaw.warrior - DEBUG - Install complete Cloning into '/home/warrior/projects/urls'...

2021-12-17 14:38:13,078 - seesaw.warrior - DEBUG - Result of the install process: True
2021-12-17 14:38:13,078 - seesaw.warrior - DEBUG - Clone project urls /home/warrior/projects/urls
2021-12-17 14:38:13,083 - seesaw.warrior - DEBUG - Cloning version bf139f4
2021-12-17 14:38:13,098 - seesaw.warrior - DEBUG - Load pipeline /home/warrior/data/projects/urls-bf139f4/pipeline.py
2021-12-17 14:38:13,098 - seesaw.warrior - DEBUG - Pipeline has been read. Begin ConfigValue collection
2021-12-17 14:38:13,099 - seesaw.warrior - DEBUG - Executing pipeline
2021-12-17 14:38:13,135 - seesaw.warrior - ERROR - Error loading pipeline
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 736, in start_selected_project
    (project, pipeline, config_values) = self.load_pipeline(
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 686, in load_pipeline
    exec(pipeline_str, local_context, global_context)
  File "<string>", line 61, in <module>
Exception: No usable Wget+At found.
2021-12-17 14:38:13,136 - seesaw.warrior - WARNING - Project urls did not install correctly and we're ignoring this problem.
2021-12-17 14:38:27,628 - seesaw.warrior - DEBUG - Select project auto
2021-12-17 14:38:27,628 - seesaw.warrior - DEBUG - Update warrior hq.
2021-12-17 14:38:27,629 - seesaw.warrior - DEBUG - Warrior ID ''.
2021-12-17 14:38:27,737 - seesaw.warrior - DEBUG - Select project reddit
2021-12-17 14:38:27,738 - seesaw.warrior - DEBUG - Start selected project reddit (reinstall=False)
2021-12-17 14:38:27,738 - seesaw.warrior - DEBUG - Check project has update reddit
2021-12-17 14:38:27,743 - seesaw.warrior - DEBUG - git fetch
2021-12-17 14:38:28,038 - seesaw.warrior - DEBUG - False
2021-12-17 14:38:28,039 - seesaw.warrior - DEBUG - Clone project reddit /home/warrior/projects/reddit
2021-12-17 14:38:28,043 - seesaw.warrior - DEBUG - Cloning version 8a3f8cd
2021-12-17 14:38:28,043 - seesaw.warrior - DEBUG - Load pipeline /home/warrior/data/projects/reddit-8a3f8cd/pipeline.py
2021-12-17 14:38:28,044 - seesaw.warrior - DEBUG - Pipeline has been read. Begin ConfigValue collection
2021-12-17 14:38:28,044 - seesaw.warrior - DEBUG - Executing pipeline
2021-12-17 14:38:28,055 - seesaw.warrior - ERROR - Error loading pipeline
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 736, in start_selected_project
    (project, pipeline, config_values) = self.load_pipeline(
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 686, in load_pipeline
    exec(pipeline_str, local_context, global_context)
  File "<string>", line 54, in <module>
Exception: No usable Wget+At found.
2021-12-17 14:38:28,056 - seesaw.warrior - WARNING - Project reddit did not install correctly and we're ignoring this problem.

 

Getting the same error.

 

Edit:

Looks like the wrong base repo is being used. Switched over following the advice here now it's running.
https://github.com/ArchiveTeam/warrior-dockerfile/issues/62#issuecomment-980438747

Edited December 17, 20214 yr by TrikkStar

3 hours ago, TrikkStar said:

Getting the same error.

 

Edit:

Looks like the wrong base repo is being used. Switched over following the advice here now it's running.
https://github.com/ArchiveTeam/warrior-dockerfile/issues/62#issuecomment-980438747

Good catch!

 

I updated the template, you should be able to force recheck and get an update.

 

On 12/17/2021 at 8:12 PM, Crocs said:

Good catch!

 

I updated the template, you should be able to force recheck and get an update.

 

That works ! Thanks

Any clue if there's a way to define where it saves the temporarily archived data? I'd rather save the files to a folder on one of my drives instead of inside the Docker image disk, especially since there's no upper limit for how much storage the application uses when downloading files to archive.

I think the unraid template is wrong: The path which is called Appdata and contains a placeholder appdata path is actually being mounted to the docker.sock file. I doubt that this is intentional:

1 hour ago, SourceDiver42 said:

I think the unraid template is wrong: The path which is called Appdata and contains a placeholder appdata path is actually being mounted to the docker.sock file. I doubt that this is intentional:

 

Yup.  Obviously wrong

I setup this path after examining in the console for a minute. Seems to be working to save projects/data outside the docker image. Disclaimer, I've been running this merely for a few minutes only so far.

I did leave the docker.sock mount as defined.

Edited February 27, 20224 yr by GoChris

On 2/27/2022 at 11:42 AM, GoChris said:

I setup this path after examining in the console for a minute. Seems to be working to save projects/data outside the docker image. Disclaimer, I've been running this merely for a few minutes only so far.

I did leave the docker.sock mount as defined.

That's pretty much how I have the updated template. I used '/mnt/user/appdata/warrior/' to map it to the appdata share vs directly to to cache drive.

I am having an issue where every time the docker container is updated, it loses the config. So I have to keep entering a nickname and choosing a project. Here is my config (I have 5 instances of warriors and each warrior is mapped to its own appdata folder.

For anyone still struggling with this. I found that the container isn't setup to work in the typical "appdata" way and permission issues prevent warrior from saving the json config file.

 

My solution was to accept all the defaults of the container and to just pass "Extra Parameters" to configure warrior. This will preserve your settings but any in-progress work gets lost when the container is re-created.

 

My settings are to just use the preferred project and to max the concurrent items. I hope this helps someone.

 

Set "Extra Parameters" to:

-e DOWNLOADER='YourUsername' -e SELECTED_PROJECT='auto' -e CONCURRENT_ITEMS='6' 

Edited October 21, 20223 yr by RackIt

On 2/27/2022 at 6:42 PM, GoChris said:

I setup this path after examining in the console for a minute. Seems to be working to save projects/data outside the docker image. Disclaimer, I've been running this merely for a few minutes only so far.

I did leave the docker.sock mount as defined.

if i try this, i get an error

2022-12-21 11:02:23,860 - seesaw.warrior - ERROR - Error loading pipeline
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 736, in start_selected_project
    (project, pipeline, config_values) = self.load_pipeline(
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 674, in load_pipeline
    with open(pipeline_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/warrior/data/projects/telegram-e0a2119/pipeline.py'
2022-12-21 11:02:23,860 - seesaw.warrior - WARNING - Project telegram did not install correctly and we're ignoring this problem.

 

Edited December 21, 20223 yr by fiergna

Sorry I stopped using this a while ago.

On 12/21/2022 at 10:32 AM, fiergna said:

if i try this, i get an error

2022-12-21 11:02:23,860 - seesaw.warrior - ERROR - Error loading pipeline
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 736, in start_selected_project
    (project, pipeline, config_values) = self.load_pipeline(
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 674, in load_pipeline
    with open(pipeline_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/warrior/data/projects/telegram-e0a2119/pipeline.py'
2022-12-21 11:02:23,860 - seesaw.warrior - WARNING - Project telegram did not install correctly and we're ignoring this problem.

 

 

Same error for me. Did you success in solving it?

Just now, Thomas K said:

 

Same error for me. Did you success in solving it?

I long time ago I did, read a few posts up. I don't use this anymore though.

On 12/21/2022 at 10:32 AM, fiergna said:

if i try this, i get an error

2022-12-21 11:02:23,860 - seesaw.warrior - ERROR - Error loading pipeline
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 736, in start_selected_project
    (project, pipeline, config_values) = self.load_pipeline(
  File "/usr/local/lib/python3.9/site-packages/seesaw/warrior.py", line 674, in load_pipeline
    with open(pipeline_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/warrior/data/projects/telegram-e0a2119/pipeline.py'
2022-12-21 11:02:23,860 - seesaw.warrior - WARNING - Project telegram did not install correctly and we're ignoring this problem.

 

 

Solved it, its an permission issue on the host path.
For me /mnt/cachepool/archiveteam-warrior-data needed the right permissions.

On 10/21/2022 at 7:10 PM, RackIt said:

-e DOWNLOADER='YourUsername' -e SELECTED_PROJECT='auto' -e CONCURRENT_ITEMS='6' 

Will there be a fix in the container so that this doesn't have to be added. Took me ages to figure out that I had to add this to extra parameters. Could be useful to add the info into the docker container.

+1 to fixing this in the app template.

 

It's trivial to add these variables to the config so it'd be nice if it was built in so the jobs automatically start.

FYI anyone that comes across this, I got an updated version of this app up in CA now.  Still called "ArchiveTeam-Warrior" but by me, darknavi.  It has a few extra params built in.

 

New support thread:

 

 

Unfortunately, I had to remove this due to the downloaded files being saved directly into the docker IMG file. I noticed it using up a larger and larger amount of storage and thus removed it, freeing up 25GB of storage. Temporary downloaded stuff should go into ram, not directly into docker IMG file.