[Support] auto-yt-dl

[LarsW]

This is the support thread for the auto-yt-dl docker container and template.

 

This docker is used to automatically download new Videos of specific YouTube Channels. It features a Web Gui to add and remove Channels from your watch list.

 

This docker utilizes pytube. When add an Channel it will fetch all Video URLs of that Channel and save it. Every 15 minutes it uses the already saved URL to check if a new video is accessible. It not find new videos immediately, but usually it should take no longer than 30 minutes.

 

It might be obvious that this is my first docker and every tricks & tips are greatly appreciated!

 

If you find any issues or have any questions please let me know.

 

Edit: There seems to be a common misconception that the app should download the whole channel when added. This is not the case as the app will only download new videos. It is however possible to download all videos by deleting the content of the <channelName>.txt in the appdata folder.

 

Edited April 11, 2022 by LarsW
Clarification of the usage of this app; Spelling

[rikki]

work well one thing i wish it had was saveing video in to youtube channel name folders

[LarsW]

4 hours ago, rikki said:

work well one thing i wish it had was saveing video in to youtube channel name folders

Just pushed an update which allows just that

[rikki]

thank very much will give it a tray

[wildfire305]

I get an error after installing it. I pasted the log below.

 

 

 

Starting loop....
Checking for new Videos
Traceback (most recent call last):
File "main.py", line 8, in <module>
pytubDef.loop()
File "/app/pytubDef/__init__.py", line 141, in loop
checkForNewURL(channelArray[m])
File "/app/pytubDef/__init__.py", line 147, in checkForNewURL
for n in range(selectedChannel.video_urls.__len__()):
File "/usr/local/lib/python3.8/site-packages/pytube/helpers.py", line 89, in __len__
self.generate_all()
File "/usr/local/lib/python3.8/site-packages/pytube/helpers.py", line 105, in generate_all
next_item = next(self.gen)
File "/usr/local/lib/python3.8/site-packages/pytube/contrib/playlist.py", line 281, in url_generator
for page in self._paginate():
File "/usr/local/lib/python3.8/site-packages/pytube/contrib/playlist.py", line 118, in _paginate
json.dumps(extract.initial_data(self.html))
File "/usr/local/lib/python3.8/site-packages/pytube/contrib/channel.py", line 78, in html
self._html = request.get(self.videos_url)
File "/usr/local/lib/python3.8/site-packages/pytube/request.py", line 53, in get
response = _execute_request(url, headers=extra_headers, timeout=timeout)
File "/usr/local/lib/python3.8/site-packages/pytube/request.py", line 37, in _execute_request
return urlopen(request, timeout=timeout) # nosec
File "/usr/local/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/local/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/local/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/local/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/local/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

[rikki]

working well still for me one outher thing if possable any way to show something is downloading in the web ui

[LarsW]

21 hours ago, wildfire305 said:

I get an error after installing it. I pasted the log below.

 

 

 

Starting loop....
Checking for new Videos
Traceback (most recent call last):
File "main.py", line 8, in <module>
pytubDef.loop()
File "/app/pytubDef/__init__.py", line 141, in loop
checkForNewURL(channelArray[m])
File "/app/pytubDef/__init__.py", line 147, in checkForNewURL
for n in range(selectedChannel.video_urls.__len__()):
File "/usr/local/lib/python3.8/site-packages/pytube/helpers.py", line 89, in __len__
self.generate_all()
File "/usr/local/lib/python3.8/site-packages/pytube/helpers.py", line 105, in generate_all
next_item = next(self.gen)
File "/usr/local/lib/python3.8/site-packages/pytube/contrib/playlist.py", line 281, in url_generator
for page in self._paginate():
File "/usr/local/lib/python3.8/site-packages/pytube/contrib/playlist.py", line 118, in _paginate
json.dumps(extract.initial_data(self.html))
File "/usr/local/lib/python3.8/site-packages/pytube/contrib/channel.py", line 78, in html
self._html = request.get(self.videos_url)
File "/usr/local/lib/python3.8/site-packages/pytube/request.py", line 53, in get
response = _execute_request(url, headers=extra_headers, timeout=timeout)
File "/usr/local/lib/python3.8/site-packages/pytube/request.py", line 37, in _execute_request
return urlopen(request, timeout=timeout) # nosec
File "/usr/local/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/local/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/local/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/local/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/local/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

Does it happen immediately after installation? Did you try to reinstall it? Is there anything special about your system? I will take a look at it as soon as possible but at the moment I don’t have much time.

[LarsW]

1 hour ago, rikki said:

working well still for me one outher thing if possable any way to show something is downloading in the web ui

That’s a great idea but at the moment I don’t have time. I will take a look at this after addressing the issue above. 

[wildfire305]

12 hours ago, LarsW said:

Does it happen immediately after installation? Did you try to reinstall it? Is there anything special about your system? I will take a look at it as soon as possible but at the moment I don’t have much time.

I don't have anything special that I know of. 12 core xeon, 64gb ecc, 3x 3tb array + 2x 3tb parity, 2x 256gb ssd cache in raid 0. I'm a docker noob, but I set the download directory while installing and that was first run (I think). I deleted the app using the unraid gui and reinstalled it and got the same result.

[wildfire305]

I have NOT tried "turning it off and back on again". Should I?

[rikki]

im running AMD Turion™ II Neo N54L Dual-Core @ 2200 MHz if that help narrow down the problem if you want aly log let me know

[LarsW]

On 9/16/2021 at 12:06 PM, wildfire305 said:

I have NOT tried "turning it off and back on again". Should I?

The intention was to know if the bug is reproducible. I was not able to reproduce the bug. Did you try to go to the appdata/auto-yt-dl folder and empty the monitoredChannel.txt (also delete the pre-written text) ?

 

On 9/16/2021 at 11:34 PM, rikki said:

im running AMD Turion™ II Neo N54L Dual-Core @ 2200 MHz if that help narrow down the problem if you want aly log let me know

 

Do you also have this issue?

[rikki]

no it runing ok for me might be worth removing it using console commands as folders left behind even if you remove using the gui work on outher dockers for me

 

[wildfire305]

I will get back to you on this. I deleted the docker and removed the folder from appdata and reinstalled it and it cleared the error. I got to the webpage ui for the app and I couldn't get the syntax correct to download my channel. I will post the results after I finish dealing with a permission issue.

[MrUNoWho]

I am having an issue where even though I specified a directory in the setup as to where to save downloads, it always saves to a share of 'unRaidLars.'  If I clear out that share and delete it, the app will not function and if I restart it, it will recreate that share.

[Squid]

 

Did you change the path here?  Based upon how the template is setup, it saves to /app/Downloads and you map that path accordingly

[MrUNoWho]

Here is what my settings say.

 

 

[Squid]

Show your docker run command 

 

[wildfire305]

The error I get when trying to add a channel is "405 Method not allowed" 

 

syntax I'm using is https://www.youtube.com/c/PracticalEngineeringChannel with or without a trailing backslash it fails. But I managed to add one channel this way, it just won't add more.

[jj_uk]

Does this docker still work? After adding a YT channel and clicking "Add", the input box is cleared and nothing happens. The screen looks exactly as it did previously. 

 

[LarsW]

1 hour ago, jj_uk said:

Does this docker still work? After adding a YT channel and clicking "Add", the input box is cleared and nothing happens. The screen looks exactly as it did previously. 

 

Yes. Can you dm me the link you put in the box? Then I will try to investigate 

[loz678]

when i start the container:

 

Starting loop....
Checking for new Videos
Traceback (most recent call last):
File "main.py", line 8, in <module>
pytubDef.loop()
File "/app/pytubDef/__init__.py", line 202, in loop
channelArray = returnMonitoredChannels()
File "/app/pytubDef/__init__.py", line 256, in returnMonitoredChannels
channelURLs = monitoredChannelsFile.readlines()
io.UnsupportedOperation: not readable

[LarsW]

On 2/15/2022 at 12:16 AM, loz678 said:

when i start the container:

 

Starting loop....
Checking for new Videos
Traceback (most recent call last):
File "main.py", line 8, in <module>
pytubDef.loop()
File "/app/pytubDef/__init__.py", line 202, in loop
channelArray = returnMonitoredChannels()
File "/app/pytubDef/__init__.py", line 256, in returnMonitoredChannels
channelURLs = monitoredChannelsFile.readlines()
io.UnsupportedOperation: not readable

 

I had this error once when installing the Docker for the first time. Since then I was not able to reproduce this error. For me simply trying to restart worked.

I suggest just to try to restart and see if it works. If it doesn't feel free to reply.

[keithcroshaw]

Thanks a lot for this container, it's the most reliable that I can find.

I don't see an option for resolution settings, at least in the web UI.

Is that hidden somewhere in a config file?  I don't see it in Config.ini

[LarsW]

On 3/29/2022 at 6:31 AM, keithcroshaw said:

Thanks a lot for this container, it's the most reliable that I can find.

I don't see an option for resolution settings, at least in the web UI.

Is that hidden somewhere in a config file?  I don't see it in Config.ini

 

 

Thanks for the feedback! At the moment there is no option to specify the resolution. This project uses pytube, which should get the highest available resolution.