[Support] auto-yt-dl


LarsW

Recommended Posts

8 hours ago, keithcroshaw said:

Hmm thanks for the response.

They are allllll 720p for some reason.

I love how simple this container is, and it actually works great.

Would love for a way to expose some setting for the resolution to I could see what I told it to do vs what it's doing.

 

I will look into this, but as of right now I don´t have much time to invest, I would not expect major updates until summer, depending on when my exams are done.

But anyone is welcome to improve the project by opening a Pull Request on GitHub

Link to comment

Not sure if there was a solution, but i too am getting the following error, no matter how many times i start the container.


 

Starting loop....
Checking for new Videos
Traceback (most recent call last):
File "main.py", line 8, in <module>
pytubDef.loop()
File "/app/pytubDef/__init__.py", line 202, in loop
channelArray = returnMonitoredChannels()
File "/app/pytubDef/__init__.py", line 256, in returnMonitoredChannels
channelURLs = monitoredChannelsFile.readlines()
io.UnsupportedOperation: not readable

 

wen the docker image is setup i can see the following: 
 

[email protected]:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker create --name='auto-yt-dl' --net='br0' --ip='192.168.10.105' -e TZ="Europe/London" -e HOST_OS="Unraid" -e 'TCP_PORT_5000'='5000' -v '/mnt/user/MediaYouTube/':'/app/Downloads':'rw,slave' -v '/mnt/user/appdata/auto-yt-dl':'/app/data':'rw' 'guhu007/auto-yt-dl'

 

Edited by karldonteljames
Link to comment

is there a way to add playlists? i was able to add a channel but not any of the playlists i've tried. however the appdata has a empty file named monitoredplaylists.txt

was thinking i'd rather get specific playlists and their updates vs getting the whole channel.

 

thanks,

-Dev

Link to comment
5 hours ago, DevXen said:

is there a way to add playlists? i was able to add a channel but not any of the playlists i've tried. however the appdata has a empty file named monitoredplaylists.txt

was thinking i'd rather get specific playlists and their updates vs getting the whole channel.

 

thanks,

-Dev


yes, you should be able to add playlists. Have you made sure the playlist are public?

Link to comment
10 hours ago, LarsW said:


yes, you should be able to add playlists. Have you made sure the playlist are public?

yup. here's another one i just tried. when i paste it and click add. the page refreshes and its not listed.

 

https://www.youtube.com/playlist?list=PLimhOT0Avg-RrTX14TmLCSeB90wCK69x-

 

but the log for the container shows this:

192.168.0.2 - - [10/Apr/2022 13:41:47] "POST /index.html HTTP/1.1" 200 -
192.168.0.2 - - [10/Apr/2022 13:42:10] "[36mGET /static/style.css HTTP/1.1[0m" 304 -

not sure if thats important or not.

 

also not sure why it has 192.168.0.2 listed. thats the pc i'm not. not my server.

howerver when i added the Ryan George the full channel. it did add that and got all his videos. just not the playlists. strange.

 

 

Link to comment
4 hours ago, DevXen said:

yup. here's another one i just tried. when i paste it and click add. the page refreshes and its not listed.

 

https://www.youtube.com/playlist?list=PLimhOT0Avg-RrTX14TmLCSeB90wCK69x-

 

but the log for the container shows this:

192.168.0.2 - - [10/Apr/2022 13:41:47] "POST /index.html HTTP/1.1" 200 -
192.168.0.2 - - [10/Apr/2022 13:42:10] "[36mGET /static/style.css HTTP/1.1[0m" 304 -

not sure if thats important or not.

 

also not sure why it has 192.168.0.2 listed. thats the pc i'm not. not my server.

howerver when i added the Ryan George the full channel. it did add that and got all his videos. just not the playlists. strange.

 

 

 

I am not sure if I am reproducing the same bug as you are experiencing. On the first try I was able to add the playlist without issues.

But when I "spammed" the GUI, it becomes unresponsive and no new channels or videos can be added/removed. Even after restarting the container.
I am investigating, I will come back on this when I find out what is causing this.

 

Meanwhile, you could try to add the playlist manually. This should be able to be done by adding a new line with the channel-URL to monitoredPlaylist.txt

Link to comment
50 minutes ago, LarsW said:

 

I am not sure if I am reproducing the same bug as you are experiencing. On the first try I was able to add the playlist without issues.

But when I "spammed" the GUI, it becomes unresponsive and no new channels or videos can be added/removed. Even after restarting the container.
I am investigating, I will come back on this when I find out what is causing this.

 

Meanwhile, you could try to add the playlist manually. This should be able to be done by adding a new line with the channel-URL to monitoredPlaylist.txt

Yeah even trying to add the playlist manually and restarting the docker it's not loading them.

but the log does keep showing this.

 

* Serving Flask app 'app' (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on all addresses.
WARNING: This is a development server. Do not use it in a production deployment.
* Running on http://[redacted]:5000/ (Press CTRL+C to quit)
* Serving Flask app 'app' (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on all addresses.
WARNING: This is a development server. Do not use it in a production deployment.

Link to comment

So i've done a bunch of testing. and i've taken spaces out of my download folder and brackets, i've moved it to the array. and it'll only let me add one channel or playlist but its not downloading it. even if i manually add it, it'll show but not download. hmm.

Link to comment
9 minutes ago, DevXen said:

So i've done a bunch of testing. and i've taken spaces out of my download folder and brackets, i've moved it to the array. and it'll only let me add one channel or playlist but its not downloading it. even if i manually add it, it'll show but not download. hmm.

 

I think I have a lead on the issue. I hope I can publish a update later today or tomorrow.

 

Note: There seems to be a common misconception that the app will download the whole channel when added. This is not the case as the app will only download new videos. It is however possible to download all videos by deleting the content of the <channelName>.txt in the appdata folder.

Link to comment
Just now, LarsW said:

 

I think I have a lead on the issue. I hope I can publish a update later today or tomorrow.

 

Note: There seems to be a common misconception that the app will download the whole channel when added. This is not the case as the app will only download new videos. It is however possible to download all videos by deleting the content of the <channelName>.txt in the appdata folder.

 

Yeah I saw that in another reply. but its  not even making the .txt of the manually added playlist or second channel. the first one i have gotten it to download twice (one playlist and one channel) but only the ones it has allowed me to add through the webui. and again it's only allowed me to add the one unless i remove it and install it fresh. then it has let me add one item. and thats it. nothing in the docker log. weird.

Link to comment
  • 2 weeks later...
Use a production WSGI server instead.
* Debug mode: off
WARNING:werkzeug: * Running on all addresses.

WARNING: This is a development server. Do not use it in a production deployment.

INFO:werkzeug: * Running on http://172.17.0.9:5000/ (Press CTRL+C to quit)
ERROR:root:Something went wrong while checkig for new Videos from Biographics

 

Wanted to start using this to build my Plex youtube library since you've included the youtube-agent schema and I couldn't get meta data to load properly when trying to use TubeSync. However, adding anything via the GUI does not work; and after manually adding a channel to the monitoredChannels.txt I get this error with no other errors.

 

Any ideas?

 

Link to comment
14 hours ago, Astryl said:
Use a production WSGI server instead.
* Debug mode: off
WARNING:werkzeug: * Running on all addresses.

WARNING: This is a development server. Do not use it in a production deployment.

INFO:werkzeug: * Running on http://172.17.0.9:5000/ (Press CTRL+C to quit)
ERROR:root:Something went wrong while checkig for new Videos from Biographics

 

Wanted to start using this to build my Plex youtube library since you've included the youtube-agent schema and I couldn't get meta data to load properly when trying to use TubeSync. However, adding anything via the GUI does not work; and after manually adding a channel to the monitoredChannels.txt I get this error with no other errors.

 

Any ideas?

 

 

I currently don't have access to my own server, so I can't test it at the moment. Can you open the config.ini in the app data folder and change log = False to log = True? After running the docker again, there should be two files: flaskDEBUG and mainDEBUG.

 

I have the suspicion that it's actually the pytube library which is causing these issues.

Link to comment
  • 2 weeks later...
On 4/24/2022 at 4:35 PM, LarsW said:

 

I currently don't have access to my own server, so I can't test it at the moment. Can you open the config.ini in the app data folder and change log = False to log = True? After running the docker again, there should be two files: flaskDEBUG and mainDEBUG.

 

I have the suspicion that it's actually the pytube library which is causing these issues.

 

As suspected, pytube was / is the issue. It currently has an critical issue. I updated the docker to use youtube-dl if pytube fails. Sadly you will have to delete the links, which have not been downloaded from the appdata/<ChannelName>.txt file to download them.

Link to comment
  • 1 month later...

Hey, I noticed my container keeps stopping. even though it's set to auto run. it'll run. and mostly get videos. anyways today i had a minute so i checked the log and it has this in it:

 

Checking for new Videos
Checking for new Videos
Traceback (most recent call last):
File "/usr/local/lib/python3.9/urllib/request.py", line 1346, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/usr/local/lib/python3.9/http/client.py", line 1285, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.9/http/client.py", line 1331, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.9/http/client.py", line 1280, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.9/http/client.py", line 1040, in _send_output
self.send(msg)
File "/usr/local/lib/python3.9/http/client.py", line 980, in send
self.connect()
File "/usr/local/lib/python3.9/http/client.py", line 1447, in connect
super().connect()
File "/usr/local/lib/python3.9/http/client.py", line 946, in connect
self.sock = self._create_connection(
File "/usr/local/lib/python3.9/socket.py", line 823, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/local/lib/python3.9/socket.py", line 954, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/main.py", line 13, in <module>
pytubDef.loop()
File "/app/pytubDef/__init__.py", line 296, in loop
playlistArray = returnMonitoredPlaylist()
File "/app/pytubDef/__init__.py", line 399, in returnMonitoredPlaylist
logstr = logstr + monitoredPlaylistArray[monitoredPlaylistArray.__len__() - 1].title
File "/usr/local/lib/python3.9/site-packages/pytube/contrib/playlist.py", line 351, in title
return self.sidebar_info[0]['playlistSidebarPrimaryInfoRenderer'][
File "/usr/local/lib/python3.9/site-packages/pytube/contrib/playlist.py", line 93, in sidebar_info
self._sidebar_info = self.initial_data['sidebar'][
File "/usr/local/lib/python3.9/site-packages/pytube/contrib/playlist.py", line 81, in initial_data
self._initial_data = extract.initial_data(self.html)
File "/usr/local/lib/python3.9/site-packages/pytube/contrib/playlist.py", line 58, in html
self._html = request.get(self.playlist_url)
File "/usr/local/lib/python3.9/site-packages/pytube/request.py", line 53, in get
response = _execute_request(url, headers=extra_headers, timeout=timeout)
File "/usr/local/lib/python3.9/site-packages/pytube/request.py", line 37, in _execute_request
return urlopen(request, timeout=timeout) # nosec
File "/usr/local/lib/python3.9/urllib/request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/lib/python3.9/urllib/request.py", line 517, in open
response = self._open(req, data)
File "/usr/local/lib/python3.9/urllib/request.py", line 534, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/usr/local/lib/python3.9/urllib/request.py", line 494, in _call_chain
result = func(*args)
File "/usr/local/lib/python3.9/urllib/request.py", line 1389, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "/usr/local/lib/python3.9/urllib/request.py", line 1349, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -3] Temporary failure in name resolution>

olution>

Link to comment
12 hours ago, DevXen said:

Hey, I noticed my container keeps stopping. even though it's set to auto run. it'll run. and mostly get videos. anyways today i had a minute so i checked the log and it has this in it:

 

Checking for new Videos
Checking for new Videos
Traceback (most recent call last):
File "/usr/local/lib/python3.9/urllib/request.py", line 1346, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/usr/local/lib/python3.9/http/client.py", line 1285, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.9/http/client.py", line 1331, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.9/http/client.py", line 1280, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.9/http/client.py", line 1040, in _send_output
self.send(msg)
File "/usr/local/lib/python3.9/http/client.py", line 980, in send
self.connect()
File "/usr/local/lib/python3.9/http/client.py", line 1447, in connect
super().connect()
File "/usr/local/lib/python3.9/http/client.py", line 946, in connect
self.sock = self._create_connection(
File "/usr/local/lib/python3.9/socket.py", line 823, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/local/lib/python3.9/socket.py", line 954, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/main.py", line 13, in <module>
pytubDef.loop()
File "/app/pytubDef/__init__.py", line 296, in loop
playlistArray = returnMonitoredPlaylist()
File "/app/pytubDef/__init__.py", line 399, in returnMonitoredPlaylist
logstr = logstr + monitoredPlaylistArray[monitoredPlaylistArray.__len__() - 1].title
File "/usr/local/lib/python3.9/site-packages/pytube/contrib/playlist.py", line 351, in title
return self.sidebar_info[0]['playlistSidebarPrimaryInfoRenderer'][
File "/usr/local/lib/python3.9/site-packages/pytube/contrib/playlist.py", line 93, in sidebar_info
self._sidebar_info = self.initial_data['sidebar'][
File "/usr/local/lib/python3.9/site-packages/pytube/contrib/playlist.py", line 81, in initial_data
self._initial_data = extract.initial_data(self.html)
File "/usr/local/lib/python3.9/site-packages/pytube/contrib/playlist.py", line 58, in html
self._html = request.get(self.playlist_url)
File "/usr/local/lib/python3.9/site-packages/pytube/request.py", line 53, in get
response = _execute_request(url, headers=extra_headers, timeout=timeout)
File "/usr/local/lib/python3.9/site-packages/pytube/request.py", line 37, in _execute_request
return urlopen(request, timeout=timeout) # nosec
File "/usr/local/lib/python3.9/urllib/request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/lib/python3.9/urllib/request.py", line 517, in open
response = self._open(req, data)
File "/usr/local/lib/python3.9/urllib/request.py", line 534, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/usr/local/lib/python3.9/urllib/request.py", line 494, in _call_chain
result = func(*args)
File "/usr/local/lib/python3.9/urllib/request.py", line 1389, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "/usr/local/lib/python3.9/urllib/request.py", line 1349, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -3] Temporary failure in name resolution>

olution>

 

It seems like its not an issue with the program itself, but for some reason the name resolution (DNS Services, I guess?) fail. I've uploaded an update which prevents the docker from stopping, and just ignoring this error.

Link to comment
On 6/18/2022 at 8:22 PM, LarsW said:

 

It seems like its not an issue with the program itself, but for some reason the name resolution (DNS Services, I guess?) fail. I've uploaded an update which prevents the docker from stopping, and just ignoring this error.

 

ok. thank you.

Link to comment
  • 1 month later...
On 4/5/2022 at 10:03 PM, keithcroshaw said:

Hmm thanks for the response.

They are allllll 720p for some reason.

I love how simple this container is, and it actually works great.

Would love for a way to expose some setting for the resolution to I could see what I told it to do vs what it's doing.

 

I agree with this guy.

 

Being able to change the format/resolution would be awesome. Also, for some reason it's not properly naming folders after channels. It's also not properly naming files. But this container literally does exactly what I was looking for besides these few gripes it works perfectly.

 

- oh and also, having the upload date for the video in the video title would be an awesome feature as well.

Edited by donniedice
Link to comment
13 hours ago, donniedice said:Also, for some reason it's not properly naming folders after channels. It's also not properly naming files.

Can you give me more information? Or some examples? 
 

Regarding the format/resolution, I am looking into it. But I am not sure how much time it will take.

 

PS: Anyone is welcome to contribute on github

Link to comment
  • 1 month later...

Any word on adding the ability to append the YouTubeID to the file and folder names?

 

I was thinking of editing the docker container myself to do that. But I wouldn't know how to edit the container directly or if you can even edit them. Or if they have to just be loaded from the GitHub for changes. And if I could any update would remove that.. So I'm not sure if that's worth it. So I haven't looked into it yet. 

 

But I didn't use the YouTube agent without the YouTube IDs. I can't see the name then with the YouTube IDs cause then I would have no ones what each video is looking at them in the folders..heck I wouldn't even import which folder has what. And being able to find videos is important for me. But I did read the YouTube agent works if the id is appended to end of the folder/file so that's nice. Haha.

 

Link to comment
22 hours ago, DevXen said:

Any word on adding the ability to append the YouTubeID to the file and folder names?

 

Hey,

currently I am only interested in fixing bugs. I wanted to work on higher resolutions and naming but I am currently working on a project for study. I don´t think I will add new features till Christmas.

 

22 hours ago, DevXen said:

I was thinking of editing the docker container myself to do that. But I wouldn't know how to edit the container directly or if you can even edit them. Or if they have to just be loaded from the GitHub for changes. And if I could any update would remove that.. So I'm not sure if that's worth it. So I haven't looked into it yet.

 

If you want to contribute or edit the container to your needs, you can download the project on Github here. Basically you´ll need to edit the files accordingly to your needs and then build the container with docker (there are plenty of tutorials on Youtube).  If your build then is without issues you can create a pull request on github, I will test it, and then merge it to the main branch. This would make it available to everyone via an update.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.