[Support] binhex - rTorrentVPN


Recommended Posts

Good evening. I have been using the qBittorrentVPN image for some time now, but it appears to be choking more and more lately. Probably because I have over 1,600 torrents in it. (I get the "qBitorrent client is not reachable" message.) And that number is only going to go up steadily over time.

 

I've heard that rTorrent can handle up to 5,000–6,000 torrents before choking, so I attempted to implement BinHex's rTorrentVPN image. Unfortunately, I cannot seem to get it working and was hoping someone here could help me. It seems that it times out while attempting to start rTorrent. I'm attaching my log file (edited to remove my email address and VPN password, natch) in the hopes that it might shed some light on what's going on. Could someone please help me to diagnose and fix this?

supervisord.log

Link to comment
6 hours ago, Sturm said:

Good evening. I have been using the qBittorrentVPN image for some time now, but it appears to be choking more and more lately. Probably because I have over 1,600 torrents in it. (I get the "qBitorrent client is not reachable" message.) And that number is only going to go up steadily over time.

 

I've heard that rTorrent can handle up to 5,000–6,000 torrents before choking, so I attempted to implement BinHex's rTorrentVPN image. Unfortunately, I cannot seem to get it working and was hoping someone here could help me. It seems that it times out while attempting to start rTorrent. I'm attaching my log file (edited to remove my email address and VPN password, natch) in the hopes that it might shed some light on what's going on. Could someone please help me to diagnose and fix this?

supervisord.log 138.6 kB · 1 download

hmm ok so as the log suggests there is an issue with rtorrent starting, i suspect a dodgy entry in the config file, can you please attach the file /config/rtorrent/config/rtorrent.rc

Link to comment

I have noticed recently that my rutorrent web interface seems to be very unresponsive.  I frequently get time out errors when trying to connect to the web interface.  When it does connect, it appears to be downloading, but very slow to respond.

 

Attached is my redacted supervisord.log, but I dont see anything that stands out to me.  I have around 240 completed and inactive torrents, and 1 active torrent.  The web interface seems to indicate the CPU is at 100%.  But my Unraid system is not.  My unraid is using a Ryzen 9 3900X 12 core CPU, so there should be plenty of computing power available.

supervisord.log

Link to comment

I'm having issues trying to get my reverse proxy to work for rtorrent.  I'm not sure what setting is off.  I'm getting a 502 bad gateway error when I try to load the page.  What should I post for someone to look at?  Thanks in advance and thanks for all these awesome containers!

Link to comment
On 12/17/2019 at 10:42 AM, binhex said:

@Cat_Seeder a thought i had whilst driving home, you havent by any chance set ENABLE_RPC2 to a value of 'no' have you?, if so this will be the cause of the issue, as i use xmlrpc in order to reconfigure the incoming port on port closed.

 

On 12/23/2019 at 10:43 AM, binhex said:

yep, xmlrpc needs a url specifying to connect to rpc2, thus it needs to be exposed.

 

 

On 12/24/2019 at 1:26 AM, Cat_Seeder said:

I see, that's a limitation from the xmlrpc cli tool. Libraries can talk to rtorrent straight over SCGI. This is how software like pyrocore talks with rttorent.

 

Actually, pyrocore even exposes its own xmlrpc cli tool that does not require an http endpoint https://pyrocore.readthedocs.io/en/latest/references.html#cli-usage-rtxmlrpc

 

Would it be much of a hassle to replace xmlrpc with pyrocore's rtxmlrpc client?

 

As far I understand you already have pyrocore installed. Changes would probably be limited to the first if statement in  https://github.com/binhex/arch-rtorrentvpn/blob/master/run/nobody/rtorrent.sh and then, AFAICT, the requirement for default credentials can go away.

 

If you are busy and accept contributions I can even take a stab at it myself.

 

What do you think?

 

 

@binhex, I've modified the code as I've mentioned above it works on my machine :D. I have been running the modified version for a while with ENABLE_RPC2=no, things have been stable for over 10 days with PIA, the watchdog process has triggered port changes a couple of times with no issues whatsoever. It also works with ENABLE_RPC2=yes.

 

PR: https://github.com/binhex/arch-rtorrentvpn/pull/134 , if for whatever reason you decide not to merge the PR, I hereby grant you the right to use it in any way that suits you, no strings attached. Having said that, if you choose to merge my changes and then eliminate defaults for RPC2_USER, RPC2_PASS, WEBUI_USER and WEBUI_PASS I'll be a very happy man =).

 

----

 

If anyone else is willing to try it, I've also uploaded a standalone version to a (temporary) Gist: https://gist.github.com/flatmapthatshit/6ff8d1ac441092b33339890b5145de3a

Testers with VPNS other than PIA are especially welcome.


For a quick experiment you can use docker to mount the new version of rtorrent.sh over /home/nobody/rtorrent.sh, e.g.:

docker run -d \
    --cap-add=NET_ADMIN \
    -p 9080:9080 \
    -p 9443:9443 \
    -p 8118:8118 \
    --name=rtorrentvpn \
    -v /root/docker/data:/data \
    -v /root/docker/config:/config \
    -v /etc/localtime:/etc/localtime:ro \
    -v ./rtorrent.sh:/home/nobody/rtorrent.sh \
    -e ENABLE_RPC2=no `#followed by all other environment variables` \
    binhex/arch-rtorrentvpn

Once everything is running, other than waiting for the watchdog to do it's job, you can also run the script manually to force changes, e.g:

docker exec -it -e rtorrent_running='true' \
     -e VPN_INCOMING_PORT='30163' \
     -e vpn_ip='10.16.11.6' \
     -e external_ip='31.168.172.14 \
     <my_container> /home/nobody/rtorrent.sh

Hopefully this is helpful to the community.

Link to comment
On 12/31/2019 at 5:23 AM, binhex said:

hmm ok so as the log suggests there is an issue with rtorrent starting, i suspect a dodgy entry in the config file, can you please attach the file /config/rtorrent/config/rtorrent.rc

So, nothing, @binhex? I posted the requested file, yet I've not received a response. I apologize if I seem impatient. It's just that your qBittorrent-vpn image appears to have a great deal of trouble handling my 1,600+ torrents (at least, via its WebUI) and it's getting worse the more torrents I add to it, so I'd really like to move them all to one that—I'm told—should be able to handle up to 6,000 torrents. i.e., your rTorrent-vpn image. I'll re-attach it to this post.

 

As a side note, because I am still relatively new to this, I don't know what `ENABLE_AUTODL_IRSSI` is for, as well as `ENABLE_RPC2`. Could someone please explain those to me so I know whether or not I should have them enabled?

rtorrent.rc

Link to comment
So, nothing, @binhex? I posted the requested file, yet I've not received a response. I apologize if I seem impatient. It's just that your qBittorrent-vpn image appears to have a great deal of trouble handling my 1,600+ torrents (at least, via its WebUI) and it's getting worse the more torrents I add to it, so I'd really like to move them all to one that—I'm told—should be able to handle up to 6,000 torrents. i.e., your rTorrent-vpn image. I'll re-attach it to this post.
 
As a side note, because I am still relatively new to this, I don't know what `ENABLE_AUTODL_IRSSI` is for, as well as `ENABLE_RPC2`. Could someone please explain those to me so I know whether or not I should have them enabled?
rtorrent.rc
I've got a lot going on in my real life as well as my online life so you will have to be patient. I have had a look at your file and it looks ok so I will attempt to use it tonight and see if I can spot any errors.

Fyi rpc2 is required for external apps to connect to rtorrent and autodl is an automated way of grabbing torrents via IRC.

Sent from my CLT-L09 using Tapatalk

Link to comment
11 hours ago, Sturm said:

So, nothing, @binhex? I posted the requested file, yet I've not received a response. I apologize if I seem impatient. It's just that your qBittorrent-vpn image appears to have a great deal of trouble handling my 1,600+ torrents (at least, via its WebUI) and it's getting worse the more torrents I add to it, so I'd really like to move them all to one that—I'm told—should be able to handle up to 6,000 torrents. i.e., your rTorrent-vpn image. I'll re-attach it to this post.

 

As a side note, because I am still relatively new to this, I don't know what `ENABLE_AUTODL_IRSSI` is for, as well as `ENABLE_RPC2`. Could someone please explain those to me so I know whether or not I should have them enabled?

rtorrent.rc 5.14 kB · 0 downloads

I'm seeding 2k+ torrents at the moment (not on Unraid though, cheap NAS with 8GB of memory and two SSD disks acting as cache that handle most of the IO). I didn't really do much with rtorrent.rc other than slightly increasing the number of files and sockets available due to the nature of the load that my system in running, as well as slightly increase buffers so that my disks don't get hit as hard. This isn't a silver bullet though (see: https://github.com/rakshasa/rtorrent/wiki/Performance-Tuning for more info). Maybe start over with the container's default rtorrent.rc file?
Also make sure that your OS and cgroups aren't limiting system resources. I don't really use ruTorrent all that much, but it still loads fine, although UI is very slow. 

 

Binex explained the RPC2 mount above. autodl_irssi is used to automatically download torrents from IRC announce channels (it's mainly used by people on private trackers trying to build buffer).

Link to comment

@Sturm ok so i used your rtorrent.rc file and had no issues starting, the only thing i have spotted is that your rtorrent.rc file is a little old, there was a change a couple of weeks ago to fix up the hard coded name of 'admin' in the rtorrent.rc file, this MIGHT be the issue. so firstly update to the latest image, then simply delete the file /config/rtorrent/config/rtorrent.rc and restart the container, this will force the file to be re-created, see if this fixes the issue, if it doesnt then please re-post your rtorrent.rc file, and also follow the procedure below:-

https://github.com/binhex/documentation/blob/master/docker/faq/help.md

Link to comment

Hi @Cat_Seeder  firstly thanks for the PR, its nicely done and im pretty sure i will accept it as it does mean no exposure of rpc2 in order to have just the enhancement of auto restart on port change, which is very welcome!. 

 

now onto the prickly subject of rpc2 and default credentials, so as much as i would love to say yes to remove all default credentials i have to keep in mind the hundreds or even thousands of active users out there who (i suspect) are mostly using this rightly or wrongly with default credentials (due to it being difficult to change - see end paragraph).

 

if i remove any default password (or set a random password) then there is the potential for a massive amount of support posts as people flood in with stories of sonarr/radarr/<insert metadata downloader app> stopping working on the latest update due to rpc2 credentials being invalid, and whilst this will make you a very happy man it leaves me being an extremely overloaded and unhappy man. 

 

Just to be clear here, default credentials for the web ui and rpc2 has been in place since the inception of this docker image, ive just made it easier now for people to set them independently via environment variables, before it was a single set of credentials based off the web ui credentials for both, it was not exposed and not obvious on how to change them - in my opinion the change has improved security.

 

Link to comment
13 hours ago, binhex said:

I've got a lot going on in my real life as well as my online life so you will have to be patient. I have had a look at your file and it looks ok so I will attempt to use it tonight and see if I can spot any errors.

I apologize for sounding impatient. I did not mean to sound ungrateful for the work that you put in these images nor insensitive to your busy life. Indeed, I am grateful for your willingness to help a poor newbie with his trivial issues.

 

13 hours ago, binhex said:

Fyi rpc2 is required for external apps to connect to rtorrent and autodl is an automated way of grabbing torrents via IRC.

Excellent, thank you for the clarification. I'm guessing ruTorrent needs RPC2, then? If so, I'll leave it enabled.

 

@Cat_Seeder Thank you very much for your help, as well, but I'm afraid most of what you have said goes a bit over my head. I mean, I understand the idea of using SSDs to cache data, but making sure my "cgroups" aren't limiting resources? No idea how to determine that or even what cgroups are. Anyway, my issues with the WebUI being unresponsive are with qBittorrentVPN, thus the reason I'm attempting to use rTorrentVPN instead. (Because I've heard it can handle a larger number of torrents.) Still, I may check out that optimizing rTorrent link you provided if it turns out that rTorrent does, indeed, cope well with 1,600+ torrents.

 

4 hours ago, binhex said:

firstly update to the latest image, then simply delete the file /config/rtorrent/config/rtorrent.rc and restart the container, this will force the file to be re-created, see if this fixes the issue

Alright, I gave this a shot, but I'm afraid it didn't help. Even deleted everything in that /config folder except /config/openvpn to let it re-create everything. Still no help.

 

I will say that I have discovered at least part of the problem. In my supervisord.log file, you may have noticed that I set up a volume map of /volume2/media:/data. This poses a problem with your rTorrentVPN image whereas it didn't with qBittorrentVPN. Why? I'm not sure, but I think it has to do with changing permissions or indexing or whatever.

 

You see, In my /volume2/media is a folder called /dockervolumes, and that's where all of my docker container configs and such reside. This includes Plex. Well, /volume2/media/dockervolumes/plex/config/Library/Application Support/Plex Media Server contains nearly 800,000 files and folders. I've no idea what it's all for, but I'm sure a great deal of that is for metadata, thumbnails, etc. And it probably could use some cleanup somehow, but I don't know how to do so without messing things up. And this is in addition to another folder under /volume2/media that contains almost 140,000 files and folders itself.

 

So, I edited my docker-compose.yml file to only have several volume mappings rather than just the one, overarching volume mapping of /volume2/media:/data. Each of the new volume mappings points to just those folders that I need rTorrent to have access to. Once I did another container re-creation with these new mappings, it seems rTorrent and ruTorrent start up fine. (Well, except for rTorrent complaining about ffmpeg with "screenshots: Plugin will not work. rTorrent user can't access external program (ffmpeg)." But I don't think that's necessary.)

 

Now, I just need to figure out how to copy over my 1,600 .torrent files from qBittorrent to rTorrent while keeping their download locations intact and such. Then we'll see whether or not rTorrent really can handle that kind of load.

 

Again, thank you guys so much for your help in this matter.

Edited by Sturm
Number update
Link to comment
18 hours ago, binhex said:

Hi @Cat_Seeder  firstly thanks for the PR, its nicely done and im pretty sure i will accept it as it does mean no exposure of rpc2 in order to have just the enhancement of auto restart on port change, which is very welcome!. 

Thanks for merging my PR @binhex . Hopefully it turns out to be useful for everyone else. I've latter realised that we can probably safely remove this line from the watchdog script as well as the xmlrpc-c client itself if we want, but this can be done at a later stage.

18 hours ago, binhex said:

now onto the prickly subject of rpc2 and default credentials, so as much as i would love to say yes to remove all default credentials i have to keep in mind the hundreds or even thousands of active users out there who (i suspect) are mostly using this rightly or wrongly with default credentials (due to it being difficult to change - see end paragraph).

 

if i remove any default password (or set a random password) then there is the potential for a massive amount of support posts as people flood in with stories of sonarr/radarr/<insert metadata downloader app> stopping working on the latest update due to rpc2 credentials being invalid, and whilst this will make you a very happy man it leaves me being an extremely overloaded and unhappy man. 

Hahaha, I understand, and I've noticed the huge amount of support tickets when you added the extra options as well. I can't really ask you to sacrifice even more of your time to handle support tickets. 

To be honest, my main worry is that your containers may become a target for malware and endanger the exact type of inexperienced user that today struggles with docker environment variables. With 5M+ pull requests and considering that your container is part of DockSTARTer, I expect tens of thousands of installations, maybe more. Since, IMO, you have the best and most complete rtorrentt + rutorrent container out there I expect it's popularity to grow. As you said, right or wrong people will use default credentials. How many of those installations are exposing unprotected RPC2 mounts over the internet? How many of them are only protecting XMLRPC with default credentials? That's a juicy target for hackers, and unfortunately it's trivial to modify the scripts that already target unprotected RPC2 mounts and attempt a few passwords like admin / rutorrent.


My take on it though is that maybe removing defaults can be done in a way that does not disrupt your existing user base too much but still provides better security out of the box for newcomers? See bellow.

 

18 hours ago, binhex said:

Just to be clear here, default credentials for the web ui and rpc2 has been in place since the inception of this docker image, ive just made it easier now for people to set them independently via environment variables, before it was a single set of credentials based off the web ui credentials for both, it was not exposed and not obvious on how to change them - in my opinion the change has improved security.

 

So, here's my take on it, we can leverage the environment variables to change defaults without affecting old users (that already have rpc2_auth and webui_auth files in place). I think that most of the changes would need to be done in: https://github.com/binhex/arch-rtorrentvpn/blob/master/run/nobody/rutorrent.sh#L225-L328 and https://github.com/binhex/arch-rtorrentvpn/blob/master/build/root/install.sh#L479-L539

 

Here's my proposal ordered by most painful change to least painful one:

  1. The default for ENABLE_RPC2, when not specified, should be no. I know, this one may result in some support requests, but most of the users will probably follow your guide and have it set to yes already. For everyone else I can give a hand by asking them to set ENABLE_RPC2=yes. This change, by itself, will make your container way safer.
  2. There should be no default for RPC2_USER, RPC2_PASS, WEBUI_USER and WEBUI_PASS, instead one of 3 things should happen:
        a. If non-empty rpc2_auth and web_auth exists and use credentials match admin / rutorrent then we print a giant warning telling users not to use
            default credentials. But the container still boots and works as expected - this is the best that we can do for older insecure installations without a
            flood of support requests.
        b. If non-empty rpc2_auth and web_auth exists and there isn't a admin / rutorrent user present we do nothing. Security conscious users that know what
            they are doing will not be bothered
        c. If rpc_auth and web_auth aren't there or are empty (new users), then we fail the container startup with a clear explanation about how to use
            RPC_USER, RPC2_PASS, WEBUI_USER and WEBUI_PASS. New users will set their own credentials as supposed
  3. README.md examples should be changed so that admin / rutorrent are not there by default. Although not Ideal, I'm ok with myusername / mypassword. I doubt that we'll get support tickets from new users, although some will actually use myusername and mypassword, we may also want to check for it in 2.a. :D).

IMO old users will basically be taken care of by 2.a and 2.b and the support requests that we get about RPC2 can be solved by asking the user to set ENABLE_RPC2=yes. Maybe I'm being naive about the typical user knowledge level (you know your user base better than anyone else), however it feels like this can be done without resulting in an endless ****storm of support requests. What do you think? If you agree I again volunteer to help with the changes as well as to monitor the forums and tell users to ENABLE_RPC2 when Sonarr / Radarr / **rr fails :D.

 

 

Link to comment
16 hours ago, Sturm said:

@Cat_Seeder Thank you very much for your help, as well, but I'm afraid most of what you have said goes a bit over my head. I mean, I understand the idea of using SSDs to cache data, but making sure my "cgroups" aren't limiting resources? No idea how to determine that or even what cgroups are. Anyway, my issues with the WebUI being unresponsive are with qBittorrentVPN, thus the reason I'm attempting to use rTorrentVPN instead. (Because I've heard it can handle a larger number of torrents.) Still, I may check out that optimizing rTorrent link you provided if it turns out that rTorrent does, indeed, cope well with 1,600+ torrents.

Hi @Sturm, I understand. We all have been starters someday. To be honest though, seeding several thousand torrents from a single docker container on a single machine requires some sysadmin skills. While you may get 2k torrents to work out of the box, it is important to understand that rtorrent, although very capable, is not the only piece required to make it work. It also important to set realistic expectations, as you add more torrents, ruTorrent (the web interface) will begin to slowdown and eventually become unstable. Most of us seeding over 3k torrents are using rTorrent exclusively through the command line and have automated flows to import, organise and automatically remove torrents. Other than that, what I was trying to say is that you will eventually hit configuration / OS / Kernel limits and ultimately hardware limits. Seeding 5k+ torrents from a single client is quite a challenging sport that a few of us nerds like to play :D.

 

I may be wrong, but it seems to be like you are more interested in the end result (being able to seed your torrents) than the technical challenge of running everything in a single container right? If so, an alternative that may prove easier to setup is to run multiple containers. Just seed 1k / 1.5k torrents per container, keep an eye on CPU, network and memory usage. As for storage, spread the load over multiple HDs. Maybe finish setting up your first rtorrent box and then use docker-compose or a orchestration tool to run multiple instances in different http / https ports.

16 hours ago, Sturm said:

 

So, I edited my docker-compose.yml file to only have several volume mappings rather than just the one, overarching volume mapping of /volume2/media:/data. Each of the new volume mappings points to just those folders that I need rTorrent to have access to. Once I did another container re-creation with these new mappings, it seems rTorrent and ruTorrent start up fine. (Well, except for rTorrent complaining about ffmpeg with "screenshots: Plugin will not work. rTorrent user can't access external program (ffmpeg)." But I don't think that's necessary.)

That's ok. It's just a ruTorrent plugin failing to load, no need to worry unless you want to screenshoot videos from ruTorrent. @binhex, I can confirm the error message. We may want to get rid of the screenshoots plugin (or install ffmpeg, although I doubt that many people will actually care for it).

16 hours ago, Sturm said:

Now, I just need to figure out how to copy over my 1,600 .torrent files from qBittorrent to rTorrent while keeping their download locations intact and such. Then we'll see whether or not rTorrent really can handle that kind of load.

 

Again, thank you guys so much for your help in this matter.

Autotools may be handy ( https://github.com/Novik/ruTorrent/wiki/PluginAutotools ) to add torrents. I would also temporarily disable hash checks (assuming that you are just pointing rtorrent to the same files that qBittorrent was serving) so that IO operations do not overload your server. Finally, add torrents in small batches (up to 100 at a time). I've migrated a few of my seedboxes to binhex containers and it's nowhere as painful as it looks :).

 

Cheers,

Link to comment

Hey, thanks for being patient with me, Cat_Seeder.

11 hours ago, Cat_Seeder said:

It also important to set realistic expectations, as you add more torrents, ruTorrent (the web interface) will begin to slowdown and eventually become unstable. Most of us seeding over 3k torrents are using rTorrent exclusively through the command line and have automated flows to import, organise and automatically remove torrents.

Okay, so this is exactly the problem I'm running into with qBittorrent. If I'm going to have the same issue with ruTorrent, then perhaps I shouldn't really even bother moving to rTorrent. qBittorrent works fine with my automated workflows from Radarr and Sonarr; it's just when adding/managing torrents manually that I run into the WebUI slowdown.

 

I don't suppose you could share some of your automated workflows through the CLI that I could adapt for qBittorrent? I'm fairly comfortable with bash/zsh. Or at least just point me in some good directions for how to manage torrents manually without the WebUI?

11 hours ago, Cat_Seeder said:

I may be wrong, but it seems to be like you are more interested in the end result (being able to seed your torrents) than the technical challenge of running everything in a single container right? If so, an alternative that may prove easier to setup is to run multiple containers. Just seed 1k / 1.5k torrents per container, keep an eye on CPU, network and memory usage.

That's pretty much correct. As I said, it the actual backend torrenting appears to be running okay, it's just the WebUI that's super-sluggish and 90% non-responsive.

I had the same idea of running multiple containers, since I'd only need two or three (for now) and I have plenty of CPU/RAM headroom on my NAS. However, I have been told by others that it would start using up more resources and overhead. I have a feeling they overestimate how much resources the qBittorrentVPN container uses on my NAS. Currently, it's using less than 4% CPU and about 2.6GB out of 8GB total RAM. Spread across 2–3 containers, it ought to be fine, don't you think?

 

I just need to make sure I set up the port mappings on each one correctly. I tried doing it last week and couldn't get both qBittorrentVPN containers running at the same time because it seems like they have some hard-coded ports that cannot be changed in `docker-compose.yml`, causing conflicts. I should probably post that issue in the qBittorrentVPN forum.

Link to comment

Hey all!

 

I am very new to all of this, but am working on building my own seedbox. I am running into some issues, and I think I am not filling the run file out correctly. Can someone break down for me exactly needs to be put in the NAME_SERVER section?

 

This is where the log stops every time.

 

2020-01-07 14:17:37,500 DEBG 'watchdog-script' stdout output:

[info] rTorrent process listening on port 5000

2020-01-07 14:17:37,506 DEBG 'watchdog-script' stdout output:

[info] Initialising ruTorrent plugins (checking rTorrent is running)...

2020-01-07 14:17:37,510 DEBG 'watchdog-script' stdout output:

[info] rTorrent running

[info] Initialising ruTorrent plugins (checking nginx is running)...

Link to comment
1 hour ago, steffanaache said:

Hey all!

 

I am very new to all of this, but am working on building my own seedbox. I am running into some issues, and I think I am not filling the run file out correctly. Can someone break down for me exactly needs to be put in the NAME_SERVER section?

 

This is where the log stops every time.

 

2020-01-07 14:17:37,500 DEBG 'watchdog-script' stdout output:

[info] rTorrent process listening on port 5000

2020-01-07 14:17:37,506 DEBG 'watchdog-script' stdout output:

[info] Initialising ruTorrent plugins (checking rTorrent is running)...

2020-01-07 14:17:37,510 DEBG 'watchdog-script' stdout output:

[info] rTorrent running

[info] Initialising ruTorrent plugins (checking nginx is running)...

To add, I realized earlier on there was a permission denied error so I changed ownership of /root/docker and that got me past that.

 

Now I am yet again stuck...

 

    

Script started, file is /home/nobody/typescript

2020-01-07 16:30:34,131 DEBG 'watchdog-script' stdout output:


Script done, file is /home/nobody/typescript

2020-01-07 16:30:35,049 DEBG 'rutorrent-script' stdout output:


[info] rtorrent started, setting up rutorrent...


[info] Setting PHP timezone to UTC...

2020-01-07 16:30:35,051 DEBG 'rutorrent-script' stdout output:


[info] nginx cert files already exists, skipping copy

2020-01-07 16:30:35,052 DEBG 'rutorrent-script' stdout output:


[info] nginx config file already exists, skipping copy

2020-01-07 16:30:35,052 DEBG 'rutorrent-script' stdout output:


[info] rutorrent conf folder already exists, skipping copy

2020-01-07 16:30:35,054 DEBG 'rutorrent-script' stdout output:


                "python"        => '/usr/bin/python',           // Something like /usr/bin/python. If empty, will be found in PATH.

2020-01-07 16:30:35,060 DEBG 'rutorrent-script' stdout output:


[info] running rsync to copy rutorrent user plugins to the plugins folder inside the container...

2020-01-07 16:30:35,062 DEBG 'rutorrent-script' stdout output:


sending incremental file list

2020-01-07 16:30:35,062 DEBG 'rutorrent-script' stdout output:


README.txt

2020-01-07 16:30:35,062 DEBG 'rutorrent-script' stdout output:


theme/themes/README.txt

2020-01-07 16:30:35,063 DEBG 'rutorrent-script' stdout output:

sent 428 bytes  received 60 bytes  976.00 bytes/sec


total size is 237  speedup is 0.49

2020-01-07 16:30:35,063 DEBG 'rutorrent-script' stdout output:


[info] rutorrent share folder already exists, skipping copy

2020-01-07 16:30:35,064 DEBG 'rutorrent-script' stdout output:


[info] nginx /rpc2 location enabled

2020-01-07 16:30:35,070 DEBG 'rutorrent-script' stdout output:


[info] Updating password for rpc2 account 'admin'...

2020-01-07 16:30:35,071 DEBG 'rutorrent-script' stderr output:


Updating password for user admin

2020-01-07 16:30:35,076 DEBG 'rutorrent-script' stdout output:


[info] Updating password for web ui account 'admin'...

2020-01-07 16:30:35,077 DEBG 'rutorrent-script' stderr output:


Updating password for user admin

2020-01-07 16:30:35,077 DEBG 'rutorrent-script' stdout output:


fo] starting php-fpm...

2020-01-07 16:30:35,103 DEBG 'rutorrent-script' stderr output:


[NOTICE] [pool www] 'user' directive is ignored when FPM is not running as root


[NOTICE] [pool www] 'group' directive is ignored when FPM is not running as root

2020-01-07 16:30:35,106 DEBG 'rutorrent-script' stdout output:


[info] starting nginx...

2020-01-07 16:30:36,148 DEBG 'watchdog-script' stdout output:


[info] rTorrent process started


[info] Waiting for rTorrent process to start listening on port 5000...

2020-01-07 16:30:36,151 DEBG 'watchdog-script' stdout output:


[info] rTorrent process listening on port 5000

2020-01-07 16:30:36,154 DEBG 'watchdog-script' stdout output:


[info] Initialising ruTorrent plugins (checking rTorrent is running)...

2020-01-07 16:30:36,158 DEBG 'watchdog-script' stdout output:


[info] rTorrent running


[info] Initialising ruTorrent plugins (checking nginx is running)...

2020-01-07 16:30:36,162 DEBG 'watchdog-script' stdout output:


[info] nginx running


[info] Initialising ruTorrent plugins...

2020-01-07 16:30:36,627 DEBG 'watchdog-script' stdout output:


2020-01-07 16:30:36,643 DEBG 'watchdog-script' stdout output:


[info] ruTorrent plugins initialised

2020-01-07 16:30:36,647 DEBG 'watchdog-script' stdout output:


[info] Attempting to start Privoxy...

2020-01-07 16:30:37,658 DEBG 'watchdog-script' stdout output:


[info] Privoxy process started


[info] Waiting for Privoxy process to start listening on port 8118...

2020-01-07 16:30:37,661 DEBG 'watchdog-script' stdout output:


[info] Privoxy process listening on port 8118

Link to comment
20 hours ago, Sturm said:

I don't suppose you could share some of your automated workflows through the CLI that I could adapt for qBittorrent? I'm fairly comfortable with bash/zsh. Or at least just point me in some good directions for how to manage torrents manually without the WebUI?

Sure, I don't know much about qBittorrent, but here's my setup for rtorrent.

 

1. I have a watch folder, an incomplete folder and a complete folder, each with a bunch of subdirectories, e.g.:

    /watch

        /tv

        /movies

        /bonus

        /...

    /incomplete

        /tv

        /....

    /complete

        /tv

        /...

 

2. /watch should be monitored by pyrotorque's Watch Job (see: https://pyrocore.readthedocs.io/en/latest/advanced.html#rtorrent-queue-manager). I'm still using old school rtorrent watched directories (https://github.com/rakshasa/rtorrent/wiki/Common-Tasks-in-rTorrent#watch-a-directory-for-torrents) because I'm too lazy to upgrade, but pyrotorque is certainly easier and more powerful to use.

3. /incomplete is where rtorrent keeps data from downloads in progress (see 2).

4. /complete is where downloads are moved after they are done.

5. To automatically delete torrents I do two things.

     a) All of my important trackers have aliases configured: (see https://pyrocore.readthedocs.io/en/latest/setup.html#setting-values-in-config-ini)

     b) I have a cron job that deletes properly seeded torrents. It uses a bunch of rcontrol --cull commands - one rule per tracker, plus a general overall rule. It's a slight variation of: https://pyrocore.readthedocs.io/en/latest/usage.html#deleting-download-items-and-their-data

6. autodl-irssi is set to download stuff from the trackers where I want to build ratio, it monitors IRC Channels and add a certain number of filtered torrent files to the watched folder every week (see: https://autodl-community.github.io/autodl-irssi/configuration/overview/ )

7. I can add a torrent manually just by dropping it in the watch folder. I can delete torrents manually with rcontrol and I can use rtorrent-ps (CLI tool) instead of ruTorrent to check how things are going.

8. I use Downloads Router (tohttps://www.google.com/url?sa=t&source=web&rct=j&url=https://chrome.google.com/webstore/detail/downloads-router/fgkboeogiiklpklnjgdiaghaiehcknjo%3Fhl%3Den&ved=2ahUKEwiAs93ujPPmAhXAUxUIHWzsAZAQFjADegQIBBAB&usg=AOvVaw2Fwk4fqI4MValYtWgVXOrE) go save torrent files to the most appropriate watched subfolder according to the tracker from where I'm downloading stuff (e.g., if the private tracker is specialized in in movies, any torrent files that I download will end up in watch/movies by default). I can always override where torrent files are saved or move them to a different folder later.

9. My Plex libraries are configured to read from suvolders of /complete

 

That's pretty much it for torrenting. I have Sonarr and Radarr but I only use them with Usenet (they can quickly destroy your buffer on private trackers if you are not careful with what you are doing). 

20 hours ago, Sturm said:

However, I have been told by others that it would start using up more resources and overhead. I have a feeling they overestimate how much resources the qBittorrentVPN container uses on my NAS. Currently, it's using less than 4% CPU and about 2.6GB out of 8GB total RAM. Spread across 2–3 containers, it ought to be fine, don't you think?

You will have no problems running a few torrent containers in your NAS. Yes, theoretically more processes means more things to manage, but from personal experience, it's easier to split the load. If you reach an OS or hardware limit you will know when it happens, but I know for a fact that quite a few seedbox providers are running dozens of rtorrent clients per VM with very happy and satisfied paying customers.

 

Cheers,

  • Thanks 1
Link to comment
On 12/22/2019 at 9:07 PM, dprus said:

2019-12-22 18:52:34,820 DEBG 'watchdog-script' stdout output:                                                                                                                                                  
[warn] Wait for rTorrent process to start aborted, too many retries                                                                                                                              
2019-12-22 18:52:34,827 DEBG 'watchdog-script' stdout output:                                                                                                                                                  
[info] Autodl-irssi not enabled, skipping startup                                                                                                                                                               
2019-12-22 18:52:34,828 DEBG 'watchdog-script' stdout output:                                                                                                                                                  
[info] Initialising ruTorrent plugins (checking rTorrent is running)...  

 

It's not immediately clear to me where I can see log files to tell why rTorrent is failing to start so I'm a bit stuck right now. Would appreciate some help at this point. Thanks!

 

I'm having the same issue, and same log output: "[warn] Wait for rTorrent process to start aborted, too many retries"

Link to comment
On 1/6/2020 at 5:32 PM, Cat_Seeder said:

@binhex, I can confirm the error message. We may want to get rid of the screenshoots plugin (or install ffmpeg, although I doubt that many people will actually care for it).
 

I've just updated to the latest version (after over 2 weeks of uptime, yay!). Everything is working great, including the screenshots plugin. As always, thanks @binhex.

On 1/6/2020 at 4:46 PM, Cat_Seeder said:

So, here's my take on it, we can leverage the environment variables to change defaults without affecting old users (that already have rpc2_auth and webui_auth files in place). I think that most of the changes would need to be done in: https://github.com/binhex/arch-rtorrentvpn/blob/master/run/nobody/rutorrent.sh#L225-L328 and https://github.com/binhex/arch-rtorrentvpn/blob/master/build/root/install.sh#L479-L539

 

Here's my proposal ordered by most painful change to least painful one:

  1. The default for ENABLE_RPC2, when not specified, should be no. I know, this one may result in some support requests, but most of the users will probably follow your guide and have it set to yes already. For everyone else I can give a hand by asking them to set ENABLE_RPC2=yes. This change, by itself, will make your container way safer.
  2. There should be no default for RPC2_USER, RPC2_PASS, WEBUI_USER and WEBUI_PASS, instead one of 3 things should happen:
        a. If non-empty rpc2_auth and web_auth exists and use credentials match admin / rutorrent then we print a giant warning telling users not to usr default credentials. But the container still boots and works as expected - this is the best that we can do for older insecure installations without a flood of support requests.
        b. If non-empty rpc2_auth and web_auth exists and there isn't a admin / rutorrent user present we do nothing. Security conscious users that know what they are doing will not be bothered
        c. If rpc_auth and web_auth aren't there or are empty (new users), then we fail the container startup with a clear explanation about how to use RPC_USER, RPC2_PASS, WEBUI_USER and WEBUI_PASS. New users will set their own credentials as supposed
  3. README.md examples should be changed so that admin / rutorrent are not there by default. Although not Ideal, I'm ok with myusername / mypassword. I doubt that we'll get support tickets from new users, although some will actually use myusername and mypassword, we may also want to check for it in 2.a. :D).

IMO old users will basically be taken care of by 2.a and 2.b and the support requests that we get about RPC2 can be solved by asking the user to set ENABLE_RPC2=yes. Maybe I'm being naive about the typical user knowledge level (you know your user base better than anyone else), however it feels like this can be done without resulting in an endless ****storm of support requests. What do you think? If you agree I again volunteer to help with the changes as well as to monitor the forums and tell users to ENABLE_RPC2 when Sonarr / Radarr / **rr fails :D.

 

 

binhex, Just letting you know that I'm available and willing to write the above code if I have your blessing - just give me a shout since those changes are a little bit more involved than the rtxmlrpc stuff. If you rather me not to do it / still think that the cost of disruption is too high please let me know (no hard feelings at all, I promise not to pester you with security stuff any more).

Edited by Cat_Seeder
Link to comment
On 1/11/2020 at 8:13 PM, Cat_Seeder said:

If you rather me not to do it / still think that the cost of disruption is too high please let me know (no hard feelings at all, I promise not to pester you with security stuff any more).

after having a long hard think about it, the only change i would be prepared to make is to randomise the web ui and rpc2 password if not already set, everything else i want to keep as it currently is. i think this would be a good compromise, it improves security by not having a hard coded default password, whilst giving the users a decent out of the box experience with rpc2 enabled (most people will still require this for external access for apps).

 

this will cause some disruption, im fairly certain of that as i suspect a lot of users are probably running with the password not defined (unraid templates do not auto update), but i can mitigate some of the support by links to docs, posts in this thread etc.

 

let me know if you feel its worth going ahead with this, it shouldn't take me too long to code up, if not then the alternative is leave it as is, or of course create your own fork on github and do your own thing :-)

Link to comment
On 1/13/2020 at 12:09 PM, binhex said:

after having a long hard think about it, the only change i would be prepared to make is to randomise the web ui and rpc2 password if not already set, everything else i want to keep as it currently is. i think this would be a good compromise, it improves security by not having a hard coded default password, whilst giving the users a decent out of the box experience with rpc2 enabled (most people will still require this for external access for apps).

 

this will cause some disruption, im fairly certain of that as i suspect a lot of users are probably running with the password not defined (unraid templates do not auto update), but i can mitigate some of the support by links to docs, posts in this thread etc.

 

let me know if you feel its worth going ahead with this, it shouldn't take me too long to code up, if not then the alternative is leave it as is, or of course create your own fork on github and do your own thing 🙂

Thanks @binhex. I understand and actually think this is a good solution, it will at least force users to think about passwords and over time it will reduce the attack surface quite a bit (just make sure that the first example that a user finds in the documentation does not use admin/rutorrent as default credentials). I know that it's going to be a little painful, but trust me on this one, you will be doing a great thing for your userbase, even if they hate you for it.

 

I have to admit that I already mantain my own private fork of your repo, as well as a docker-compose build that handles the stuff that I need: Multiuser support - as in one rtorrent instance per user  - rar support, rutorrent plugins such as File Manager, File Upload, etc for my wife and for the rare occasion when I use the web client (ruTorrent is no longer working that we'll for me), some tune-up, tools to beef up security a little bit (fail2ban, etc) as well as a reverse proxy + ddns client + auto certificate manager combo :). I don't plan to make it public anytime soon, but if any of the rtorrent / ruTorrent / security features sound interesting I'll be happy to contribute upstream.

 

Cheers and thanks again :)

Edited by Cat_Seeder
Link to comment
12 hours ago, Cat_Seeder said:

I have to admit that I already mantain my own private fork of your repo, as well as a docker-compose build that handles the stuff that I need

Imitation is the sincerest form of flattery. :-) :-)

 

coding is done, just working on the documentation, posts etc and then i will release.

  • Thanks 1
Link to comment
  • binhex locked this topic
Guest
This topic is now closed to further replies.