[Support] binhex - rTorrentVPN


Recommended Posts

I switched from qbittorrentvpn to rtorrentvpn because I had some strange problems where in qbittorrent all torrents would stop downloading for several minutes and after a while everything was fine for about 1min then all downloads would freeze again. That problem repeated forever.

rtorrentvpn have worked perfect since I switched for about 6 months ago. Now I get the same problems suddenly with rtorrentvpn, has something changed ?

Edited by Cliff
Link to comment

Is the docker just having issues lately or is it because I've grown to about 1000 torrents in the past few months? I have it pinned to 2 cores, 4 threads, on an E3-1230 v3 (3.3GHz processor). I know that's a really general question, but I just have no clue if my server is the bottleneck or the container is just having performance issues. I'm finding it times out often, crashes often, very slow UI updates, and can't seem to handle more than 5-7 active downloads now without crashing.

 

If it helps, I'm using AirVPN which doesn't seem to have any reports of issues as far as I've seen.

Edited by s449
Additional info
Link to comment
17 hours ago, Cliff said:

I switched from qbittorrentvpn to rtorrentvpn because I had some strange problems where in qbittorrent all torrents would stop downloading for several minutes and after a while everything was fine for about 1min then all downloads would freeze again. That problem repeated forever.

rtorrentvpn have worked perfect since I switched for about 6 months ago. Now I get the same problems suddenly with rtorrentvpn, has something changed ?

 

3 hours ago, s449 said:

Is the docker just having issues lately or is it because I've grown to about 1000 torrents in the past few months? I have it pinned to 2 cores, 4 threads, on an E3-1230 v3 (3.3GHz processor). I know that's a really general question, but I just have no clue if my server is the bottleneck or the container is just having performance issues. I'm finding it times out often, crashes often, very slow UI updates, and can't seem to handle more than 5-7 active downloads now without crashing.

 

If it helps, I'm using AirVPN which doesn't seem to have any reports of issues as far as I've seen.

I was having issues with my seedbox over a year ago (closer to 2 probably), and the tldr of the quoted support response from provider is too many active is a problem (currently tend to have around 75 active with no prob on there). Issue on seedbox was move when completed not always working, torrents going into pausing state, and rtorrent crashing.

 

Over the past couple months i remembered the seedbox issue and applied that to binhexrtorrentvpn (was having issues similar to cliff), and container has been working pretty well since. Getting uploads and downloads, but run into issues if there are more than 100 to 150 non-stopped torrents, or if active count is over like 30. I'm running dual e5-2695v2 (2*12 cores, hyperthreaded, 2.4ghz), 128gb ram, with nothing pinned, pia vpn. My plan has been to make a manager to coordinate between multiple containers like a "rtorrent container swarm" thing (multiple containers networked thru 1 running vpn & manager)...but have not gotten around to it yet as it feels like an overengineered solution.

 

Maybe others can weigh in as to how many non-stopped and active torrents they have...or other knowledge of what our problems might be.

 

Quote

I believe it's possible that both of these issues were being caused by the amount of torrents that you had in active downloading states. For whatever reason this seems to strain rtorrent a lot past a certain point, it may be too aggressive in trying to find peers on so many torrents that it impacts performance of other aspects. You should find right now that most of your torrents have been put into stopped states--this was necessary to troubleshoot because the client could hardly load at all--and in doing so the client is significantly more responsive. I expect that you won't have any issues with torrents being moved or randomly pausing as long as your active downloads remains at a lower number, for example in the single digits.

...

rtorrent is regrettably the best client for handling large amounts of torrents overall. The other clients we offer, Deluge and Transmission, have better queuing systems that allow for specifying a maximum number of torrents to be active simultaneously (downloading, uploading, and overall), but these clients cannot handle as many torrents overall as rtorrent and I would generally recommend keeping them at 2000 or fewer, with the higher you go, the worse performance will become. You could run multiple clients at once if you wished, to help spread out the load between them.

 

Link to comment
3 hours ago, DontWorryScro said:

Any one able to point me in the right direction for how to install File Manager?

For the old version of the container with rtorrent-ps + old version of file-manager (before it was split on its own repo) see: https://github.com/binhex/arch-rtorrentvpn/issues/96

 

For the latest version of the container + newest version of file manager I can't get it to work at all. Had to revert back. Please do let me know if you figure out.

  • Thanks 1
Link to comment
15 hours ago, Cat_Seeder said:

For the latest version of the container + newest version of file manager I can't get it to work at all. Had to revert back. Please do let me know if you figure out.

fyi and everybody else, i have rolled back to rtorrent-ps (as previously mentioned) and have also kludged a build of pyrocore back into the image as it was, so tagged 'latest' should now be identical to how it was before the upgrade to rtorrent 0.9.8.

 

i have also identified the issue with http (missing iptables rules) so this should also be working for 'latest'.

  • Thanks 2
Link to comment

Recently installed the container.

 

I was having a start-up issue where the container wouldn't launch because it was hung up on not having passwords for the webui and the contrainer itself "RPC2_USER not defined (via -e RPC2_USER), defaulting to 'admin' /usr/local/bin/init.sh: line 346: /config/nginx/security/rpc2_pass: No such file or directory"

 

After reading through someone else's post, they fixed this by defining a user/pass for the webui and the RPC2 and this worked for me too.

 

However, when I try to log into the webui using https://IP:9443, it gives me a certificate error.

 

I can essentially tell my browser "Yes, it's fine. I promise." and I can log in, but I'd like to get this issue fixed if possible.

 

Any help would be appreciated!

Link to comment
2 minutes ago, godzillafanclub said:

I was having a start-up issue where the container wouldn't launch because it was hung up on not having passwords for the webui and the contrainer itself "RPC2_USER not defined (via -e RPC2_USER), defaulting to 'admin' /usr/local/bin/init.sh: line 346: /config/nginx/security/rpc2_pass: No such file or directory"

i will take a look at this, thanks.

 

3 minutes ago, godzillafanclub said:

However, when I try to log into the webui using https://IP:9443, it gives me a certificate error.

 

I can essentially tell my browser "Yes, it's fine. I promise." and I can log in, but I'd like to get this issue fixed if possible.

the certificate included is a self signed cert, thus you get the warning, if you dont want the warning then you will need to purchase a certificate from a trusted internet CA, or generate one from letsencrypt website.

Link to comment
2 hours ago, binhex said:

fyi and everybody else, i have rolled back to rtorrent-ps (as previously mentioned) and have also kludged a build of pyrocore back into the image as it was, so tagged 'latest' should now be identical to how it was before the upgrade to rtorrent 0.9.8.

 

i have also identified the issue with http (missing iptables rules) so this should also be working for 'latest'.

And everything is working again. Thanks Binex, you are awesome.

I know that eventually we'll need to jump off rtorrent-ps (unless a miracle happens and it pick up the pace again). Hopefully there's a way to do it without having to force users to expose RPC2 mounts and someone will find a way to make file manager before that.

 

For now, I'm glad to be on good ol' versions :).

Link to comment
11 hours ago, Cat_Seeder said:

And everything is working again. Thanks Binex, you are awesome.

I know that eventually we'll need to jump off rtorrent-ps (unless a miracle happens and it pick up the pace again). Hopefully there's a way to do it without having to force users to expose RPC2 mounts and someone will find a way to make file manager before that.

 

For now, I'm glad to be on good ol' versions :).

tbh im not overly happy to be on either version as of right now, 0.9.6 has tracker overload issues (https://github.com/pyroscope/rtorrent-ps/issues/132) and DDOS vulnerabilities (apparently) and 0.9.8 has a major flaw which is triggered by offline trackers (https://github.com/rakshasa/rtorrent/issues/999), bit of a rock and a hard place situation.

 

for now i will monitor the issues and if things change then review the situation.

 

p,s. having said all of the above, rtorrent/rutorrent is STILL my torrent client of choice for me, its fast and has most features you want from a torrent client so im not abandoning it...yet :-).

  • Like 3
Link to comment

i've notived that rutorrent gets tuck with 100% cpu usage, sometimes in a way that it affects other dockers. is there anything i can do to figure out whats going on? "htop" only shows me that it does that, but i don't see why. i only have about 20 torrents open, most of them idling. appreciate any help!

Link to comment
On 2/15/2021 at 11:02 AM, s449 said:

Is the docker just having issues lately or is it because I've grown to about 1000 torrents in the past few months? I have it pinned to 2 cores, 4 threads, on an E3-1230 v3 (3.3GHz processor). I know that's a really general question, but I just have no clue if my server is the bottleneck or the container is just having performance issues. I'm finding it times out often, crashes often, very slow UI updates, and can't seem to handle more than 5-7 active downloads now without crashing.

 

If it helps, I'm using AirVPN which doesn't seem to have any reports of issues as far as I've seen.

So I got my client down to about 800 torrents and it seemed to improve. However, I added 3 torrents that were 100GB and another that was 50GB. I've had some pretty big torrents before but I'm not sure ever this big, plus trying to download it all at once. Extremely slow UI and it kept crashing and because of that kept trying to re-check everything and crashing again. I even had my Unraid shares crash to where I had to reboot my server to see them again, even once had to fix a drive's file system. Not sure if it's related, I'm on Unraid 6.9.0-RC2. Anyway, when the UI would load it just wouldn't download/upload anything. I had zero peers across the board. After the stability being on-and-off it ended up getting stuck in a loop of rTorrent just crashing at start. It wouldn't start at all and just stopped working 99% of the time. I checked the rtorrent session folder and there was almost 2000 files created in the past day. I think it was stuck in a loop trying to fix itself or something.

 

After loads of debugging all week I ended up just going with a fresh install and configuration of the docker and so far it seems to be stable and working just fine. It's a pain to have to re-add all my torrents, still working on that, and that I lost all my stats. I'm just dying for this docker to feel stable again. I feel like I've spent 1000 hours debugging it the past few months.

 

So my question is once again...does it sound like I just hit some sort of performance bottleneck? Can the docker just not handle large size or large amounts of torrents? I just want to know if this sounds even remotely like a simple performance issue so I can conclude my endless debugging as: keep the torrent list low, don't flood the docker with big downloads all at once, and download beefier torrents on another desktop PC.

 

It seems similar to the issue this user had, at one point I had the same exact error (which is how I found this post):

  

On 10/29/2020 at 9:26 PM, plantsandbinary said:

My entire rTorrent has stopped working for some reason whilst downloading a large 1.6TB torrent.

 

My first drive filled up and most of my docker images crash when that happens. So I turned off all containers and moved files to the next drive.

Figured there would be no problems so I restarted the container.

 

Now I get this in the log.

 



2020-10-30 06:23:05,894 DEBG 'rutorrent-script' stderr output:
[NOTICE] [pool www] 'user' directive is ignored when FPM is not running as root
[NOTICE] [pool www] 'group' directive is ignored when FPM is not running as root

2020-10-30 06:23:05,903 DEBG 'rutorrent-script' stdout output:
[info] starting nginx...

2020-10-30 06:23:16,542 DEBG 'watchdog-script' stdout output:
[warn] Wait for rTorrent process to start aborted, too many retries

2020-10-30 06:23:16,543 DEBG 'watchdog-script' stdout output:
[warn] Failed to start rTorrent, skipping initialisation of ruTorrent Plugins...

2020-10-30 06:33:23,916 DEBG 'watchdog-script' stdout output:
[info] rTorrent listening interface IP 0.0.0.0 and VPN provider IP 10.14.0.3 different, marking for reconfigure

2020-10-30 06:33:24,772 DEBG 'watchdog-script' stdout output:
0

2020-10-30 06:33:25,099 DEBG 'watchdog-script' stderr output:
INFO: Bad data packets written to '/tmp/xmlrpc2scgi-99.xml'

2020-10-30 06:33:25,100 DEBG 'watchdog-script' stdout output:
ERROR While calling network.local_address.set('', '216.239.32.10\n216.239.34.10\n216.239.36.10\n216.239.38.10'): <Fault -503: 'Could not set local address: Name or service not known.'>

I've even rebooted my server. It was running for 415 days without issues and I must have updated this container maybe 3 or 4 times during that time because I like to update only after a good period of time.

 

Any advice is appreciated. Thank you!!

Edited by s449
Link to comment
1 hour ago, s449 said:

So my question is once again...does it sound like I just hit some sort of performance bottleneck? Can the docker just not handle large size or large amounts of torrents?

there are people using this docker image to seed thousands of torrents so by the sound of it you most probably have not hit the limit of what rtorrent/rutorrent can handle. however, having that many torrents seeding does require a reasonably powerful system, and perhaps some tweaking too, @Cat_Seeder is an old hand at this sort of thing and maybe able to tell you more about any tweaks you have to make to get things running smoothly.

 

one thing to be aware of that i have seen in the past is that btrfs can be an issue, especially when in a cache pool, im assuming you are an unraid user, if you are then you could try a single unassigned device and write to that and see what your performance is like - personally i have a dislike of btrfs and all my drives are xfs only, including my cache drive (nvme).

Link to comment
5 minutes ago, binhex said:

there are people using this docker image to seed thousands of torrents so by the sound of it you most probably have not hit the limit of what rtorrent/rutorrent can handle. however, having that many torrents seeding does require a reasonably powerful system, and perhaps some tweaking too

 

so i guess the support from my seedbox a while back was not 100% accurate and that it was more of an issue with whatever hardware they use, missing optimizations, and/or maybe even if they use btrfs. Will dedicate a couple drives for p2p share to be on and disable cache on that share to remove possible problems. Would also love to know what tricks Cat_Seeder recommends

Link to comment
42 minutes ago, binhex said:

there are people using this docker image to seed thousands of torrents so by the sound of it you most probably have not hit the limit of what rtorrent/rutorrent can handle. however, having that many torrents seeding does require a reasonably powerful system, and perhaps some tweaking too, @Cat_Seeder is an old hand at this sort of thing and maybe able to tell you more about any tweaks you have to make to get things running smoothly.

 

one thing to be aware of that i have seen in the past is that btrfs can be an issue, especially when in a cache pool, im assuming you are an unraid user, if you are then you could try a single unassigned device and write to that and see what your performance is like - personally i have a dislike of btrfs and all my drives are xfs only, including my cache drive (nvme).

 

Thank you so much for the response! Yeah I know I've read before that around 2000 torrents is where it can slow down which is why I was so confused. I know my system isn't that powerful, my CPU (Xeon E3-1230 v3) passmark's at 6806 and ruTorrent only gets half the cores and shares those cores. Although I've never seen my CPU max load due to anything going on with ruTorrent. It does go up if it's checking torrents but nothing concerning.

 

Yeah I'm an Unraid user, my shares are on xfs and my cache is btrfs. I actually have always had my media share, where my torrent downloads go, set to "No" for using the cache since I did the math and figured out that SSD is pointless if I'm writing at my internet/VPN's half gigabit speed. So torrents should be downloading to xfs file system.

 

As far as performance I noticed changing my UI update to 3000ms (vs 1000ms) helped a tiny bit for preventing timeouts. And I disabled some plugins that I was clearly not using, although that didn't seem to help honestly.

 

26 minutes ago, Cull2ArcaHeresy said:

so i guess the support from my seedbox a while back was not 100% accurate and that it was more of an issue with whatever hardware they use, missing optimizations, and/or maybe even if they use btrfs. Will dedicate a couple drives for p2p share to be on and disable cache on that share to remove possible problems. Would also love to know what tricks Cat_Seeder recommends

 

Yeah definitely leaning more towards that my hardware is just not up to what I was asking it to do.

 

I know binhex just said btrfs can be an issue in the cache pool, but I'm considering switching back to cache "prefer" so new downloads will be on the SSD vs HDD. Maybe that could help because I think my performance issues are usually when new stuff downloads. If nothing is downloading it can be a bit more stable. Couldn't hurt to try. I'll have to do more research on btrfs vs xfs for cache but I'm not sure if that's too off topic for this support thread.

 

Thanks again!

Edited by s449
Typo
Link to comment

Any reason why it needs to bind so often?

 

Client crashed randomly whilst deleting a 10GB file.

 

Here's the log:

 

Quote

2021-02-23 15:20:18,612 DEBG 'start-script' stdout output:
[info] Successfully assigned and bound incoming port '25538'

2021-02-23 15:35:18,686 DEBG 'start-script' stdout output:
[info] Successfully assigned and bound incoming port '25538'

2021-02-23 15:50:18,758 DEBG 'start-script' stdout output:
[info] Successfully assigned and bound incoming port '25538'

2021-02-23 16:05:18,839 DEBG 'start-script' stdout output:
[info] Successfully assigned and bound incoming port '25538'

2021-02-23 16:20:18,914 DEBG 'start-script' stdout output:
[info] Successfully assigned and bound incoming port '25538'

2021-02-23 16:35:18,998 DEBG 'start-script' stdout output:
[info] Successfully assigned and bound incoming port '25538'

2021-02-23 16:50:19,073 DEBG 'start-script' stdout output:
[info] Successfully assigned and bound incoming port '25538'

2021-02-23 16:52:26,887 DEBG 'rutorrent-script' stderr output:
2021/02/23 16:52:26 [error] 2144#2144: *203 upstream timed out (110: Unknown error) while reading response header from upstream, client: 192.168.1.133, server: localhost, request: "GET /plugins/autodl-irssi/getfiles.php?_=1614091794234 HTTP/1.1", upstream: "fastcgi://127.0.0.1:7777", host: "192.168.1.50:9080", referrer: "http://192.168.1.50:9080/"

2021-02-23 16:52:26,928 DEBG 'rutorrent-script' stderr output:
2021/02/23 16:52:26 [error] 2146#2146: *206 upstream timed out (110: Unknown error) while reading response header from upstream, client: 192.168.1.133, server: localhost, request: "GET /plugins/autodl-irssi/getlines.php?cid=&_=1614091794235 HTTP/1.1", upstream: "fastcgi://127.0.0.1:7777", host: "192.168.1.50:9080", referrer: "http://192.168.1.50:9080/"

2021-02-23 16:54:57,522 DEBG 'rutorrent-script' stderr output:
2021/02/23 16:54:57 [error] 2144#2144: *205 upstream timed out (110: Unknown error) while reading response header from upstream, client: 192.168.1.133, server: localhost, request: "GET /plugins/autodl-irssi/getlines.php?cid=&_=1614091794279 HTTP/1.1", upstream: "fastcgi://127.0.0.1:7777", host: "192.168.1.50:9080", referrer: "http://192.168.1.50:9080/"

2021-02-23 16:57:27,930 DEBG 'rutorrent-script' stderr output:
2021/02/23 16:57:27 [error] 2144#2144: *205 upstream timed out (110: Unknown error) while reading response header from upstream, client: 192.168.1.133, server: localhost, request: "GET /plugins/autodl-irssi/getlines.php?cid=&_=1614091794321 HTTP/1.1", upstream: "fastcgi://127.0.0.1:7777", host: "192.168.1.50:9080", referrer: "http://192.168.1.50:9080/"

2021-02-23 17:01:11,558 DEBG 'rutorrent-script' stderr output:
2021/02/23 17:01:11 [error] 2144#2144: *602 upstream timed out (110: Unknown error) while reading response header from upstream, client: 192.168.1.133, server: localhost, request: "GET /php/getplugins.php HTTP/1.1", upstream: "fastcgi://127.0.0.1:7777", host: "192.168.1.50:9080", referrer: "http://192.168.1.50:9080/"

2021-02-23 17:05:19,175 DEBG 'start-script' stdout output:
[info] Successfully assigned and bound incoming port '25538'

2021-02-23 17:20:19,303 DEBG 'start-script' stdout output:
[info] Successfully assigned and bound incoming port '25538'

 

Couldn't access the web UI at all after this:

 

Just getting an infinite loading circle when trying to load the frontend.

Link to comment
23 minutes ago, plantsandbinary said:

Any reason why it needs to bind so often?

yes its a pia constraint, they require you re-confirm (rebind) the incoming port, if you fail to do this the port will be unassigned and you will loose port forwarding, nothing i can do about it, however....

 

24 minutes ago, plantsandbinary said:

Client crashed randomly whilst deleting a 10GB file.

this is highly likely to NOT be related to the rebind, the rebind is purely a hit of the pia api, it does not change the port or reconfigure rtorrent/rutorrent, so the crash will not be related to this.

Link to comment

Alright thanks. For some reason the container is still randomly timing out after a while and taking ages to load. I have a pretty powerful server and only 1180 torrents. I've also turned off autodl-irssi. I'm going to remove some of the torrents I think and see if it manages to improve.

 

The log is utterly useless :( it's totally empty and doesn't show anything out of the ordinary except the rebinds but the ruTorrent web page just shows:  Loading...

 

The only other possible hint I have is that when this started happening, a bunch of my torrents went red (errored) and give the error:

 

Tracker: [Object operator [peers] could not find element]

Link to comment
8 minutes ago, plantsandbinary said:

Alright thanks. For some reason the container is still randomly timing out after a while and taking ages to load. I have a pretty powerful server and only 1180 torrents. I've also turned off autodl-irssi. I'm going to remove some of the torrents I think and see if it manages to improve.

 

The log is utterly useless :( it's totally empty and doesn't show anything out of the ordinary except the rebinds but the ruTorrent web page just shows:  Loading...

 

The only other possible hint I have is that when this started happening, a bunch of my torrents went red (errored) and give the error:

 

Tracker: [Object operator [peers] could not find element]

a question that may sound random (but isnt), are you by any chance using a cache pool? or have your cache drive formatted to BTRFS?, if so this maybe part of the issue, BTRFS is a wee bit crap when it comes to disk I/O and can cause all sorts of fun and games when using a torrent client, this is double so when using a cache pool, i have had much better experience overall with XFS, but obviously you cannnot (currently) create a XFS cache pool on unraid, so you are then limited to a single cache device, or multi pool single cache devices in 6.9.x

Link to comment

Thanks for this great docker, binhex!

 

I'm using this on two machines. a synology and in ubuntu. Using PIA

It's working but not port forwarding. Is it supposed to work without the usual port forwarding you have to do from your router? If so I'm must be overseeing somthing as the port icon on the bottom of rutorrent stays red with the hover message "Port is closed".

 

Can you advise?

Link to comment

I have some other dockers using the binhex-rtorrentvpn docker for VPN, and cannot access their UIs. My workflow had been to add a port variable to the binhex docker with the UI port of the other docker (following a SpaceInvader video I think). Summarizing Q25 from the FAQ, add ports to 'ADDITIONAL_PORTS' env var value for the VPN container," should I delete all those port entries? I have also updated this docker, but don't see the ADDITIONAL_PORTS env variable - just add it?

 

edit: Just adding the variable worked. Also had to go into each docker and change config from IP to localhost (Q24).

Edited by tiphae
Link to comment

Latest release on synology doesn't work (in Synology Docker)
Container is in a constant reboot loop with error: FATAL: Kernel too old

 

Tried on a DS1817+ on both Synology DSM 6.2 as DSM 7 Beta.

 

Any idea on how to fix this?

 

 

EDIT: reverted to rtorrent-ps-1.1.r54.ga787dd9-1-30 and that runs fine.

Edited by Borkjev
extra info
Link to comment
On 2/25/2021 at 7:02 PM, binhex said:

indeed!, just to be clear read Q15:- https://github.com/binhex/documentation/blob/master/docker/faq/vpn.md

 

so in order to debug this please do the following:- https://github.com/binhex/documentation/blob/master/docker/faq/help.md

Thanks! 

 

I managed to get things going. I removed a port I had set up in rtorrent.rc when testing with another vpn provider. Once that was gone the port forwarding started working as expected.

 

Is it in any way problematic to have other containers sharing the network stack of rtorrentvpn to utilize the vpn connection?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.