unRaid and Seedbox integration


Recommended Posts

 

I'm wondering how the unRaiders out there are integrating their Seedboxes with unRaid home server.

 

My unRaid server is my workhorse and it does a fine job automating downloads with Sickbeard, Sabnzbd and Couchpotato. But I am a member of a few private trackers and I use a seedbox for downloads. Unfortunately, getting this integrated with unraid hasn't been easy for me. Right now, here is my workflow:

 

1. Manually add .torrent files from the tracker to my seedbox via its web gui.

2. Wait for files to be downloaded to the seedbox.

3. Use an FTP client (CuteFTP, Filezilla) to transfer downloads to my home PC.

4. Rename files in my home PC and copy it to the correct network share on my unraid home server.

 

Does anyone have a more streamlined way to accomplish this. I've thought about a few mechanisms but don't lack the know how to impement it seamlessly.

 

OPTION 1: USE COUCHPOTATO/SICKBEARD TO PROCESS TORRENTS

This works well in theory, but I don't think it has been implemented. My issue is that this process will make it difficult to seed downloaded torrents since the post-process system will make the files no longer available to the torrent client. This also seems a bit cumbersome although in theory, this is perhaps the most elegant way to make it work. I'm curious to find out if anyone has this mechanism working.

 

OPTION 2: USE A SERVER-SIDE FTP CLIENT

I'm not aware of an FTP client that can be used to FTP files directly from the seedbox onto an unraid share (like the cache drive), but if such an implementation existed, it might be amazing. I've thought about two ways to get around this but haven't had success with either.

 

#1 -- Using RSYNC to sync remote folder (on seedbox) with temporary share (on cache drive). I don't know enough linux to get this to run on a regular basis. If anyone can suggest, I'd be very grateful.

 

#2 -- Using BitTorrent Sync (BTSYNC) -- unraid has a BTSYNC plugin and I thought another option might be to use unraid to sync the seedbox to a temporary share on the cache drive, which I can then use to process files and move out of the cache drive.  The problem with this is that there is a small bug with BTSYNC that keeps from completing the sync process. My synchronization stops halfway. The folks at BTSYNC think that they maybe due to a permissions issue.

 

If anyone else has this setup working well (i.e. Torrenting via Seedbox to unraid) I would love to hear how they implemented their workflow.

 

Link to comment
  • 5 weeks later...

Hey rwickra,

 

I too had problems with the way to go about this - to add torrents I use 'Remote Torrent Adder' chrome extension and set it up for my Deluge Seedbox.

I have my seedbox move once completed into the same directory structure as I have on my Unraid box.

 

To get the files i ended up writing 2 scripts to using lftp and the mirror option which runs every 15 minutes from my Unraid box.

 

I also had a problem with permissions once downloaded so in the script I have it change permissions of the Seedbox before downloading.

 

Basically it runs, locks a file on the disk so it won't run again while it's still downloading and downloads 5 parts of each file simultaneously (which thoroughly maxes out my connection)

 

I have attached the 2 files - both go into /boot/custom or flash/custom

I have this line in my go file to add the cronjob at start up:

# FTP Downloader

cat /boot/custom/lftpcron >> /var/spool/cron/crontabs/root

 

I hope they make sense, you will need to adjust the synctorrents.sh script to match your username, password, seedbox ftp address and the directory structures you use (have added comments to help you in the file)

 

You will also need the lftp package in the extra folder ( on Unraid 5+ i used lftp-4.3.8-i486-1.txz and now on Unraid 6+ I am using lftp-4.4.9-x86_64-1.txz) just google those and you will find them.

 

If I was smarter it could be made into a plugin - but sadly i haven't got those smarts!

 

Attachments need to be renamed to lftpcron.    and synctorrents.sh - remove the .txt's

 

Good luck

lftpcron.txt

synctorrents.sh.txt

Link to comment

hey drnackers,

 

thank you so much!! this is exactly what i was looking for. yes, a plugin maybe awesome -- and who knows someone might develop it, but it seems like most unraid-ers are focused on optimizing usenet access since torrents are so unreliable (trackers getting shut down left and right).

 

i will give this a quick spin once my server is up and running again.. i'm having some major issues getting my system running, but once it is up to speed, i'll be checking your script out. thank you for sharing it.

Link to comment

Heres how i did mine and automated it.

 

On your unraid box add the following scripts "putfileswatched.sh" (add this to a cron 1 per hr)  This uploads all of the .torrent files on your unraid machine to the seedbox.

 

#!/bin/bash
login=pyrater
pass=******
host=***********
remote_dir=/torrents/watch
local_dir=/mnt/user/Data/Misc/Downloading/Blackhole

trap "rm -f /tmp/syncwatched.lock" SIGINT SIGTERM
if [ -e /tmp/syncwatched.lock ]
then
  echo "syncwatched is running already."
  exit 1
else
  touch /tmp/syncwatched.lock

lftp  << EOF
set ftp:ssl-protect-data yes
set ftp:ssl-protect-list yes
set mirror:use-pget-n 2
lftp -u $login,$pass $host
mirror -c -P5 --Remove-source-files $remote_dir $local_dir 
quit
EOF

  rm -f /tmp/syncwatched.lock
  rm -f /mnt/user/Data/Misc/Downloading/Blackhole/*.torrent
  exit 0
fi

 

 

synctorrents.sh - this downloads all of the completed torrents from your seedbox. (using Deluge hardlinked for completed downloads)

 

#!/bin/bash
login=pyrater
pass=*******
host=*******
remote_dir=/torrents/hardlinked
local_dir=/mnt/user/Data/Misc/Downloading/Downloaded

trap "rm -f /tmp/synctorrent.lock" SIGINT SIGTERM
if [ -e /tmp/synctorrent.lock ]
then
  echo "Synctorrent is running already."
  exit 1
else
  touch /tmp/synctorrent.lock
  

lftp  << EOF
set ftp:ssl-protect-data yes
set ftp:ssl-protect-list yes
set mirror:use-pget-n 2
lftp -u $login,$pass $host
mirror -c -P5 --Remove-source-files $remote_dir $local_dir 
quit
EOF
  rm -f /tmp/synctorrent.lock
  chown -hR nobody /mnt/user/Data/Misc/Downloading/Downloaded/*
  chmod -R 777 /mnt/user/Data/Misc/Downloading/Downloaded/*
  /boot/extra/Scripts/unrarall -s --clean=all /mnt/user/Data/Misc/Downloading/Downloaded/
  exit 0
fi

 

 

End result = A copy of the media at both locations, one for you on your unraid machine for viewing, one on your seedbox for seeding.

unrarall.zip

Link to comment
  • 4 months later...

I am trying to implement this but I'm having a hard time of it.....mostly because this is my first experience using the CLI.

I have followed (drnackers) instructions and to "test it" I'm trying to run it using this command:

root@UNRAID:/boot/custom#  ./synctorrents.sh

 

but I am getting this error:

./synctorrents.sh: line 21: lftp: command not found

 

any ideas?

Link to comment

Try as I might I still can't get this to work  :-[ . There has been progress though, I no longer get the lftp or the libssl.so.1 errors I was getting, but now I am getting this:

mirror: Access failed: No such file (/mnt/user/Other/test_02)

The odd thing is if I use this simple command it works fine:

rsync -a -v -e ssh [email protected]:/home/denzoid/test_02/ /mnt/user/Other/test_02

 

But if I have the same directories in the script I get the "Access failed" error. Here's the script:

#!/bin/sh

login="denzoid"
pass="1234567"
host="sftp://ceedbox.com"

trap "rm -f /tmp/synctorrent.lock" SIGINT SIGTERM
if [ -e /tmp/synctorrent.lock ]
then
echo "Synctorrent is running already."
exit 1
else
touch /tmp/synctorrent.lock

lftp <<EOF
set ftp:ssl-auth TLS
set ftp:ssl-force true
set ftp:ssl-protect-list yes
set ftp:ssl-protect-data yes
set ssl:verify-certificate off
lftp -u $login,$pass $host
mirror -c --use-pget-n=5 --only-missing --log=/var/log/synctorrents_media.log /home/denzoid/test_02/
mirror -c --use-pget-n=5 --only-missing --log=/var/log/synctorrents_downloads.log /mnt/user/Other/test_02

quit 0
EOF

rm -f /tmp/synctorrent.lock
trap - SIGINT SIGTERM
exit 0
fi

 

I did strip out the permission lines from the original script because they caused (chmod) errors themselves.

I also ran the "New Permissions" utility at some point. I am making the edits to the script using Notepad++. I've once again tried everything I could think of,but to no avail.

What else should I try?

 

TIA,

denzoid

Link to comment

Try as I might I still can't get this to work  :-[ . There has been progress though, I no longer get the lftp or the libssl.so.1 errors I was getting, but now I am getting this:

mirror: Access failed: No such file (/mnt/user/Other/test_02)

The odd thing is if I use this simple command it works fine:

rsync -a -v -e ssh [email protected]:/home/denzoid/test_02/ /mnt/user/Other/test_02

 

But if I have the same directories in the script I get the "Access failed" error. Here's the script:

#!/bin/sh

login="denzoid"
pass="1234567"
host="sftp://ceedbox.com"

trap "rm -f /tmp/synctorrent.lock" SIGINT SIGTERM
if [ -e /tmp/synctorrent.lock ]
then
echo "Synctorrent is running already."
exit 1
else
touch /tmp/synctorrent.lock

lftp <<EOF
set ftp:ssl-auth TLS
set ftp:ssl-force true
set ftp:ssl-protect-list yes
set ftp:ssl-protect-data yes
set ssl:verify-certificate off
lftp -u $login,$pass $host
mirror -c --use-pget-n=5 --only-missing --log=/var/log/synctorrents_media.log /home/denzoid/test_02/
mirror -c --use-pget-n=5 --only-missing --log=/var/log/synctorrents_downloads.log /mnt/user/Other/test_02

quit 0
EOF

rm -f /tmp/synctorrent.lock
trap - SIGINT SIGTERM
exit 0
fi

 

I did strip out the permission lines from the original script because they caused (chmod) errors themselves.

I also ran the "New Permissions" utility at some point. I am making the edits to the script using Notepad++. I've once again tried everything I could think of,but to no avail.

What else should I try?

 

TIA,

denzoid

 

hey denzoid,

 

looks like your script only has the host folder not the target folder in these lines:

mirror -c --use-pget-n=5 --only-missing --log=/var/log/synctorrents_media.log /home/denzoid/test_02/

mirror -c --use-pget-n=5 --only-missing --log=/var/log/synctorrents_downloads.log /mnt/user/Other/test_02

 

My one has both host and target folders:

 

mirror -c --use-pget-n=5 --only-missing --log=/var/log/synctorrents_downloads.log /Downloads/ /mnt/user/Torrents

 

Syntax:

mirror (COMMANDS) --log=(YOUR LOG DESTINATION) /(SOURCE) /(Destination)

 

Hope that helps

 

 

 

Link to comment
  • 2 weeks later...

Legends!

 

I have been searching for WEEKS for something like this! Great work @drnackers @pyrater!

 

I am currently pre-clearing my drives, and once it is done I will give this script a shot...Just need to read up a little on this stuff as I am fairly new to linux. I know that I have to use notepad++ to compile/edit the script, any other tips?

 

Few questions  @drnackers  @pyrater + anyone else who will kindly help :)

 

1) I would like to install the lftp to my ssd cache drive and not the flash (boot), Do I just need to create the "custom" folder on the ssd and put the files in there?

 

2) With Rar'd downloads - If I use the "unpack" feature on rutorrent (remotely), and set the unpack folder to "the torrents current dir" .....won't i get a hardlink containing BOTH the rar;d and unrar;d content in the sync folder which will be sent over to my NAS (double the data)?  Is there a way around this?

 

3) Because the hardlink is still present on the seedbox, wont it re-download it on its next cycle as the movie on local nas would have moved from its processing folder to my Movies share? Is there a way of removing source files?

tl;dr - is there a way of deleting source files after downloading using lftp to prevent it from redownloading?

 

4t0L5nA.png

 

Many thanks!

Link to comment
  • 7 months later...

Just in case anyone else runs across the issue I had....

 

I used the lftp scripts, but they didn't work for me.  chmod wasn't supported by my provider and every torrent that was uploaded didn't have the right permissions.  Rtorrent would try to load it, but wasn't able to hash and it would fail.

 

The answer, for me, turned out to be very simple.  I added "-a" (without quotes) to the

This.  Also, using reverse mirror worked out better.

 

mirror -c [b]-R -a[/b] --use-pget-n=5 --only-missing --log=/var/log/synctorrents_downloads.log /mnt/user/temp/torrents/blackhole /blackhole/

 

Before the -a, I was getting the same errors around directories not existing, access denied, etc...

 

for the torrents directory, I also use --Remove-source-files to remove the torrents after they have been uploaded so they don't get uploaded again.

 

For the mirror that pulls the media, I have headphones, couchpotato, or sonar processing which in turn moves the media from the temp directories to my media shares that plex can access.

 

Hope this helps someone as I banged my head on this for 2 days.  Still having an issue with directory structure and autolabel but that's easier to troubleshoot.

 

http://lftp.yar.ru/lftp-man.html

 

 

Link to comment
  • 2 years later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.