z0ki Posted January 8, 2020 Share Posted January 8, 2020 (edited) Hey guys I am wanting to use LFTP on my unRaid server so I can easily grab files from a seedbox. I downloaded LFTP from the NerdTools (lftp-4.8.4-x86_64-1.txz) so what happens now? I just assumed it'll be installed and I can open Terminal from my unRaid homepage type LFTP (hit enter) and login to my seedbox that way. Sure enough LFTP shows up in terminal, and I can login to my server (so it seems) but when typing LS to view my directory it just sits at `ls' at 0 [Connecting...] So maybe there is actual extra steps needed after grabbing LFTP from the NerdTools that I am now clear on? Could someone please give me step by step instructions in 'laymens' terms for me please? Thanks guys Edited January 10, 2020 by z0ki Quote Link to comment
z0ki Posted January 8, 2020 Author Share Posted January 8, 2020 1 minute ago, trurl said: This part is unclear and makes me wonder how you are accessing "terminal" on your server. Sorry I meant via the command terminal on unRaid's home page? Quote Link to comment
trurl Posted January 8, 2020 Share Posted January 8, 2020 And in what way does LFTP "show up" there? Quote Link to comment
trurl Posted January 8, 2020 Share Posted January 8, 2020 OK I see. I have not used that FTP before. Quote Link to comment
z0ki Posted January 8, 2020 Author Share Posted January 8, 2020 This is what happens. I did look at a number of posts on here about it but nothing concrete on how they got it to actually work. Quote Link to comment
ken-ji Posted January 8, 2020 Share Posted January 8, 2020 are you able to ssh to that server from unraid? Quote Link to comment
z0ki Posted January 8, 2020 Author Share Posted January 8, 2020 11 hours ago, ken-ji said: are you able to ssh to that server from unraid? Hi @ken-ji I just tested this and yes I can ssh into my seedbox via Terminal in unRaid. Quote Link to comment
ken-ji Posted January 8, 2020 Share Posted January 8, 2020 When you ssh to your seedbox, do you keep getting prompted to save the fingerprint like above? Because if lftp tries to connect to via sftp and a prompt about the fingerprint tries to show, the lftp session will get stuck. you can try lftp -d sftp://.... command and see how the underlying ssh connection is having issues. 1 Quote Link to comment
z0ki Posted January 8, 2020 Author Share Posted January 8, 2020 15 minutes ago, ken-ji said: When you ssh to your seedbox, do you keep getting prompted to save the fingerprint like above? Because if lftp tries to connect to via sftp and a prompt about the fingerprint tries to show, the lftp session will get stuck. you can try lftp -d sftp://.... command and see how the underlying ssh connection is having issues. I only got prompted once to say key/fingerprint yes/no on my first attempted. I just tried with lftp -d sftp:// and it worked! I can finally login via LFTP and browse all of my folders. I haven't attempted to download anything yet, but how would I download to a specific folder in my array? In windows when I used LFTP I used the command "lcd /cygdrive/c/Users/username/Downloads" for example, but how would I do this with unraid's terminal? Same principle? Quote Link to comment
z0ki Posted January 9, 2020 Author Share Posted January 9, 2020 Ok. So I got it to work One thing I am curious about is my terminal window looks like this after starting the segmented downloading to my array (see video) Normally you could press Ctrl+C to cancel a transfer but you can't. Any way around this? Even closing the terminal window the transfer is still going which is great but would be nice to cancel. Untitled.mp4 Quote Link to comment
z0ki Posted January 9, 2020 Author Share Posted January 9, 2020 (edited) Never mind this actually seems to be normal behavior. Here is a quick tutorial/walk-through on how I did it and how to grab files from your seedbox that will then be downloaded to my unRaid box. Also thanks to @ken-ji for getting me over my main hurdle. ___________________________________________________________________________________ 1) Download NerdPack GUI from CA 2) Open NerdPack via the Plugins Tab 3) Search for LFTP (latest version since this post is lftp-4.8.4-x86_64-1.txz) 4) Open unRaids Terminal 5) Type lftp -d sftp://seedboxusername:seedboxpassword@your.host.com 6) Once connected you can then use commands such as LS to view directory and CD Directory Name to change directory 7) You can choose what folder your wish to download the file(s) to. For example: typing lcd /mnt/user/Media/"TV Shows"/"Show Name"/"Season 1" then hit enter. Notice the quotation marks? You need this whenever a directory has a space in it. You can then go to the directory in LFTP where your content is stored on your seedbox. By typing something like cd /home/deluge/torrents Type LS and it will show you what is inside that directory path 9) To download an entire folder your can simply type mirror --use-pget-n=8 Then.the.name.of.your.folder The number in the command --use-pget-n=8 means you will download a total of 8 segments. This will allow you to download at your maximum connection speed. Hit enter and the folder will begin to download. 10) If you wish to simply only download a single file then you would change directory by going into the actual folder where the file resides that you wish to download. Then you will type pget -n 8 then.the.file.name.mkv (hit enter) and it will start to download. And that is all _____________________________________________ I am sure there is a automated way for this to work with a script but that is way above my knowledge base. If anyone knows of a better easier way with a script then please let me know. As writing all of those commands will get tedious very fast Edited January 9, 2020 by z0ki Quote Link to comment
ken-ji Posted January 9, 2020 Share Posted January 9, 2020 you should be able to skip "-d" as this is the debug option hence all the other stuff on your terminal. and you can use lftp like: lftp sftp://user:password@server:/path -c "mirror folder" lftp sftp://user:password@server:/path -c "get file" And for security's sake, use keyfiles to login to your seedbox. You can refer to this thread for passwordless logins to other servers - its a bit long but cover all the cases people have encountered so far to allow unattended ssh logins from unraid to any other server. 1 Quote Link to comment
z0ki Posted January 9, 2020 Author Share Posted January 9, 2020 you should be able to skip "-d" as this is the debug option hence all the other stuff on your terminal. and you can use lftp like: lftp sftp://user:password@server:/path -c "mirror folder"lftp sftp://user:password@server:/path -c "get file" Actually I am finding that there is some permission issues after using LFTP. I cannot delete or move the files I’ve downloaded via LFTP as it says I do not have the correct permissions.Is there something I can add to the command line that will not add permissions? Quote Link to comment
ken-ji Posted January 9, 2020 Share Posted January 9, 2020 Well permission issues are expected, since lftp will be running as the root user, and therefore be creating the files as root. You can't add to the command line, but you might be able to either run lftp as the either your user (or the default nobody) via su -s /bin/sh - nobody -c "lftp ..." but this will definitely prompt you for the seedbox hostkey again as your are now running as a different user. Another option is to run this command chown -Rv nobody:users /path/to/downloaded/files.and.directories chmod -Rv 770 /path/to/downloaded/files.and.directories A lot of users here prefer 777 on the chmod command, but I'd rather use 770 Quote Link to comment
z0ki Posted January 10, 2020 Author Share Posted January 10, 2020 (edited) 9 hours ago, ken-ji said: Well permission issues are expected, since lftp will be running as the root user, and therefore be creating the files as root. You can't add to the command line, but you might be able to either run lftp as the either your user (or the default nobody) via su -s /bin/sh - nobody -c "lftp ..." but this will definitely prompt you for the seedbox hostkey again as your are now running as a different user. Another option is to run this command chown -Rv nobody:users /path/to/downloaded/files.and.directories chmod -Rv 770 /path/to/downloaded/files.and.directories A lot of users here prefer 777 on the chmod command, but I'd rather use 770 Hmmm well I did try chmod -Rv 770 downloads/Folder.Name.Of.Content.On.SeedBox It resulted in the this Mode of `downloads/Folder.name.of.content.on.seedbox' changed to 0770 (rwxrwx---). I then proceeded to download the content via mirror --use-pget-n=8 The files were downloaded as normal, but permissions did not change. I am still unable to move or delete the files. Edited January 10, 2020 by z0ki Quote Link to comment
ken-ji Posted January 10, 2020 Share Posted January 10, 2020 You run the chmod and chown after you exited lftp. Not inside lftp. Running inside lftp changes the permissions of the files on the seedbox. Quote Link to comment
z0ki Posted January 10, 2020 Author Share Posted January 10, 2020 16 minutes ago, ken-ji said: You run the chmod and chown after you exited lftp. Not inside lftp. Running inside lftp changes the permissions of the files on the seedbox. Well doing it after exiting lftp I ran the command but it completely locked me out from even opening it (read permissions seemed to be gone) I then tested with chmod ugo+rwx foldername/path and now I can click into the folder, and delete it, but I cannot move it as it will throw the "You need permissions to perform this action" Thanks for the help too @ken-ji I'm just trying to learn along the way Quote Link to comment
z0ki Posted January 11, 2020 Author Share Posted January 11, 2020 (edited) Ok! I am an idiot! 😂 I had to run chmod -Rv 770 AND chown -Rv! I was only doing chmod -Rv 770! So this little script I did consists of the following. With the two lines for chmod & chown added at the end I can now edit, move, delete freely. #!/bin/bash lftp << EOF set ssl:verify-certificate no set sftp:auto-confirm yes open sftp://username:[email protected] mirror --use-pget-n=8 /home/username/downloads/deluge /mnt/disk5/Downloads/completed quit EOF chmod -Rv 770 /mnt/disk5/Downloads/completed/ chown -Rv nobody:users /mnt/disk5/Downloads/completed/ How does it look? Is there anything I am missing here? Ideally this is what I would like to happen with " LFTP automation" as such, but I know this is going to be complicated for someone like me with beyond limited knowledge about this stuff haha. I would like to have one script to download 4K files from my seedbox, and put them into a 4K directory on my array. Then another script to grab TV, and dump them into a TV folder on my unRaid. Is something like this possible? Edited January 11, 2020 by z0ki Quote Link to comment
ken-ji Posted January 11, 2020 Share Posted January 11, 2020 Easy. You can write 1 script for the whole thing as long as its possible to differentiate 4k shows from TV shows - like having separate directories for each. If you don't - you'll some criiteria to tell them apart and we can create the script lines. #!/bin/bash lftp << EOF set ssl:verify-certificate no set sftp:auto-confirm yes open sftp://username:[email protected] mirror --use-pget-n=8 /home/username/downloads/deluge/4k /mnt/disk5/Downloads/incoming/4k mirror --use-pget-n=8 /home/username/downloads/deluge/TV /mnt/disk5/Downloads/incoming/TV quit EOF chmod -Rv 770 /mnt/disk5/Downloads/incoming/4k chown -Rv nobody:users /mnt/disk5/Downloads/incoming/4k chmod -Rv 770 /mnt/disk5/Downloads/incoming/TV chown -Rv nobody:users /mnt/disk5/Downloads/incoming/TV mv /mnt/disk5/Downloads/incoming/4k/* /mnt/disk5/Downloads/completed/4k mv /mnt/disk5/Downloads/incoming/TV/* /mnt/disk5/Downloads/completed/TV I've created an incoming directory in my script above since you might not want to run chmod and chown on all the files in the 4k and TV folders, which take forvever ion my system because the folders have a whole ton of assorted files and I only sort and remove every so often. 1 Quote Link to comment
z0ki Posted January 13, 2020 Author Share Posted January 13, 2020 Perfect @ken-ji! I actually decided i'll do TV manually as I don't really get a whole lot of TV Shows anyway as i'm quite selective with what shows I like. So I went ahead and went with this.. #!/bin/bash lftp << EOF set ssl:verify-certificate no set sftp:auto-confirm yes open sftp://user:[email protected] mirror --use-pget-n=8 /home/user/downloads/deluge/ /mnt/cache/Downloads/incoming/"4K Movies" quit EOF chmod -Rv 770 /mnt/cache/Downloads/incoming/"4K Movies" chown -Rv nobody:users /mnt/cache/Downloads/incoming/"4K Movies" mv /mnt/cache/Downloads/incoming/"4K Movies"/* /mnt/disk3/Media2/"4K Movies" Now one thing I really want to do is get the filenames of folders sorted during the move. Is this possible? I was hoping to do this on my seedbox but it looks like I can't. So what I was hoping for it to do was. Download to seedbox > folder gets renamed > continues to seed (hardlinks) > moved to unRaid > moved to appropriate folder. Now Radarr does exactly this in the way of keeping it seeding with a renamed folder but the main issue is I need to do it manually. Anyway around this? Quote Link to comment
ken-ji Posted January 13, 2020 Share Posted January 13, 2020 2 minutes ago, z0ki said: Now one thing I really want to do is get the filenames of folders sorted during the move. Is this possible? I'm not even sure what the benefit of this sorting operation is... Is there a specific effect that doing this will do? 3 minutes ago, z0ki said: So what I was hoping for it to do was. Download to seedbox > folder gets renamed > continues to seed (hardlinks) > moved to unRaid > moved to appropriate folder. I don't use deluge or a seedbox, but transmission on my Unraid server, and I wrote a script to create hardlinks in another directory, which Emby and Kodi look at. So I can move any downloaded files out to the proper directory, and just run another script daily to remove any file without any links remaining (when the downloaded files have been moved). I can't / don't use automation, as I'm matching against an unsupported database and need to do some filename cleanup / season reassign/renumber. Quote Link to comment
z0ki Posted January 18, 2020 Author Share Posted January 18, 2020 Just wanted to say thanks @ken-ji I got everthing setup the way i wanted it (inclding with Filebot) on my seedbox. Everything downloaded to my seedbox is now renamed, and then fetched by my LFTP script on my unraid box and then is put in it's appropriate folder :) I do have a question though. I setup a scheduled time for my LFTP script to run, but I notice it doesn't stop if it's not pulling anything from my seedbox. Is it possible to have the script run and if it doesn't find anything after say 5 minutes that the script aborts, then tries again in say 60 minutes? Quote Link to comment
ken-ji Posted January 19, 2020 Share Posted January 19, 2020 @z0ki Odd, i'm not sure why your LFTP is not exitting. as that's what the commands for it say. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.