Jump to content
SlrG

ProFTPD Plugin for unRAID v6.3.5

523 posts in this topic Last Reply

Recommended Posts

@rmeaux

Cool. I didn't know about this plugin yet. Thank you for posting. If I get the time I'll take a look at how it does things. Maybe I'll be able to get my the proftpd config editor working again that way. It has the benefit of syntax checking the proftpd.conf file and pointing out errors, so I was sad it stopped working.

Share this post


Link to post
4 hours ago, SlrG said:

If you still have both icons in your settings, its a bit odd.

 

the apache plugin webserver stopped running with the latest versions and will get no update. 

 

 

@SlrG

I only have the Proftpd icon in settings.  But in your comment to Meaux you said the Apache plugin wasn't working and you can't edit the file.  I edited my config just to test and it looked liked it edited, but then got an error saving.  Is that the current behavior?  If the Apache Plugin is not working properly, could it be the cause of the problems I'm having?

 

I've deleted the plugin from my system and disabled the reboot cron.  I'll let you know if that solves my problem.

 

I see there are a couple other Apache plugins out there.  Do you recommend using them?  Or is the editor Meaux is using be your new recommendation?

Edited by jeffreywhunter

Share this post


Link to post

@jeffreywhunter

The last time I tested the apache plugin it didn't start at all. I don't think it could be the cause of your problems, but I would recommend to uninstall it anyway. The plugin rmeaux suggested would probably be my recommendation for the time being. The other apache "plugins" you mentioned are probably dockers and not plugins. All of those I tested could not access the config files and didn't work for the purpose of my plugin. It makes sense from a security viewpoint, so I don't think it has changed.

Share this post


Link to post

@SlrG - So I've built a little log using User Scripts.  Works really well.  In trying to uncover what's causing my server to crash and trying to eliminate ProFTPd as related to the source.  What I'm finding (log attached) is that the problem appears at night, while backups are running.  I have about 10 jobs that backup via ProFTPd.  They start around midnight and end around 6am.  If you look at my log, you see that during the day, memory usage is pretty stable.  But at night, when the backup jobs run, memory usage expands until (and this is frustrating, SOMETIMES) it crashes.  Perhaps it has to do with how much data movement?  No idea how to diagnose that. 

 

One other important note.  It appears it often crashes on the job that backups my workstation image.  It can be anything from a small file (48mb) to a large file (10gb) depending on changes, etc.  I have a 512gb cache drive, so that shouldn't be a problem.

 

Perhaps this log will shed some light.  Perhaps there are some additional commands I can add to my log to help diagnose the cause?  Not saying its ProFTPd, but maybe something related to backups?

 

My log script is pretty simple:

 

#!/bin/bash
echo '======Start=======' >> /boot/log_free_memory.log
date >> /boot/log_free_memory.log
free -h >> /boot/log_free_memory.log
echo '======End=======' >> /boot/log_free_memory.log

And I run it hourly.  In the attached log, the server crashed after the 4:47AM log dump.  I was out all day and didn't get the server restarted until 4PM.

 

Thanks in advance for taking a look!  log_free_memory.log

Edited by jeffreywhunter

Share this post


Link to post

@jeffreywhunter

I really want to help you solve your problem. But please do as I say and remove my proftpd plugin and see if the server still crashes. Then we will know for sure it is related to proftpd and can start digging in only that direction. Do you use ftp for other purposes as your personal backups? Because if only yourself are accessing your server, you don't need to jail users. In that case you could use unRAIDs internal ftp to keep your backup jobs running while the plugin is gone.

 

Looking at your log and seeing it is hourly, the memory goes down, but in the last log entry before the crash it is back to 6.6 gig again. Maybe if you increase the logging interval to every 5 minutes or minute we will get a better view. Also it could be helpful to increase logging in proftpd as described here and reenable the transfer log as described here.

Share this post


Link to post
On 5/19/2018 at 1:44 AM, SlrG said:

@jeffreywhunter

I really want to help you solve your problem. But please do as I say and remove my proftpd plugin and see if the server still crashes. Then we will know for sure it is related to proftpd and can start digging in only that direction. Do you use ftp for other purposes as your personal backups? Because if only yourself are accessing your server, you don't need to jail users. In that case you could use unRAIDs internal ftp to keep your backup jobs running while the plugin is gone.

 

Looking at your log and seeing it is hourly, the memory goes down, but in the last log entry before the crash it is back to 6.6 gig again. Maybe if you increase the logging interval to every 5 minutes or minute we will get a better view. Also it could be helpful to increase logging in proftpd as described here and reenable the transfer log as described here.

 

Ok, I've followed your directions.  I've been up for more than 4 days.  Far longer than I've ever gone (usually crashes within 24 hours).  I think it has to do with FTP file transfers.  As I've mentioned I have about 12 goodsync FTP jobs that run nightly to backup various directories on my PC and such.  So, evidently, on my system, something about ProFTPd causes my system to crash.  What diagnostics can I conduct to isolate the problem?

 

FYI, I've included my latest diagnostics.

 

Thanks in advance!

hunternas-diagnostics-20180524-1644.zip

Edited by jeffreywhunter

Share this post


Link to post

@jeffreywhunter

As you don't crash without proftpd anymore it indeed seems to be a problem of proftpd in conjunction with your backup jobs. In my last post I gave you two links that will help you to increase the logging of proftpd. Maybe we will get some new information from that.

Share this post


Link to post

Ok, I'll reinstall and try that.  Will keep you posted.  In thinking about how a backup job might work.  Is there some aspect of backing up via FTP that could cause ProFTPd to have a problem/memory leak over time?  On a nightly basis I might backup 1000 files.  Mostly documents, sometimes music/video files.  If you had an idea, I could build a test case that would push the boundaries...

Share this post


Link to post

Well I'm not the developer of ProFTPd, I merely made it available on the unRAID platform. So I really can't answer you question. For me it is running fine, but I use it very seldom and for few and small files only. Maybe we can see something from the new debug logs. If not, I fear I can't help and you'll have to ask the ProFTPd developers themselves. :(

 

Share this post


Link to post

Can anybody help me with setting up FTPS or SFTP with ProFTPd? I cant seem to find an easy working guide out there

Share this post


Link to post

I apologise if this has been answered before but i cannot find the answer I have searched.

Is there an FTP client plugin for unraid as i prefer only to use plugins and not containers.

 

Thank You for ANY help with an FTP client plugin.

Share this post


Link to post

AFAIK there is no client plugin. Most clients have a graphical interface and thus need a lot of additional packages, which would greatly increase the possibility of breaking unRAID base functionality and making the system unstable. So the only way is to use a docker one or run the client of your choice in a VM.

Share this post


Link to post
12 hours ago, SlrG said:

AFAIK there is no client plugin. Most clients have a graphical interface and thus need a lot of additional packages, which would greatly increase the possibility of breaking unRAID base functionality and making the system unstable. So the only way is to use a docker one or run the client of your choice in a VM.

thank you

 

makes sense I understand

Share this post


Link to post

I just updated to unRAID 6.6.0 and the proFTPd settings page is a bit unreadable. I don't know if it's on your end, @SlrG or if it's something to do with @bonienl's excellent new WebGUI, but they're not playing nice together:

1082012259_2018-09-2419_25_30-NAS_ProFTPd.thumb.png.e22c4247193e0e55319fae487448ac3d.png

Share this post


Link to post

By highlighting the text, of course, I was able to read it all and duplicate the settings from my primary server to my new backup server.

 

In trying to connect to the FTP server on my backup server from the Duplicati docker running on my primary server, Duplicati is asking me to confirm the SSH-RSA2048 key. Where do I go to find the key to confirm it's the right one?

 

I can't imagine it's not, but it's always good to be 100% certain, plus it'll be handy for future reference.

Share this post


Link to post

@FreeMan

I'm still on 6.5 on my server, so I did not notice the problems until now. Thank you for reporting. I'll have a look and try to fix it.

 

The SSH key should be located in /etc/ssh, if the unRAID ssh key is used it is probably the file ssh_host_key.pub. If not, its probably one of the others located there.

Share this post


Link to post
15 hours ago, SlrG said:

The SSH key should be located in /etc/ssh, if the unRAID ssh key is used it is probably the file ssh_host_key.pub. If not, its probably one of the others located there.

I don't see anything in any of those files that looks remotely like the key I'm being presented with. That key is a bunch of hex pairs like this. Any additional thoughts?

 

2018-09-25 18_01_16-91d370e4e434 - Duplicati.png

Share this post


Link to post

That is an fingerprint md5 hash of a rsa key. You have to open the unRAID shell and issue the following command:

ssh-keygen -E md5 -lf /etc/ssh/ssh_host_rsa_key

It will show you the fingerprint md5 hash of the key stored in the file ssh_host_rsa_key. Change it if necessary to get the info it for the other keys.

Share this post


Link to post

Thanks!

As you can tell, I'm not 100% up to speed on all this, but I'm learning...

Sent from Tapatalk

Share this post


Link to post

Just wanting to make sure I am setting this correctly. In the mount script

 

I would like to have two setups basically a read directory and a read/write directory that writes to the cache drive for Mover to handle later to move it to the array. 

#Read Directory 
mkdir /mnt/cache/FTP/Read/Movies;mount --bind /mnt/user0/Movies /mnt/cache/FTP/Read/Movies
mkdir /mnt/cache/FTP/Read/TV-Shows;mount --bind /mnt/user0/TV-Shows /mnt/cache/FTP/Read/TV-Shows

#Read Write to cache folder for mover to handle later
mkdir /mnt/cache/FTP/Write/Movies;mount --bind /mnt/user/Movies /mnt/cache/FTP/Read/Movies
mkdir /mnt/cache/FTP/Write/TV-Shows;mount --bind /mnt/user/TV-Shows /mnt/cache/FTP/Read/TV-Shows

Then I would have 

FTPUSER-READ /mnt/cache/FTP/Read/

 

FTPUSER-WRITE /mnt/cache/FTP/Write/

 

Is there any issue with having FTP write to user instead of user0

Share this post


Link to post

I do not see how that is going to work as /mnt/user is a super-set of /mnt/user0 + the cache drive.

Share this post


Link to post
10 minutes ago, itimpi said:

I do not see how that is going to work as /mnt/user is a super-set of /mnt/user0 + the cache drive.

Could be my misunderstanding of how cache functions. I guess If I have cache set to on for a share. If I write anything to /mnt/user/share it will hit the cache drive first? I thought user0 bypassed cache altogether. 

Share this post


Link to post
4 hours ago, halfelite said:

Could be my misunderstanding of how cache functions. I guess If I have cache set to on for a share. If I write anything to /mnt/user/share it will hit the cache drive first? I thought user0 bypassed cache altogether. 

The reason I think it will not work is that anything in ‘user0’ will also exist in ‘user’.   The second part of your suggested commands will therefore fail as by that time the targets will already exist under ‘user’ from when you set them up under ‘user0’.

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now