ProFTPD Plugin for unRAID v6.8.x


SlrG

Recommended Posts

@rmeaux

Cool. I didn't know about this plugin yet. Thank you for posting. If I get the time I'll take a look at how it does things. Maybe I'll be able to get my the proftpd config editor working again that way. It has the benefit of syntax checking the proftpd.conf file and pointing out errors, so I was sad it stopped working.

Link to comment
4 hours ago, SlrG said:

If you still have both icons in your settings, its a bit odd.

 

the apache plugin webserver stopped running with the latest versions and will get no update. 

 

 

@SlrG

I only have the Proftpd icon in settings.  But in your comment to Meaux you said the Apache plugin wasn't working and you can't edit the file.  I edited my config just to test and it looked liked it edited, but then got an error saving.  Is that the current behavior?  If the Apache Plugin is not working properly, could it be the cause of the problems I'm having?

 

I've deleted the plugin from my system and disabled the reboot cron.  I'll let you know if that solves my problem.

 

I see there are a couple other Apache plugins out there.  Do you recommend using them?  Or is the editor Meaux is using be your new recommendation?

Edited by jeffreywhunter
Link to comment

@jeffreywhunter

The last time I tested the apache plugin it didn't start at all. I don't think it could be the cause of your problems, but I would recommend to uninstall it anyway. The plugin rmeaux suggested would probably be my recommendation for the time being. The other apache "plugins" you mentioned are probably dockers and not plugins. All of those I tested could not access the config files and didn't work for the purpose of my plugin. It makes sense from a security viewpoint, so I don't think it has changed.

Link to comment

@SlrG - So I've built a little log using User Scripts.  Works really well.  In trying to uncover what's causing my server to crash and trying to eliminate ProFTPd as related to the source.  What I'm finding (log attached) is that the problem appears at night, while backups are running.  I have about 10 jobs that backup via ProFTPd.  They start around midnight and end around 6am.  If you look at my log, you see that during the day, memory usage is pretty stable.  But at night, when the backup jobs run, memory usage expands until (and this is frustrating, SOMETIMES) it crashes.  Perhaps it has to do with how much data movement?  No idea how to diagnose that. 

 

One other important note.  It appears it often crashes on the job that backups my workstation image.  It can be anything from a small file (48mb) to a large file (10gb) depending on changes, etc.  I have a 512gb cache drive, so that shouldn't be a problem.

 

Perhaps this log will shed some light.  Perhaps there are some additional commands I can add to my log to help diagnose the cause?  Not saying its ProFTPd, but maybe something related to backups?

 

My log script is pretty simple:

 

#!/bin/bash
echo '======Start=======' >> /boot/log_free_memory.log
date >> /boot/log_free_memory.log
free -h >> /boot/log_free_memory.log
echo '======End=======' >> /boot/log_free_memory.log

And I run it hourly.  In the attached log, the server crashed after the 4:47AM log dump.  I was out all day and didn't get the server restarted until 4PM.

 

Thanks in advance for taking a look!  log_free_memory.log

Edited by jeffreywhunter
Link to comment

@jeffreywhunter

I really want to help you solve your problem. But please do as I say and remove my proftpd plugin and see if the server still crashes. Then we will know for sure it is related to proftpd and can start digging in only that direction. Do you use ftp for other purposes as your personal backups? Because if only yourself are accessing your server, you don't need to jail users. In that case you could use unRAIDs internal ftp to keep your backup jobs running while the plugin is gone.

 

Looking at your log and seeing it is hourly, the memory goes down, but in the last log entry before the crash it is back to 6.6 gig again. Maybe if you increase the logging interval to every 5 minutes or minute we will get a better view. Also it could be helpful to increase logging in proftpd as described here and reenable the transfer log as described here.

Link to comment
On 5/19/2018 at 1:44 AM, SlrG said:

@jeffreywhunter

I really want to help you solve your problem. But please do as I say and remove my proftpd plugin and see if the server still crashes. Then we will know for sure it is related to proftpd and can start digging in only that direction. Do you use ftp for other purposes as your personal backups? Because if only yourself are accessing your server, you don't need to jail users. In that case you could use unRAIDs internal ftp to keep your backup jobs running while the plugin is gone.

 

Looking at your log and seeing it is hourly, the memory goes down, but in the last log entry before the crash it is back to 6.6 gig again. Maybe if you increase the logging interval to every 5 minutes or minute we will get a better view. Also it could be helpful to increase logging in proftpd as described here and reenable the transfer log as described here.

 

Ok, I've followed your directions.  I've been up for more than 4 days.  Far longer than I've ever gone (usually crashes within 24 hours).  I think it has to do with FTP file transfers.  As I've mentioned I have about 12 goodsync FTP jobs that run nightly to backup various directories on my PC and such.  So, evidently, on my system, something about ProFTPd causes my system to crash.  What diagnostics can I conduct to isolate the problem?

 

FYI, I've included my latest diagnostics.

 

Thanks in advance!

hunternas-diagnostics-20180524-1644.zip

Edited by jeffreywhunter
Link to comment

Ok, I'll reinstall and try that.  Will keep you posted.  In thinking about how a backup job might work.  Is there some aspect of backing up via FTP that could cause ProFTPd to have a problem/memory leak over time?  On a nightly basis I might backup 1000 files.  Mostly documents, sometimes music/video files.  If you had an idea, I could build a test case that would push the boundaries...

Link to comment

Well I'm not the developer of ProFTPd, I merely made it available on the unRAID platform. So I really can't answer you question. For me it is running fine, but I use it very seldom and for few and small files only. Maybe we can see something from the new debug logs. If not, I fear I can't help and you'll have to ask the ProFTPd developers themselves. :(

 

Link to comment
  • 1 month later...
  • 5 weeks later...
  • 1 month later...

AFAIK there is no client plugin. Most clients have a graphical interface and thus need a lot of additional packages, which would greatly increase the possibility of breaking unRAID base functionality and making the system unstable. So the only way is to use a docker one or run the client of your choice in a VM.

Link to comment
12 hours ago, SlrG said:

AFAIK there is no client plugin. Most clients have a graphical interface and thus need a lot of additional packages, which would greatly increase the possibility of breaking unRAID base functionality and making the system unstable. So the only way is to use a docker one or run the client of your choice in a VM.

thank you

 

makes sense I understand

Link to comment

By highlighting the text, of course, I was able to read it all and duplicate the settings from my primary server to my new backup server.

 

In trying to connect to the FTP server on my backup server from the Duplicati docker running on my primary server, Duplicati is asking me to confirm the SSH-RSA2048 key. Where do I go to find the key to confirm it's the right one?

 

I can't imagine it's not, but it's always good to be 100% certain, plus it'll be handy for future reference.

Link to comment

@FreeMan

I'm still on 6.5 on my server, so I did not notice the problems until now. Thank you for reporting. I'll have a look and try to fix it.

 

The SSH key should be located in /etc/ssh, if the unRAID ssh key is used it is probably the file ssh_host_key.pub. If not, its probably one of the others located there.

Link to comment
15 hours ago, SlrG said:

The SSH key should be located in /etc/ssh, if the unRAID ssh key is used it is probably the file ssh_host_key.pub. If not, its probably one of the others located there.

I don't see anything in any of those files that looks remotely like the key I'm being presented with. That key is a bunch of hex pairs like this. Any additional thoughts?

 

2018-09-25 18_01_16-91d370e4e434 - Duplicati.png

Link to comment

That is an fingerprint md5 hash of a rsa key. You have to open the unRAID shell and issue the following command:

ssh-keygen -E md5 -lf /etc/ssh/ssh_host_rsa_key

It will show you the fingerprint md5 hash of the key stored in the file ssh_host_rsa_key. Change it if necessary to get the info it for the other keys.

Link to comment
  • 2 months later...

Just wanting to make sure I am setting this correctly. In the mount script

 

I would like to have two setups basically a read directory and a read/write directory that writes to the cache drive for Mover to handle later to move it to the array. 

#Read Directory 
mkdir /mnt/cache/FTP/Read/Movies;mount --bind /mnt/user0/Movies /mnt/cache/FTP/Read/Movies
mkdir /mnt/cache/FTP/Read/TV-Shows;mount --bind /mnt/user0/TV-Shows /mnt/cache/FTP/Read/TV-Shows

#Read Write to cache folder for mover to handle later
mkdir /mnt/cache/FTP/Write/Movies;mount --bind /mnt/user/Movies /mnt/cache/FTP/Read/Movies
mkdir /mnt/cache/FTP/Write/TV-Shows;mount --bind /mnt/user/TV-Shows /mnt/cache/FTP/Read/TV-Shows

Then I would have 

FTPUSER-READ /mnt/cache/FTP/Read/

 

FTPUSER-WRITE /mnt/cache/FTP/Write/

 

Is there any issue with having FTP write to user instead of user0

Link to comment
10 minutes ago, itimpi said:

I do not see how that is going to work as /mnt/user is a super-set of /mnt/user0 + the cache drive.

Could be my misunderstanding of how cache functions. I guess If I have cache set to on for a share. If I write anything to /mnt/user/share it will hit the cache drive first? I thought user0 bypassed cache altogether. 

Link to comment
4 hours ago, halfelite said:

Could be my misunderstanding of how cache functions. I guess If I have cache set to on for a share. If I write anything to /mnt/user/share it will hit the cache drive first? I thought user0 bypassed cache altogether. 

The reason I think it will not work is that anything in ‘user0’ will also exist in ‘user’.   The second part of your suggested commands will therefore fail as by that time the targets will already exist under ‘user’ from when you set them up under ‘user0’.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.