ProFTPD Plugin for unRAID v6.8.x


SlrG

Recommended Posts

Latest version of the ProFTPd docker running on 6.3.4.  I'm using FTP for doing backups from my windows 10 stations.  Its worked flawlessly.  In my last backup attempt, I ran into the following errors showing up in my log.

 

Dec 3 12:07:00 HunterNAS emhttp: err: sendFile: sendfile /usr/local/emhttp/update.htm: Broken pipe
Dec 3 12:45:14 HunterNAS emhttp: err: sendFile: sendfile /usr/local/emhttp/webGui/styles/font-awesome.css: Broken pipe
Dec 3 12:45:14 HunterNAS emhttp: err: sendFile: sendfile /usr/local/emhttp/update.htm: Broken pipe
Dec 3 12:45:15 HunterNAS emhttp: err: sendFile: sendfile /usr/local/emhttp/update.htm: Broken pipe
Dec 3 12:45:17 HunterNAS proftpd[14758]: 127.0.0.1 (192.168.29.11[192.168.29.11]) - notice: user ftpaccount: aborting transfer: Data connection closed
Dec 3 13:10:23 HunterNAS proftpd[20140]: 127.0.0.1 (192.168.29.11[192.168.29.11]) - notice: user ftpaccount: aborting transfer: Data connection closed
Dec 3 13:39:01 HunterNAS emhttp: err: sendFile: sendfile /usr/local/emhttp/update.htm: Broken pipe
Dec 3 14:17:14 HunterNAS emhttp: err: sendFile: sendfile /usr/local/emhttp/update.htm: Broken pipe
Dec 3 14:42:43 HunterNAS emhttp: err: sendFile: sendfile /usr/local/emhttp/update.htm: Broken pipe
Dec 3 14:44:57 HunterNAS emhttp: err: sendFile: sendfile /usr/local/emhttp/update.htm: Broken pipe
Dec 3 15:00:01 HunterNAS root: mover started
Dec 3 15:00:01 HunterNAS root: moving "archives" to array
Dec 3 15:00:01 HunterNAS root: .d..t...... ./
Dec 3 15:00:01 HunterNAS root: .d..t...... archives/
Dec 3 15:00:01 HunterNAS root: .d..t...... archives/_gsdata_/
Dec 3 15:00:01 HunterNAS root: .d..t...... archives/
Dec 3 15:00:01 HunterNAS root: mover finished
Dec 3 15:20:04 HunterNAS emhttp: err: sendFile: sendfile /usr/local/emhttp/update.htm: Broken pipe
Dec 3 15:20:04 HunterNAS emhttp: err: sendFile: sendfile /usr/local/emhttp/update.htm: Broken pipe
Dec 3 15:28:10 HunterNAS emhttp: err: sendFile: sendfile /usr/local/emhttp/update.htm: Broken pipe

 

The last entry continues to repeat.  I've canceled the backup job that was running on the PC (using Goodsync via FTP).  But the messages keep repeating sporadically.

 

Thoughts on the cause?

Link to comment

@jeffreywhunter

Hmm... emhttp is the unRAID webui. It should have no connection to the proftpd plugin. Googling the error, I find a post of yourself, where you  solved it by making sure smb3 is enabled on the windows machine. So if you have multiple windows 10 machines, is it enabled on all of them?

 

The proftpd side error is lacking information. I would guess  Dec 3 12:45:17 is the moment you cancelled the backup job? If so, its no real error, but proftpd saying it was stopped by the user. If not you could try to get more information by enabling debug logging as described here.

Link to comment
5 hours ago, SlrG said:

@jeffreywhunter

Hmm... emhttp is the unRAID webui. It should have no connection to the proftpd plugin. Googling the error, I find a post of yourself, where you  solved it by making sure smb3 is enabled on the windows machine. So if you have multiple windows 10 machines, is it enabled on all of them?

 

The proftpd side error is lacking information. I would guess  Dec 3 12:45:17 is the moment you cancelled the backup job? If so, its no real error, but proftpd saying it was stopped by the user. If not you could try to get more information by enabling debug logging as described here.

 

Yep, I have several windows machines, but I'm on the same one that had the SMB issue.  I was never able to get SMB to work consistently with GoodSync.  I ended up moving to Proftpd - which worked well until this error...and the further oddity is that (and don't know why I didn't mention it), I have a dozen backup jobs running in GoodSync.  Except this one.  In fact I'm running a backup now that is working.  I've had jobs abort before, so I was watching the syslog when the above mentioned abort happened.  So it appears to be sporadic.  I'm having other problems with the server where the webgui either slows way down or locks up after a day or two of running.  Webgui goes first (unresponsive), then pieces of the system degrade until the console finally becomes unresponsive.  Then I have to do a hard reset.  I have an AOC-SAS2LP-MV8, which i've replaced with an LSI 9207-8i.  But I still have 8 drives that are on the motherboard (ASUS P8Z77-V LK Pro) which use the marvell controller, so I'm about to move all the drives off the motherboard onto a second LSI 9207-8i.  Will try that once parity completes (its also having problems being VERY slow).

 

If I enable debug logging, will that fill up the flash drive?  Would you recommend I turn it on and leave it on until the next occurrence, or should I wait to see if these other modifications fix the problem?

Link to comment

@jeffreywhunter

I had it happen to me, that if the rootfs runs full I get unexpected results like slowdowns and hangs.

 

Do you have the fix common problems plugin installed? This will do a lot of checks and warn you if there are common problems on your server. It will also check if /var/log or the rootfs in memory are running full and warn you accordingly. If this happens you can then look for the cause and fix it. If you don't have it installed yet, I would recommend giving it a try.

 

Debug logging generates a lot of output, so how big the log will get, depends on how often you have backups or other ftp tasks running and how much data is transferred in the process. So I can't say if it would fill up your flash. It should not be too dangerous to try, however. Keep an eye on it and stop the logging if it gets too big, too fast.

Link to comment
2 hours ago, SlrG said:

@jeffreywhunter

I had it happen to me, that if the rootfs runs full I get unexpected results like slowdowns and hangs.

 

Do you have the fix common problems plugin installed? This will do a lot of checks and warn you if there are common problems on your server. It will also check if /var/log or the rootfs in memory are running full and warn you accordingly. If this happens you can then look for the cause and fix it. If you don't have it installed yet, I would recommend giving it a try.

 

Debug logging generates a lot of output, so how big the log will get, depends on how often you have backups or other ftp tasks running and how much data is transferred in the process. So I can't say if it would fill up your flash. It should not be too dangerous to try, however. Keep an eye on it and stop the logging if it gets too big, too fast.

@SlrGI do use fix common problems, and not seen anything recently.  The system is still running a parity check and found a bunch of errors.  Assuming those are being corrected (I'll run another parity check after this one completes...).  Does this trigger anything for you?

 

Regarding the rootfs getting full.  I've got a 16gb USB stick.  The webgui is now unresponsive (won't load).  But I could SSH, did a df -a and only 7% of the file system is used.

DF -a from SSH.  So unless I don't understand something, appears space not an issue?  Thoughts?

 

Dec 4 08:54:44 HunterNAS root: Restarting HunterNASPlexServer
Dec 4 08:54:44 HunterNAS root: #######################
Dec 4 08:54:44 HunterNAS root: appData Backup complete
Dec 4 08:54:44 HunterNAS root: #######################
Dec 4 08:54:49 HunterNAS sSMTP[27981]: Creating SSL connection to host
Dec 4 08:54:49 HunterNAS sSMTP[27981]: SSL connection using ECDHE-RSA-AES128-GCM-SHA256
Dec 4 08:54:51 HunterNAS sSMTP[27981]: Sent mail for HunterNASAlerts@gmail.com (221 2.0.0 closing connection i63sm2650913itb.35 - gsmtp) uid=0 username=root outbytes=747
Dec 4 08:54:51 HunterNAS root: Deleting /mnt/user/cachebackup/current/2017-11-04@03.00
Dec 4 08:54:51 HunterNAS root: Backup / Restore Completed
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104224
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104232
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104240
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104248
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104256
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104264
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104272
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104280
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104288
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104296
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104304
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104312
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104320
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104328
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104336
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104344
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104352
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104360
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104368
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104376
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104384
Dec 4 09:08:52 HunterNAS kernel: md: recovery thread: P incorrect, sector=3307104392
Dec 4 09:20:03 HunterNAS kernel: md: recovery thread: P incorrect, sector=3408952616

 

Link to comment

@jeffreywhunter

7% for the rootfs probably means there is no problem there. Is there any other size thats critical?

 

16GB flash should give you plenty of room to try proftpds debug logging, maybe it gives us a clue, what is going wrong.

 

But as your parity check has  errors, it IMHO means there is a more serious problem with your server. I would try to run unRAID in safe mode, without plugins, dockers and VMs to see, if it still hangs and crashes. If it still is, you might have faulty hardware. If not, I would reenable the plugins, dockers and VMs one by one to hopefully identify the culprit.

Link to comment

Plugin installation fails. Is something wrong with the repository?

 

plugin: installing: https://raw.githubusercontent.com/SlrG/unRAID/master/Plugins/ProFTPd.plg
plugin: downloading https://raw.githubusercontent.com/SlrG/unRAID/master/Plugins/ProFTPd.plg
plugin: downloading: https://raw.githubusercontent.com/SlrG/unRAID/master/Plugins/ProFTPd.plg ... done
Cleaning up previous install files...
Executing selfremoval commands if present...
...Cleanup complete!
Downloading any missing plugin files...
Latest plugin control files downloading
...Unable to download missing plugin control files. Aborting install
plugin: run failed: /bin/bash retval: 1

Link to comment

@quad

For me it runs fine. I removed the plugin on my system, rebooted and installed it again without errors.

The log shows it downloads the plg fine, which is hosted on the same server (github) as the plugin

control files, that fail. So it must be some hickup in your connection, or on the github servers at the

time you were trying to install. Please give it another try.

Link to comment

It's very bizarre..... I tried to go through the ProFTPd.plg file.

 

After concatenating the wget gz file target url I get the following:

https://github.com/SlrG/unRAID/raw/master/AppSupport/ProFTPd/ProFTPd-SlrG-Control-1.4.tar.gz

Which can be successfully downloaded from my connection! (I just tested it with a browser)

 

The problem stands for net connectivity check! Somehow HTTP 200 OK is not retrieved for all cases.

As seen below after 5 wget retries I got the response.

 

root@Tower:~# wget --no-check-certificate -qST4 --spider "https://github.com/SlrG"
root@Tower:~# wget --no-check-certificate -qST4 --spider "https://github.com/SlrG"

root@Tower:~# wget --no-check-certificate -qST4 --spider "https://github.com/SlrG"
root@Tower:~# wget --no-check-certificate -qST4 --spider "https://github.com/SlrG"
root@Tower:~# wget --no-check-certificate -qST4 --spider "https://github.com/SlrG"
root@Tower:~# wget --no-check-certificate -qST4 --spider "https://github.com/SlrG"
  HTTP/1.1 200 OK
  Server: GitHub.com
  Date: Sun, 10 Dec 2017 22:25:51 GMT
  Content-Type: text/html; charset=utf-8
  Status: 200 OK
  Cache-Control: no-cache
  Vary: X-Requested-With

 

Then I managed to repeat continuously the installation process but again with no luck...

Finally I downloaded the plg file and changed the following line to fool the script. (changed from no)

 

else
    HASNET="yes"
fi

 

Then using the new plg file I repeated the process after uploading it to the appropriate path and started the installation again:

 

plugin: installing: /boot/packages/ProFTPd.plg
Cleaning up previous install files...
Executing selfremoval commands if present...
...Cleanup complete!
Downloading any missing plugin files...
Latest plugin control files downloading
Plugin control files have passed MD5 check
Latest plugin dependency files downloading
Plugin dependency files have passed MD5 check
...Downloading complete!
Running install script for ProFTPd...
Installing plugin control files
Installing plugin dependency files
Detect if MySQL/MariaDB is installed...
Move Config Editor archive to plugin dir...

+==============================================================================
| Installing new package /tmp/ProFTPd-inst/install-B/ProFTPd-x86_64.tgz
+==============================================================================

Verifying package ProFTPd-x86_64.tgz.
Installing package ProFTPd-x86_64.tgz:
PACKAGE DESCRIPTION:
Executing install script for ProFTPd-x86_64.tgz.
Package ProFTPd-x86_64.tgz installed.


Disable builtin FTP-Server...
Restart inetd to apply change...
Starting Internet super-server daemon: /usr/sbin/inetd
Disable icon of builtin FTP-Server...
Remove example config file...
Symlink new config file...
ProFTPd is disabled. Please enable before starting.
Cleaning up temp files/folders
...Install complete!
plugin: installed

 

And finally succeeded....................... 

 

Link to comment
  • 2 weeks later...

Hi Slrg,

 

I'm new to the unraid. I'm trying to push for it in my office and I came across your plugin. I enable it and seems active but I'm not able to use the webserver access. Nor do i see anything the proftpd config windows in the proftpd section. If I change the webserver port to 80, the config section is blank but if i use anything other than 21 i see that the browser is not able to connect.

 

Do you have any idea what I'm doing wrong?

Link to comment

@Vicktork

You don't need the config editor to change your proftpd.conf.  You can always use nano or something and edit it in an unRAID console. :) If you want to enable it anyway, you need to install the Apache Web Server Plugin, too (using a docker based webserver will not work). Make sure to set the same directory and port in both plugins or the plugin won't be able to setup and access the config editor correctly.

Link to comment
  • 2 weeks later...

Hi, I'm trying to enable an sftp port in order to be able to connect to my uraind server from the internet.

I have followed this topic : https://www.digitalocean.com/community/tutorials/how-to-configure-proftpd-to-use-sftp-instead-of-ftp

... but the keys are protected I don't know how, I'm getting this error :

mod_sftp/1.0.0: error reading passphrase for SFTPHostKey '/etc/ssh/ssh_host_rsa_key': Permission denied

What is the best way to allow acess to these keys and keeping my server at least a little bit secure (I can't find enought info on internet about that)? Is there important security breaches I should be aware of while connecting this server to the internet  (for ssh, ftp, sftp, ...) and solutions other than do not use common port numbers ?

Link to comment
  • 2 weeks later...

Hi SIrG-

 

Still love the server...  had a question.  was reading on line about blocking IP addresses from connecting and wasn't sure if this was available in the implementation you have?  Found below on this website:  http://www.proftpd.org/docs/howto/ConnectionACLs.html

 

<Limit LOGIN>
    Order deny,allow

    # -- Add blacklisted addresses here --
    Deny from ...
    Deny from ...
    # Many more Deny from lines

    # Allow everyone not explicitly blacklisted
    AllowAll
  </Limit>

Been trying to test but it doesn't seem to be working.  I may be screwing something up, but was curious if it would even work at all?

 

Let me know when you get a chance!

 

Thanks-

 

Al

 

****** I figured it out.  It was an issue between the keyboard and chair (in other words...me).  Deny statements work just fine!

 

Edited by althoralthor
Figured out the issue
Link to comment

Hi, thanks for making this plug-in available for us.

I am having exactly the same issue as Quad' above, ie. when trying to install the plugin the script aborts with this message:

 

plugin: installing: https://raw.githubusercontent.com/SlrG/unRAID/master/Plugins/ProFTPd.plg
plugin: downloading https://raw.githubusercontent.com/SlrG/unRAID/master/Plugins/ProFTPd.plg
plugin: downloading: https://raw.githubusercontent.com/SlrG/unRAID/master/Plugins/ProFTPd.plg ... done
Cleaning up previous install files...
Executing selfremoval commands if present...
...Cleanup complete!
Downloading any missing plugin files...
Latest plugin control files downloading
...Unable to download missing plugin control files. Aborting install
plugin: run failed: /bin/bash retval: 1
 

Now, I am not as clever as Quad and despite trying to follow his solution I am always getting the same error message.

UnRaid 6.4.0, other plugins install and run fine and plenty of bandwidth is available.

 

I'd appreciate any ideas to make it work.

 

 

Link to comment

I might be stupid or something, but how do I even setup and start the FTP server? =)

 

I'm running unRAID 6.4.0 and got this ProFTPd plugin installed. Do I have to create users in unRAID to make it work? I can't create users just in ProFTPd to be used only for the FTP server?

 

Also, where do I find the settings file for this plugin? When I got dockers installed I kind of know where to find them, but for this plugin I don't.

 

Sorry for noob questions, but would like to get some help with this.

 

Thanks.

 

/Söder

Link to comment
1 hour ago, soder said:

I might be stupid or something, but how do I even setup and start the FTP server? =)

 

I'm running unRAID 6.4.0 and got this ProFTPd plugin installed. Do I have to create users in unRAID to make it work? I can't create users just in ProFTPd to be used only for the FTP server?

 

Also, where do I find the settings file for this plugin? When I got dockers installed I kind of know where to find them, but for this plugin I don't.

 

Sorry for noob questions, but would like to get some help with this.

 

Thanks.

 

/Söder

try CrushFTP...much easier to setup - has UI

  • Like 1
Link to comment

@satdreamer

The problem seems to happen when the internet connection of the server is problematic. I had it never happen on my system, so it is hard to troubleshoot. I'll try to improve the process of checking the connection. But it will take some time. I don't have much freetime at the moment.


@soder

Please check the first post of this thread. There is a short explanation and a link to the readme, which contains more info on how to setup the users through the unraid user configuration interface. The settings file is on your flash drive and is named proftpd.conf. It is symlinked on boot to /etc/proftpd.conf.

 

 

  • Like 1
Link to comment
  • 3 weeks later...

Thanks for your replies. Now when I've used unRAID a little bit more, I tried with proFTPd again.

 

I managed to set a new port, and setup a user, and it works to login.

 

But I still got some questions. First, how do I best set up a path for the FTP structure? Say the FTP shall have access to my /mnt/user/media/videos, but with only read access? Or say I want to use a special structure for the FTP, with some stuff from the path /mnt/user/media/videos, but also some pictures. How do I do that?

 

EDIT: OK, I've read some more, and managed to create a FTP dir in /mnt/cache, and also mount some dirs in to that dir.

But, can I use http://www.proftpd.org/docs/howto/Directory.html instead of binding stuff to that folder? Is that a way to make folders protected?

 

EDIT2: In proftpd.conf I've now added:

 

<Directory /mnt/cache/FTP>
    <Limit WRITE>
      DenyAll
    </Limit>
  </Directory>

 

is that enough?

 

/Söder

Edited by soder
Link to comment
9 hours ago, SlrG said:

@soder

You can't use it to bind other directories into your ftp root (that should be done using the plugins mount and unmount scripts), but yes, that's the way to protect your folders. If you want to make your FTP dir read only to all users, what you have should be enough. Please also have a look at http://www.proftpd.org/docs/howto/Limit.html.

 

 

Thanks.

Another question. I right now got all my media on /mnt/user/media, and I don't use the cache (80GB) for that share. I got problem with downloading torrents then.

But the FTP, I put in /mnt/cache/FTP, and with the bind script I mount my /mnt/user/media/videos to /mnt/cache/FTP. I put on cache on the FTP share, but, maybe I shouldn't?

 

Can someone please help me here?

 

/Söder

Link to comment
1 hour ago, SlrG said:

@soder

Maybe you would like it better to explain in german language? Please send me a pm. What exactly is the problem with torrents? And why do you think you shouldn't bind the media share to the cache?

 

 

Well, I DO understand german, but I think english is better for me (Ich bin schwede). About the cache, I think I thought of it like some kind of temp folder, but I guess that's stupid.

 

But, on the share, what should I pick there in the cache options, when I create it on the cache even by default?

 

/Söder

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.