SlrG

@jeffreywhunter As you don't crash without proftpd anymore it indeed seems to be a problem of proftpd in conjunction with your backup jobs. In my last post I gave you two links that will help you to increase the logging of proftpd. Maybe we will get some new information from that.

@jeffreywhunter I really want to help you solve your problem. But please do as I say and remove my proftpd plugin and see if the server still crashes. Then we will know for sure it is related to proftpd and can start digging in only that direction. Do you use ftp for other purposes as your personal backups? Because if only yourself are accessing your server, you don't need to jail users. In that case you could use unRAIDs internal ftp to keep your backup jobs running while the plugin is gone. Looking at your log and seeing it is hourly, the memory goes down, but in the last log entry before the crash it is back to 6.6 gig again. Maybe if you increase the logging interval to every 5 minutes or minute we will get a better view. Also it could be helpful to increase logging in proftpd as described here and reenable the transfer log as described here.

@jeffreywhunter The last time I tested the apache plugin it didn't start at all. I don't think it could be the cause of your problems, but I would recommend to uninstall it anyway. The plugin rmeaux suggested would probably be my recommendation for the time being. The other apache "plugins" you mentioned are probably dockers and not plugins. All of those I tested could not access the config files and didn't work for the purpose of my plugin. It makes sense from a security viewpoint, so I don't think it has changed.

@rmeaux Cool. I didn't know about this plugin yet. Thank you for posting. If I get the time I'll take a look at how it does things. Maybe I'll be able to get my the proftpd config editor working again that way. It has the benefit of syntax checking the proftpd.conf file and pointing out errors, so I was sad it stopped working.

@jeffreywhunter Normally the plugin disables the internal unRAID webserver automatically. It is reenabled, when the plugin is uninstalled. The reason is, that both use the default port 21 and proftp won't start, if the internal ftp already uses this port. Looking at your logs, the internal ftp seems to be disabled by the plugin. The fix common error plugin says the internal ftp is disabled, but ftp users are defined. Which is normal, as the proftp plugin uses them. So normally you should have no way to disable the internal unRAID webserver yourself, as the link to the settings page gets removed when my plugin is installed? If you still have both icons in your settings, its a bit odd. @rmeaux Your proftpd.conf looks fine and if you have opened the ports on the router everything should work in theory. Did you restart the proftpd server after making the change to the config? Otherwise changes will not get applied. If it still doesn't work, it might be you need to add a MasqueradeAddress, too. Either with your fixed IP or your dyndns name. The plugins config editor needs a running webserver with access rights to the config dir. All the docker ones doesn't have that and the apache plugin webserver stopped running with the latest versions and will get no update. So the functionality to edit the config from the plugins settings page is broken and I have no idea how to make it work again at the moment. It makes sense from a security point of view, however that it has been made harder to access the config files of the server.

@jeffreywhunter In this post the instructions to monitor shfs usage should be pretty clear. You create a file on your flash containing the given line and restart the array. Afterwards it will start logging shfs problems to your syslog. So if you tail that, problems should get visible, if there are any. The process to stop the logging is also described. The syslog from the diagnostics file IMHO doesn't show a reason for your crashes but again, I'm no expert in reading them. The root file system is not running full, thats good. The top info could be a bit suspicious: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 24715 ftpacco+ 20 0 237632 204152 4456 R 81.2 1.3 17:11.28 proftpd 8150 root 20 0 947424 354676 764 S 6.2 2.2 45:31.78 shfs Sadly it doesn't show the exact time it was taken. If it was at the time of the crash, shfs is probably not responsible, as its memory usage is low. My plugin could be responsible as it is the top process at the time the info was saved (probably a file transfer in progress). To be fully sure it is/is not my plugin, I can only recommend again to remove it fully and see if the system still crashes or not. If it not crashes any longer, we could try to increase the logging of the plugin to get more clues what is going wrong. If it still does with the plugin removed, it would be better to post your diagnostics file to the official bug reports forum, so more people and people with more knowledge than me can have a look at your problem. If nothing is visible from the diagnostics file, the recommendation will be to start with a minimal system and expand from that. I know this isn't ideal, as it takes a lot of time while the system is not fully functional, but it is the only way to troubleshoot those problems. If a basic install without plugins, dockers and vms still crashes, it is probably some kind of hardware problem or a bug limetech needs to fix for your system. If that runs fine, you can start reinstalling things one by one. Install one plugin at a time and monitor if the system still crashes. If it is still stable, reinstall the next one. Enable docker. Monitor your system. Install the first docker. Monitor your system. Install the next. Enable VMs. ... That way you will find out, what exactly is responsible for your problems and then can contact the responsible developer to check it out.

@jeffreywhunter The diagnostics file is "old" from 0507 while the FCPsyslog_tail.txt is from 0510 when your system crashed. The newer syslog shows no real problems going on before the crash. At least it seems not to be an classical out of memory problem now, as both values are below 10%. For now, I would reset the values you changed to their original settings. Then I would monitor the shfs usage like described here and a syslog tail like you did before. If this doesn't show anything unusual you will really have to uninstall the proftpd plugin and see if the system still crashes without it.

For FTP to work, you need additional passive ports. In the local network they won't be blocked by your firewall and everything will work. From the WAN some routers allow the automatic opening of passive ports while others don't. So you might need to define a passive port range for proftp to use in the proftpd.conf and define the same range of open ports in your router. Make sure to really change the port to something other as 21, as that will get constantly scanned with malicious intent.

@jeffreywhunter Well do you really need the transfer log? It doesn't seem to be correctly configured on your system anyway, so why not disable it completely by setting it to NONE? That should stop the message from spamming your log. I can't imagine the ProFTP plugin being responsible for your lockups, but to be absolutely sure, you will have to uninstall and run some time without the plugin. If the system still crashes, there is another underlying problem. Looking at your log (but I am no expert in this and might be wrong) there seems to be a memory leak and you are running out of memory and thus the system crashes. Maybe you could follow the tips from here and check your shfs usage and try to change the disk cache ratio?

@quad It seems to be the same problem, you reported one page back. The plugins check for a working internet connection seems to be very flaky on your system. Sadly I still had not the time to look into it. You fixed it by modifying the checks else condition so it always reports yes and then installing the plg manually. This will probably work again.

No problem. Let me know if you find anything.

How do you web? The correct way is: ftp://ip:21 The system should ask for username and password and allow you access to the folder you designated in the description of the user you created. ftp://user@ip:21 is also possible. Then, the system should only ask for your password. Using web based ftp will only allow browsing and downloads AFAIK. So you might want to take a look at dedicated FTP client programs like FileZilla, which will allow for uploads, too.

Yes. The problem is, that proftpd does not support writing logs to world writeable directories (on your flash) any longer. Please see here on how to setup a directory on your cache drive which should work.

@geonerdist Do you have the ftpuser keyword in the comments for the root user, too? Can you try to remove the ftpuser keyword from all your user accounts comments while the ProFTP plugin is uninstalled? Then reboot. Does this solve the problem? If not, can you try to remove your personal, user(s) reboot and recreate it(them) without the ftpuser keyword? If both things do not solve your problem, I can't think of a reason how my plugin could be the cause.

I'm glad you liked it. Thank you. Conclusion from you description above is, you probably need no /mnt/cache/FTP directory. You could just point the home dir of the ftp user directly to /mnt/user/media and use the directory and limit rules to make it read only and prevent access to subdirs you don't want to be read at all. This should "solve" the fix common plugins error. Another way is to set the error to ignored in fix common problems. As for the cache running full, this is probably, because the mover runs on a shedule and not if the disk space gets low. If you have other questions, please let me know.

@soder Okay. Sorry. I was guessing wrong based on the ö in Söder. Lets forget the plugin and binding directories to the ftp side on the cache for a moment. Let's just look on the unRAID side. The question is, what do you expect from your media share. Security or speed. If you create it normally, it will not use the cache drive and reside only on the array. That has the advantage, that every file you copy to your media share will be stored directly on your parity protected array. The drawback is, it is rather slow due to the parity calculations and write operations on the number of drives involved. (e.g 35 MB/sec using normal desktop grade HDDs) If you set the share to use the cache drive any new file will be copied to the cache drive first. This doesn't involve parity calculation and is rather fast. (e.g. 115 MB/sec using desktop grade HDDs or faster using a single SSD or multiple Drive cache pool) The drawback is that the file will not be parity protected until it is moved to the array by the mover (which you should enable to run or new files will never get moved to the array and thus will never be protected). The speeds depend on your drives, network, memory and cpu, so they might be slower or faster. Please don't take them as fixed values. Now we come back to the proftpd plugin. Mounting directories or shares to the FTP directory of the cache doesn't physically double the files. This is just some kind of gateway to make it possible to access the files on both locations. Files copied to /mnt/cache/FTP/media will be saved on the array, if media is a mount of /mnt/user/media and the cache is not enabled for this share. If the cache is enabled for the share the files will be saved on the cache (but in /mnt/cache/media and not in /mnt/cache/FTP/media, though they will be accessible there, too) and later on moved to the array by the mover. I admit its a bit complex. I hope I was able to shed some light on it. edit: The problems with your torrents could be, because you set the destination to be the cache (/mnt/cache/...) but the files are moved to the array by the mover vanishing on the cache. Setting the destination to /mnt/user/... should solve those problems.

@soder Maybe you would like it better to explain in german language? Please send me a pm. What exactly is the problem with torrents? And why do you think you shouldn't bind the media share to the cache?

@soder You can't use it to bind other directories into your ftp root (that should be done using the plugins mount and unmount scripts), but yes, that's the way to protect your folders. If you want to make your FTP dir read only to all users, what you have should be enough. Please also have a look at http://www.proftpd.org/docs/howto/Limit.html.

@ljm42 @geonerdist On my system I can't replicate the problem. Did upgrade do 6.4.1 and set a password for root to enable webgui login security. Logging in and out multiple times does not result in any errors in the log. Maybe the answers to my questions from above give me an idea what is going wrong.

Thank you for the info. I'm not on 4.6.1 yet and have no security on the webgui enabled. So I did not experience this problem yet. I'll upgrade and try to replicate it. What I can say however is, that my ProFTPd plugin creates no user ftpuser, it is only a keyword the user has to enter in the description field in unRAIDs user creation dialog to identify users that should only be able to login by FTP and or should be jailed into the given directory. @geonerdist Do you have an user ftpuser defined as unRAID user or only have users with this keyword in the description field? Does the error go away if you uninstall the ProFTPd plugin?

@bonienl No problem. I'm glad you are willing to investigate. Take all the time you need. If I can help in any way, please let me know.

Me too and I really like the S3Sleep feature. I don't want to give up on it. If somehow possible please make it work again. Also we need a reliable way to detect the spinup status of the disks. Spinning up all disks on wakeup is only a stopgap solution and not working reliably for me. (The disks do spinup, but the state sooner or later gets unsynched and the plugin doesn't know anymore if the disks are running or not. I know S3Sleep is not officially supported, but maybe if @bonienl asked the unRAID devs to track the spinup state in some file, that plugin autors can access and take this info from, the problem could be solved reliably. As he/you are working with them so closely, they might be willing to listen to you. I would even be willing to pitch some money on the solution of this problem. Maybe others would, too?

@satdreamer The problem seems to happen when the internet connection of the server is problematic. I had it never happen on my system, so it is hard to troubleshoot. I'll try to improve the process of checking the connection. But it will take some time. I don't have much freetime at the moment. @soder Please check the first post of this thread. There is a short explanation and a link to the readme, which contains more info on how to setup the users through the unraid user configuration interface. The settings file is on your flash drive and is named proftpd.conf. It is symlinked on boot to /etc/proftpd.conf.

Cool. Just gave it a spin and it works fine, for me too.

@gaby-one The process to enable sftp is explained here.