Squid Posted October 9, 2016 Author Share Posted October 9, 2016 I've upgraded to 2016.10.09. I really like the automatic popup when you're on the RW page The new version is definitely faster! My windows box loses write access in about 1/10 of a second now: 9:13:57.59 - About to delete bait 9:13:57.60 - 0.txt created 9:13:57.60 - 1.txt created 9:13:57.60 - 2.txt created 9:13:57.61 - 3.txt created 9:13:57.61 - 4.txt created 9:13:57.61 - 5.txt created 9:13:57.62 - 6.txt created 9:13:57.62 - 7.txt created 9:13:57.63 - 8.txt created 9:13:57.63 - 9.txt created 9:13:57.64 - 10.txt created 9:13:57.65 - 11.txt created 9:13:57.65 - 12.txt created 9:13:57.66 - 13.txt created 9:13:57.66 - 14.txt created 9:13:57.67 - 15.txt created 9:13:57.68 - 16.txt created 9:13:57.69 - 17.txt created 9:13:57.69 - 18.txt created 9:13:57.70 - 19.txt NOT created Access is denied. One minor thing, it adds some lines to the syslog that don't have timestamps. I'm thinking that might confuse other tools that expect each line to start with a timestamp? Oct 9 15:32:24 TowerVM root: ransomware protection:Starting Background Monitoring Of Bait Files Setting up watches. Watches established. Oct 9 15:43:16 TowerVM emhttp: cmd: /usr/local/emhttp/plugins/dynamix/scripts/tail_log syslog Also... without this plugin, my VM boots in 25 seconds. With the plugin installed, it is almost 3 minutes before there is a login prompt on the console or the gui is available. Diagnostics are attached. It seems to spend a bit of time with vsftpd? Or is it trying to place the bait files before the array is online? Oct 9 15:29:49 TowerVM root: plugin: installing: /boot/config/plugins/ransomware.bait.plg Oct 9 15:29:49 TowerVM root: plugin: skipping: /boot/config/plugins/ransomware.bait/ransomware.bait-2016.10.09-x86_64-1.txz already exists Oct 9 15:29:49 TowerVM root: plugin: running: /boot/config/plugins/ransomware.bait/ransomware.bait-2016.10.09-x86_64-1.txz Oct 9 15:29:49 TowerVM root: Oct 9 15:29:49 TowerVM root: +============================================================================== Oct 9 15:29:49 TowerVM root: | Installing new package /boot/config/plugins/ransomware.bait/ransomware.bait-2016.10.09-x86_64-1.txz Oct 9 15:29:49 TowerVM root: +============================================================================== Oct 9 15:29:49 TowerVM root: Oct 9 15:29:49 TowerVM root: Verifying package ransomware.bait-2016.10.09-x86_64-1.txz. Oct 9 15:29:49 TowerVM root: Installing package ransomware.bait-2016.10.09-x86_64-1.txz: Oct 9 15:29:49 TowerVM root: PACKAGE DESCRIPTION: Oct 9 15:29:49 TowerVM root: Package ransomware.bait-2016.10.09-x86_64-1.txz installed. Oct 9 15:29:49 TowerVM root: Oct 9 15:29:49 TowerVM root: Oct 9 15:29:49 TowerVM root: plugin: running: anonymous Oct 9 15:29:49 TowerVM root: Stopping the service and deleting pre-existing bait files. This may take a bit Oct 9 15:29:49 TowerVM vsftpd[2814]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:29:50 TowerVM vsftpd[2820]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:29:52 TowerVM vsftpd[2830]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:29:55 TowerVM vsftpd[2844]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:29:59 TowerVM vsftpd[2862]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:30:04 TowerVM vsftpd[2884]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:30:10 TowerVM vsftpd[2910]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:30:17 TowerVM vsftpd[2940]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:30:25 TowerVM vsftpd[2974]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:30:34 TowerVM vsftpd[3012]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:30:44 TowerVM vsftpd[3054]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:30:54 TowerVM vsftpd[3096]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:31:04 TowerVM vsftpd[3138]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:31:14 TowerVM vsftpd[3180]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:31:24 TowerVM vsftpd[3222]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:31:34 TowerVM vsftpd[3264]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:31:44 TowerVM vsftpd[3306]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:31:54 TowerVM vsftpd[3348]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:32:04 TowerVM vsftpd[3390]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:32:14 TowerVM vsftpd[3432]: connect from 127.0.0.1 (127.0.0.1) Oct 9 15:32:14 TowerVM root: ransomware protection:Ransomware protection service not running Oct 9 15:32:14 TowerVM root: Restarting the background service Oct 9 15:32:14 TowerVM root: -------------------------------- Oct 9 15:32:14 TowerVM root: Ransomware Protection Installed Oct 9 15:32:14 TowerVM root: This plugin requires inotify-tools (available within the NerdPack plugin) to operate Oct 9 15:32:14 TowerVM root: Copyright 2016, Andrew Zawadzki Oct 9 15:32:14 TowerVM root: Version: 2016.10.09 Oct 9 15:32:14 TowerVM root: -------------------------------- Oct 9 15:32:14 TowerVM root: plugin: installed Oct 9 15:32:14 TowerVM root: Starting go script Those 2 non-time stamped lines are a direct dump from inotifywait to the syslog. I *could* linux pipe file and then have another process monitor the pipe for changes and then log those lines. However, based upon my experience with the old Checksum plugin, I elected to not go that route as it was a major PITA to get and keep everything working properly. The vsftpd, I have seen that stuff in my syslog also, but the plugin doesn't touch anything at all regarding it (and at the point the plugin installs, it attempts to start its background service, sees that the array isn't started, so promptly aborts). EDIT: Unless its something in the dockerMan dynamix library that's included by the stop routine messing that up. Technically, I'm including my "helper" file (which has all of the various subroutines used by every part of this plugin. The "helper" file also includes a dynamix library because some of the functions require access to docker. I'll try not including the helper file on the stop routine, simply because all I need it for is the logging function and see what happens Response time: Thanks Don't think I can do much better than that (or if I can we're talking miliseconds which won't make much difference at all) VM Startup time. This is because ultimately, the only way to get the plugin into a known state prior to monitoring the files is to delete any old ones that may still be on the array, and then recreate them again. (which can take a number of minutes depending upon how your useCache settings are on the shares, etc) Trouble is that if I start monitoring a file that doesn't exist already, an alert is immediately sent out, and if somehow a file got changed inbetween starts of the plugin, then its already at the wrong md5 and will no longer serve its purpose. When I do the specialized Bait shares next, I was planning on not doing this at all (simply because we're talking about possibly putting 100,000 bait files onto the array which will definitely take a while to do) Quote Link to comment
ljm42 Posted October 10, 2016 Share Posted October 10, 2016 Those 2 non-time stamped lines are a direct dump from inotifywait to the syslog This page shows how to pipe stderror somewhere: https://stackoverflow.com/questions/2342826/how-to-pipe-stderr-and-not-stdout so something like this might let you use logger rather than directly appending to the syslog: inotifywait 2>&1 > /dev/null | logger (my assumption here is that "logger" will automatically strip the line feeds. If not you might have to pipe to "sed 's/\\n/ /'g" first ) Then again, this might not even be a problem Response time. Agreed, this is awesome! VM Startup time. I have the plugin configured to add files to the "Root ony of shares", so it doesn't seem like it should add 2 minutes to the boot time. This feels like a timeout of some kind because the array hasn't started yet, or maybe something to do with vsftp. I'm leaning toward vsftp, since those extra lines appear in the syslog. The specialized bait shares sound like a great idea! Quote Link to comment
Squid Posted October 10, 2016 Author Share Posted October 10, 2016 Those 2 non-time stamped lines are a direct dump from inotifywait to the syslog This page shows how to pipe stderror somewhere: https://stackoverflow.com/questions/2342826/how-to-pipe-stderr-and-not-stdout so something like this might let you use logger rather than directly appending to the syslog: inotifywait 2>&1 > /dev/null | logger (my assumption here is that "logger" will automatically strip the line feeds. If not you might have to pipe to "sed 's/\\n/ /'g" first ) Then again, this might not even be a problem Response time. Agreed, this is awesome! VM Startup time. I have the plugin configured to add files to the "Root ony of shares", so it doesn't seem like it should add 2 minutes to the boot time. This feels like a timeout of some kind because the array hasn't started yet, or maybe something to do with vsftp. I'm leaning toward vsftp, since those extra lines appear in the syslog. The specialized bait shares sound like a great idea! Sed is a swear word in my house but thanks for the tip. Bash is a necessary evil for me Like I said I'm going to try removing the dockerMan reference and see if it solves the other issue Sent from my LG-D852 using Tapatalk Quote Link to comment
Squid Posted October 10, 2016 Author Share Posted October 10, 2016 VM Startup time. I have the plugin configured to add files to the "Root ony of shares", so it doesn't seem like it should add 2 minutes to the boot time. This feels like a timeout of some kind because the array hasn't started yet, or maybe something to do with vsftp. I'm leaning toward vsftp, since those extra lines appear in the syslog. Try the update. Absolutely no clue why dockerMan does something with vsftpd, but all references to dockerMan are now removed. (The impact however is that if your appdata is stored outside of the default appdata share, then it will not automatically get excluded) Quote Link to comment
ljm42 Posted October 10, 2016 Share Posted October 10, 2016 Yep, that took care of it. No more vsftpd references in the log and no more long delays at boot. Thanks! And no more orphan lines in the syslog! Sorry for using the "s" word Quote Link to comment
Squid Posted October 11, 2016 Author Share Posted October 11, 2016 The specialized bait shares sound like a great idea! Since you're actively using this, this is what's progressing: (And its RobJ's idea) - User selectable bait shares (multiple -> all starting with a user Selectable Prefix) + a specialized bait share for the plugin's own puposes. - Selectable "width" and "depth" of share folders - Selectable # of bait files per folder - Random selection of file / folder naming (dictionary based - using actual words, random separators between words, randomly also tosses a random date on the file name) - After directory structure created for all the shares (< 1 minute), monitoring will begin automatically while the files are placed (~ 5 minutes per 20,000 files ) - Each share regardless of number of bait files uses < 10 Meg actual disk space. - Because the impact on the file system is minimal, will probably have it regenerate every boot, as its a far, far simpler setup to handle. Spent the yesterday working out placing the hardlinks and how inotify responds to changes on them (and also how Windows through SMB affects them through various different programs (side note: MS Office destroys links, but just about everything else keeps them ), so GUI is the next step... Oh yeah, also dropped the size of the PDF from ~200K down to 9K Quote Link to comment
ljm42 Posted October 11, 2016 Share Posted October 11, 2016 Thanks for building this! I really like the direction you are taking it. The only part that concerns me a little is that the files are recreated on boot. If that will block the array from starting, can you drop a "starting ransomware protection" note on the console? unRAID does not give good feedback on the plugin portion of the boot process. Quote Link to comment
Guest Posted October 12, 2016 Share Posted October 12, 2016 First off, thanks for the awesome plugin! Now I can rest peacefully knowing that some dumbass code won't ruin my hoard of years of media collection. I deleted one of the files for a test, and it immediately detected it and shut down all the services and stopped the array. +1. Then I restarted the array and reprompted the protection. (BTW, it's a good idea to make the plugin start whenever the array launches - or provide the user an option they can set explicitly in the plugin settings if the former method is too intrusive) The SMB shares popped up a minute or two later. Problem 1: Whenever I copy over the files, the warning dialog of "Some attributes cannot be copied over" pops up. I've never had this prior to doing the fake 'ransomware' test, so it must be something that the plugin touched. Maybe Windows 10 can't copy over some attributes because this plugin modifies the SMB settings? Problem 2: If a ransomware is detected, the user is sent an email. The problem is that if the user is away from a computer and is watching a movie, they won't know about the detection and swear at the slow networking speeds while the ransomware screws over the infected device. I nominate some kind of warning buzzer through the system speaker header so the user knows about it quicker. They can watch the demise of their rapidly encrypting device while the server goes topless. Again, nice plugin. Thanks Squid! Quote Link to comment
Squid Posted October 12, 2016 Author Share Posted October 12, 2016 Problem 1: Whenever I copy over the files, the warning dialog of "Some attributes cannot be copied over" pops up. I've never had this prior to doing the fake 'ransomware' test, so it must be something that the plugin touched. Maybe Windows 10 can't copy over some attributes because this plugin modifies the SMB settings? What do you mean by copy over the files? Copying the bait files from the server to your Windows box? Beyond that, there is no SMB setting that is touched prior to it tripping. At time of trip, SMB is stopped, and the Share Configs are modified (no different than doing it yourself via the Shares Tab), and then unRaid automatically restarts the SMB service. Quote Link to comment
Squid Posted October 12, 2016 Author Share Posted October 12, 2016 The only part that concerns me a little is that the files are recreated on boot. It'll never block it from starting (nor from shutting down for that matter), but I started thinking the same thing yesterday. Revamp the starting service script to check if the files already exist and skip creation if they do - especially since with the bait shares as you can easily have 500,000+ files taking up 0 space - if an attack happens to delete one of them, not much difference if there's only 499,999 remaining, and the time savings is huge. Just delays the next rev a bit... Quote Link to comment
Squid Posted October 12, 2016 Author Share Posted October 12, 2016 Problem 2: If a ransomware is detected, the user is sent an email. The problem is that if the user is away from a computer and is watching a movie, they won't know about the detection and swear at the slow networking speeds while the ransomware screws over the infected device. I nominate some kind of warning buzzer through the system speaker header so the user knows about it quicker. They can watch the demise of their rapidly encrypting device while the server goes topless. A beep is no big deal to do... But, in case of an attack its not going to be a slowdown in networking speeds to an open stream. It'll drop the stream immediately (minus whatever the client has buffered). The network will be unavailable for up to a minute... Of course, most people will just restart playing over again, but not much I can do about that... Quote Link to comment
ljm42 Posted October 12, 2016 Share Posted October 12, 2016 Problem 2: If a ransomware is detected, the user is sent an email. The problem is that if the user is away from a computer and is watching a movie, they won't know about the detection and swear at the slow networking speeds while the ransomware screws over the infected device. I nominate some kind of warning buzzer through the system speaker header so the user knows about it quicker. They can watch the demise of their rapidly encrypting device while the server goes topless. A beep is no big deal to do... But, in case of an attack its not going to be a slowdown in networking speeds to an open stream. It'll drop the stream immediately (minus whatever the client has buffered). The network will be unavailable for up to a minute... Of course, most people will just restart playing over again, but not much I can do about that... This is a good idea, an option to beep out something ominous (taps?) so that the user knows there is an issue and can potentially find the infected pc faster. Although if your phone is nearby, the built-in pushbullet notifications should do the trick too. Quote Link to comment
Squid Posted October 12, 2016 Author Share Posted October 12, 2016 Problem 2: If a ransomware is detected, the user is sent an email. The problem is that if the user is away from a computer and is watching a movie, they won't know about the detection and swear at the slow networking speeds while the ransomware screws over the infected device. I nominate some kind of warning buzzer through the system speaker header so the user knows about it quicker. They can watch the demise of their rapidly encrypting device while the server goes topless. A beep is no big deal to do... But, in case of an attack its not going to be a slowdown in networking speeds to an open stream. It'll drop the stream immediately (minus whatever the client has buffered). The network will be unavailable for up to a minute... Of course, most people will just restart playing over again, but not much I can do about that... This is a good idea, an option to beep out something ominous (taps?) Was actually thinking of the imperial march Sent from my LG-D852 using Tapatalk Quote Link to comment
ljm42 Posted October 12, 2016 Share Posted October 12, 2016 This is a good idea, an option to beep out something ominous (taps?) Was actually thinking of the imperial march LOL I was subtly trying to steer you away from that since I already use it to signal when UD has finished copying files off a camera card But I can find something else if you decide to use it. It would be pretty appropriate here. Quote Link to comment
Squid Posted October 12, 2016 Author Share Posted October 12, 2016 This is a good idea, an option to beep out something ominous (taps?) Was actually thinking of the imperial march LOL I was subtly trying to steer you away from that since I already use it to signal when UD has finished copying files off a camera card But I can find something else if you decide to use it. It would be pretty appropriate here. Any other than just stick beep beep beep would be selectable Sent from my LG-D852 using Tapatalk Quote Link to comment
wgstarks Posted October 12, 2016 Share Posted October 12, 2016 SOS would be very simple in Beep. Quote Link to comment
FreeMan Posted October 12, 2016 Share Posted October 12, 2016 Some sort of disk space checking to ensure you're not filling a disk or share would be fantastic! I've got one share dedicated to one disk (2TB drive) that was down to 270Kb of free space. Don't think it was the fault of the plugin, but my syslog was flooded with "out of disk space" errors trying to move stuff there. Quote Link to comment
CHBMB Posted October 12, 2016 Share Posted October 12, 2016 Some sort of disk space checking to ensure you're not filling a disk or share would be fantastic! I've got one share dedicated to one disk (2TB drive) that was down to 270Kb of free space. Don't think it was the fault of the plugin, but my syslog was flooded with "out of disk space" errors trying to move stuff there. Unraid already has a minimum free space setting natively in the webui, so can't quite work out how you managed to get it down to 270Kb, although, gotta say I'm impressed. Quote Link to comment
Spritzup Posted October 13, 2016 Share Posted October 13, 2016 Since you changed it to set SMB to Read Only upon detection, is their still value in stopping the array? ~Spritz Quote Link to comment
FreeMan Posted October 13, 2016 Share Posted October 13, 2016 Unraid already has a minimum free space setting natively in the webui, so can't quite work out how you managed to get it down to 270Kb, although, gotta say I'm impressed. Thank you, thank you very much! I've hit 0 bytes when running 5.x, so I may have something not set correctly. Quote Link to comment
Squid Posted October 13, 2016 Author Share Posted October 13, 2016 Some sort of disk space checking to ensure you're not filling a disk or share would be fantastic! I've got one share dedicated to one disk (2TB drive) that was down to 270Kb of free space. Don't think it was the fault of the plugin, but my syslog was flooded with "out of disk space" errors trying to move stuff there. Unraid already has a minimum free space setting natively in the webui, so can't quite work out how you managed to get it down to 270Kb, although, gotta say I'm impressed. The plugin will respect minimum levels. If a file is not able to be created then it will not get monitored. Sent from my SM-T560NU using Tapatalk Quote Link to comment
Squid Posted October 13, 2016 Author Share Posted October 13, 2016 Since you changed it to set SMB to Read Only upon detection, is their still value in stopping the array? ~Spritz Depends upon your level of paranoia Sent from my SM-T560NU using Tapatalk Quote Link to comment
S80_UK Posted October 13, 2016 Share Posted October 13, 2016 Since you changed it to set SMB to Read Only upon detection, is their still value in stopping the array? ~Spritz Depends upon your level of paranoia Sent from my SM-T560NU using Tapatalk One possibility is that if you have some rogue software somewhere on your network, not only might it modify / encrypt your files, but it could also be sending information "back home". My view would be that I would want the server to have share access blocked until I had a chance to get control of things. Quote Link to comment
Msan Posted October 13, 2016 Share Posted October 13, 2016 How about having the ability to have a custom script run when it's triggered? This way folks could have extra/special things happen without having to have it hardcoded into the plugin.. Quote Link to comment
kizer Posted October 13, 2016 Share Posted October 13, 2016 I haven't had a chance to play with this as of yet, but I'm really intrigued now. I know on trigger it sets things to ReadOnly. Is there a way to "Return to Previous" State. Some of my Drives I have different settings depending on users, Disk Shares, User Shares Would be nice to simply return to whatever settings I had on that particular drive/Share so I don't have to figure out what was changed and how it was before. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.