Jump to content
binhex

[Script] binhex - no_ransom.sh

31 posts in this topic Last Reply

Recommended Posts

Posted (edited)

Hi guys, i got inspired by this post from @BRiT and created a bash script to allow you set media to read only to prevent ransomware attacks and accidental or malicious deletion of files.
 

The script can be executed once to make all existing files read only, or can be run using cron to catch all newly created files as well. The script has an in-built help system with example commands, any questions let me know below.

 

Download by issuing the following command from the unRAID 'Terminal' :-

curl -o '/tmp/no_ransom.sh' -L 'https://raw.githubusercontent.com/binhex/scripts/master/shell/unraid/system/no_ransom/no_ransom.sh' && chmod +x '/tmp/no_ransom.sh'

Then to view the help simply issue:-

/tmp/no_ransom.sh

Disclaimer:- Whilst i have done extensive tests and runs on my own system with no ill effects i do NOT recommend you run this script across all of your media until you are fully satisfied that it is working as intended (try a small test share), i am in no way responsible for any data loss due to the use of this script.

Edited by binhex
  • Like 5
  • Thanks 4

Share this post


Link to post
Posted (edited)

2020-06-29 - New version uploaded, im marking this as v1.0.0, new feature added --include-folders, so now you can specify which folders within a share you want to make read only, also had a bug in --exclude-folders where it was only processing the first folder name in the list, this is now fixed.

Edited by binhex

Share this post


Link to post

Very interesting. It should not be underestimated that not only a server can centrally host files and services, it can also have it's own locked-down policies that you wouldn't necessarily like to set on your desktop, because it might hinder your workflows too much.
 

Talk about truly combining the strengths of both. Add backups to the mix and you're looking at incredible data protection.

 

Now if we could have true bit rot protection in unRAID baked-in, that'd be incredible... :)

Share this post


Link to post
Posted (edited)
On 6/23/2020 at 1:24 PM, binhex said:

The script can be executed once to make all existing files read only, or can be run using cron to catch all newly created files as well. The script has an in-built help system with example commands, any questions let me know below.

Hi @binhex

 

Many thanks. I am also very concerned about ransomware.

 

For now I have my media shares set as read-only in Unraid.

 

Could you please clarify to ignorant people like me how setting all files individually as read-only is different/preferable to what Unraid does ?

 

Many thanks.

 

Best,

OP 

 

EDIT:

 

OK, I realise now that the script does not set files as read-only but sets them "immutable" via chattr :

 

Quote

Immutable:

+i to set
-i to clear

 

A file with the i attribute cannot be modified.

It cannot be deleted or renamed, no link can be created to this file and no data can be written to the file.

When set, prevents, even the superuser, from erasing or changing the contents of the file.

 

So I imagine this is indeed much more difficult for a ransomware to first remove the "i" chattr attribute.

Edited by Opawesome
Did not read carefully the script

Share this post


Link to post
Posted (edited)
1 hour ago, Opawesome said:

Could you please clarify to ignorant people like me how setting all files individually as read-only is different/preferable to what Unraid does ?

there are a 4 advantages that i can think of using my script over setting shares to read only:-
 

1. you can still add new files/folders to the share - the share is not read only and newly created files will NOT be read only, allowing you (or automated tooling such as sonarr/radarr etc) to post process the files, once you are sure its been post processed then re-run the script (or run via scheduled cron job) to lock the new files.

 

2. setting shares to read only does not mean the files are read only, thus an infection could still potentially encrypt files if the infection happened on the unraid host (or docker container) (unlikely but possible).

 

3. folders can still be renamed - the script currently locks files only, so you can still rename folders without issue, which you obviously cannot do when a share is marked read only.

 

4. one last edge case here, there is always the potential for coding issues with any application, a slip up by the developer could potentially cause mass deletion of files on your array, this script would prevent this from happening, a read only share would not (containers do not connect via shares).

1 hour ago, Opawesome said:

So I imagine this is indeed much more difficult for a ransomware to set the remove the "i" chattr attribute.

difficult yes, as the ransomware would need to firstly be running on the unraid host, and secondly be aware of chattr and know how to unlock files, this is unlikely but of course still possible, i personally think this is secure enough for my own peace of mind.

 

Edited by binhex

Share this post


Link to post
32 minutes ago, binhex said:

4. one last edge case here, there is always the potential for coding issues with any application, a slip up by the developer could potentially cause mass deletion of files on your array, this script would prevent this from happening, a read only share would not (containers do not connect via shares).

Wasn't that long ago that a bug in emby was nuking some users media libraries.
I believe the same solution that was an inspiration for your script was used to help protect users media libraries at the time, until the bug was resolved.

Share this post


Link to post
50 minutes ago, binhex said:

4. one last edge case here, there is always the potential for coding issues with any application, a slip up by the developer could potentially cause mass deletion of files on your array, this script would prevent this from happening, a read only share would not (containers do not connect via shares).

 

13 minutes ago, tjb_altf4 said:

Wasn't that long ago that a bug in emby was nuking some users media libraries.

 

This is indeed very relevant.

 

As far as I am concerned, I have a use case where my Plex Docker container has read-only access to my media share because I am afraid what Plex or users using Plex could do to my files. But because I also want Plex to grab subtitles files and and them in the same folder as the corresponding media file (there is a plugin for that), I need to switch the Plex Docker container settings to read-write every time before I update the Plex library, and then switch it back to read-only afterwards. 

 

To ease the process, I have duplicated my Plex Docker container, with both sharing the same appdata and different access permissions to my media folder but still, this is not convenient and when the read-write permissions are on, my media is vulnerable.

 

The chattr +i function just seems a much more suitable solution.

Share this post


Link to post
Posted (edited)
6 minutes ago, Opawesome said:

But because I also want Plex to grab subtitles files and and them in the same folder as the corresponding media file (there is a plugin for that), I need to switch the Plex Docker container settings to read-write every time before I update the Plex library, and then switch it back to read-only afterwards. 

yep, this is where this script would help, folders will not be locked so you are free to add an additonal files that you want, hell if you want to ensure that you can always modify certain file extensions, for example .sub files, then you can exclude this particular extension from being locked via a flag for this script thus allowing you to modify them at any time, whilst still ensuring your media stays safe.

Edited by binhex

Share this post


Link to post
1 hour ago, binhex said:

2. setting shares to read only does not mean the files are read only, thus an infection could still potentially encrypt files if the infection happened on the unraid host (or docker container) (unlikely but possible).

My only comment is, a locally executed infection on the unRAID box that has gained root access, can easily be "intelligent" enough to un-tag a read-only file. In terms of stopping an infection, it's more of a "speed bump" for the future.  For your other points I agree it can have benefits.

I "segregated" my network access to my storage array, to pretty much everything in the primary shares is "read only" (at the share level), and only expose 1 or 2 scratch/work/temp folders on the network as R+W or FULL, and then manually move the data locally onto the array.  Now, as i've said before, I use my array as a WORM device essentially, so that works for me.

 

I hope people don't think that any form of bit-rot functionality baked in to unRAID in the future, is an excuse to become lax on external backups.

 

You know what i'd like to see, and this is a pretty funky idea... the ability to create a party disk of N number of disks that are off-line, without having all the disks spun up at the same time. You'd (ab)use the parity disk pretty heavily on creation, but it'd be a one-time or extremely infrequent event occurring.

Share this post


Link to post
4 hours ago, binhex said:

1. you can still add new files/folders to the share - the share is not read only and newly created files will NOT be read only, allowing you (or automated tooling such as sonarr/radarr etc) to post process the files, once you are sure its been post processed then re-run the script (or run via scheduled cron job) to lock the new files.

I occasionally use Sonarr or Radarr to upgrade existing media. If I ran this for my Media shares, would this prevent those containers from upgrading (deleting + replacing) the files? Any suggestion for how to work around that?

 

Additionally, I have a ton of personal data that I'm still organizing. Would I be able to still move the files around within a share? If not, then I'll probably create an inclusion/exclusion folder for organized files.

 

Lastly, have you tested this with Borg + RClone? Was thinking of getting that setup (using ds-unraid's script) for local and remote backups and wouldn't want this to interfere.

 

Thanks for putting this together. Ransomware attacks have long been a concern of mine and this looks like the perfect tool to put those concerns to rest.

Share this post


Link to post
13 minutes ago, crazyhead247 said:

I occasionally use Sonarr or Radarr to upgrade existing media. If I ran this for my Media shares, would this prevent those containers from upgrading (deleting + replacing) the files? Any suggestion for how to work around that?

unfortunately this would be a limitation of the script, sonarr/radarr etc would not be able to delete existing media, it would be able add new media only.

 

15 minutes ago, crazyhead247 said:

Would I be able to still move the files around within a share?

nope, once locked you would not be able to move media, in that case i would do all your moves/renames etc and then lock it after you are satisfied it wont change.

 

17 minutes ago, crazyhead247 said:

Lastly, have you tested this with Borg + RClone? Was thinking of getting that setup (using ds-unraid's script) for local and remote backups and wouldn't want this to interfere.

i havent, no.

 

17 minutes ago, crazyhead247 said:

Ransomware attacks have long been a concern of mine

you and me both 🙂

  • Like 1
  • Thanks 1

Share this post


Link to post

Hi, nice plugin. It is running in my UNraid box.

 

Regarding the plugin I just havce one question: My server is set to copy new filkes to cache drive and at certain time of the day move it to the the main array. So, before the plugin first execution I just copy a new file to my server and as explained before it was located at my cache drive; then I execute the plugin command. Now I'm trying to guess if Unraid will be able to move this particular file from the cache drive to the main array.

 

Just fro your reference this was the command I executed:

/tmp/no_ransom.sh --lock-files 'yes' --media-shares 'Plex,General Data,system' --include-extensions '*.*' --debug 'yes'

Thanks.

Share this post


Link to post
30 minutes ago, Rommel said:

Now I'm trying to guess if Unraid will be able to move this particular file from the cache drive to the main array.

The line in the script that checks for the disks is:

ls -dv /mnt/disk*

The cache drive is located at /mnt/cache/ so files on it will not be affected. They'll be moved onto the main array with no issue.

  • Like 1
  • Thanks 1

Share this post


Link to post
Posted (edited)

Nice work on the script, great way of protecting files.

 

I was wondering if you can set a share, not a disk, immutable using this script how does that effect the file on the disk level?
For instance if I was to use unBalance to scatter/gather files across disks.

I'm not exactly clear on how Unraid treats disks -> shares. Hardlinks?

 

Would the 'real' file on disk be immutable or just the linked one on the share?

 

(As a side note, I got pretty lucky on timing as I only realised today I had nothing setup for ransomware protection on my server, cheers!)

Edited by Iormangund
P.S.

Share this post


Link to post
1 hour ago, Iormangund said:

I was wondering if you can set a share, not a disk, immutable using this script

no thats not possible, you cannot run chattr across user shares directly, it has to be done on a per disk basis.

 

when you specify the shares to process, the script wil inspect each disk in turn to check for the top level folder that matches the share name, this makes up the view the user sees when they navigate /mnt/user/<share name>.

Share this post


Link to post
1 hour ago, binhex said:

no thats not possible, you cannot run chattr across user shares directly, it has to be done on a per disk basis.

 

when you specify the shares to process, the script wil inspect each disk in turn to check for the top level folder that matches the share name, this makes up the view the user sees when they navigate /mnt/user/<share name>.

Ah ok. Guess I will have to wait to use it properly. In the process of encrypting a 24x8tb disk array that is almost full so everything is being scattered by unbalance all over the place as I empty a disk at a time. Going to need some reorganising when that's all done and then can safely immute my files.

I have a unassigned devices btrfs array mounted at /mnt/disks/btrfs_share, can the script be used on a share outside the array, modified to do so, or would I just be better off learning about chattr and doing it manually?

Share this post


Link to post
10 hours ago, Iormangund said:

Ah ok. Guess I will have to wait to use it properly. In the process of encrypting a 24x8tb disk array that is almost full so everything is being scattered by unbalance all over the place as I empty a disk at a time. Going to need some reorganising when that's all done and then can safely immute my files.

absolutely!, think of running this script as the icing on the cake, the very last thing to do once everything is in its place :-), just as a quick heads up i have NOT tested this script against encrypted files, i would suspect it will run just fine but please be cautious and try it on a small test share for starters before applying it to anything important.

 

10 hours ago, Iormangund said:

I have a unassigned devices btrfs array mounted at /mnt/disks/btrfs_share, can the script be used on a share outside the array, modified to do so, or would I just be better off learning about chattr and doing it manually?

right now it doesnt support unassigned devices, only disks in the array, do you use unassigned devices for long term storage then?, and if so why not copy to the protected array instead where you have fault tolerance?.

Share this post


Link to post
7 hours ago, binhex said:

right now it doesnt support unassigned devices, only disks in the array, do you use unassigned devices for long term storage then?, and if so why not copy to the protected array instead where you have fault tolerance?.

I would think that if you are using UD devices for offsite physical backups, you would want to apply the immutable attribute to keep your backup media extra safe when you are accessing it for recovery purposes.

Share this post


Link to post
12 hours ago, binhex said:

absolutely!, think of running this script as the icing on the cake, the very last thing to do once everything is in its place :-), just as a quick heads up i have NOT tested this script against encrypted files, i would suspect it will run just fine but please be cautious and try it on a small test share for starters before applying it to anything important.

 

right now it doesnt support unassigned devices, only disks in the array, do you use unassigned devices for long term storage then?, and if so why not copy to the protected array instead where you have fault tolerance?.

Thanks, good to know encrypted is untested. Will tread carefully.

It's a btrfs raid 6 array of 4tb disks I use as a scratch disk and steam/gaming library (awesome load times), nothing that isn't easily replaced and I don't waste space backing it up. If it was anything important I sure as hell wouldn't use btrfs 5/6 😆

Was more of a hypothetical really, nothing on there I need to be immutable.

 

5 hours ago, jonathanm said:

I would think that if you are using UD devices for offsite physical backups, you would want to apply the immutable attribute to keep your backup media extra safe when you are accessing it for recovery purposes.

 

Thanks, good idea about setting immutable on external backups, must remember to do that next time I do a cold storage backup.

Share this post


Link to post

So I'm running the script but I get this "No matching media share for disk"

 

Quote

root@Tower:~# /tmp/no_ransom.sh --lock-files 'yes' --media-shares 'TV-Shows' --include-extensions '*.*' --debug 'yes'
[info] Running no_ransom.sh script...
[info] Checking we have all required parameters before running...
[info] Finding share that match 'TV-Shows' on disk '/mnt/disk1'...
[debug] find /mnt/disk1 -maxdepth 1 -type d -name TV-Shows
[debug] No matching media share for disk '/mnt/disk1'
[info] Processing finished for disk '/mnt/disk1'
[info]
[info] Finding share that match 'TV-Shows' on disk '/mnt/disk2'...
[debug] find /mnt/disk2 -maxdepth 1 -type d -name TV-Shows
[debug] No matching media share for disk '/mnt/disk2'
[info] Processing finished for disk '/mnt/disk2'
[info]
[info] Finding share that match 'TV-Shows' on disk '/mnt/disk3'...
[debug] find /mnt/disk3 -maxdepth 1 -type d -name TV-Shows
[debug] No matching media share for disk '/mnt/disk3'
[info] Processing finished for disk '/mnt/disk3'
[info]
[info] Finding share that match 'TV-Shows' on disk '/mnt/disk4'...
[debug] find /mnt/disk4 -maxdepth 1 -type d -name TV-Shows
[debug] No matching media share for disk '/mnt/disk4'
[info] Processing finished for disk '/mnt/disk4'
[info]
[info] Finding share that match 'TV-Shows' on disk '/mnt/disk5'...
[debug] find /mnt/disk5 -maxdepth 1 -type d -name TV-Shows
[debug] No matching media share for disk '/mnt/disk5'
[info] Processing finished for disk '/mnt/disk5'
[info]
[info] Finding share that match 'TV-Shows' on disk '/mnt/disks'...
[debug] find /mnt/disks -maxdepth 1 -type d -name TV-Shows
[debug] No matching media share for disk '/mnt/disks'
[info] Processing finished for disk '/mnt/disks'
[info]
[info] no_ransom.sh script finished

the path to TV shows is: /mnt/user/media what am I doing wrong?

Share this post


Link to post
So I'm running the script but I get this "No matching media share for disk"

 

root@Tower:~# /tmp/no_ransom.sh --lock-files 'yes' --media-shares 'TV-Shows' --include-extensions '*.*' --debug 'yes'

[info] Running no_ransom.sh script...

[info] Checking we have all required parameters before running...

[info] Finding share that match 'TV-Shows' on disk '/mnt/disk1'...

[debug] find /mnt/disk1 -maxdepth 1 -type d -name TV-Shows

[debug] No matching media share for disk '/mnt/disk1'

[info] Processing finished for disk '/mnt/disk1'

[info]

[info] Finding share that match 'TV-Shows' on disk '/mnt/disk2'...

[debug] find /mnt/disk2 -maxdepth 1 -type d -name TV-Shows

[debug] No matching media share for disk '/mnt/disk2'

[info] Processing finished for disk '/mnt/disk2'

[info]

[info] Finding share that match 'TV-Shows' on disk '/mnt/disk3'...

[debug] find /mnt/disk3 -maxdepth 1 -type d -name TV-Shows

[debug] No matching media share for disk '/mnt/disk3'

[info] Processing finished for disk '/mnt/disk3'

[info]

[info] Finding share that match 'TV-Shows' on disk '/mnt/disk4'...

[debug] find /mnt/disk4 -maxdepth 1 -type d -name TV-Shows

[debug] No matching media share for disk '/mnt/disk4'

[info] Processing finished for disk '/mnt/disk4'

[info]

[info] Finding share that match 'TV-Shows' on disk '/mnt/disk5'...

[debug] find /mnt/disk5 -maxdepth 1 -type d -name TV-Shows

[debug] No matching media share for disk '/mnt/disk5'

[info] Processing finished for disk '/mnt/disk5'

[info]

[info] Finding share that match 'TV-Shows' on disk '/mnt/disks'...

[debug] find /mnt/disks -maxdepth 1 -type d -name TV-Shows

[debug] No matching media share for disk '/mnt/disks'

[info] Processing finished for disk '/mnt/disks'

[info]

[info] no_ransom.sh script finished

the path to TV shows is: /mnt/user/media what am I doing wrong?

 

Then the share name is 'media' not 'TV-Shows'

 

Sent from my CLT-L09 using Tapatalk

 

 

 

Share this post


Link to post
2 hours ago, binhex said:

the path to TV shows is: /mnt/user/media what am I doing wrong?

 

Then the share name is 'media' not 'TV-Shows'

 

Sent from my CLT-L09 using Tapatalk

 

 

 

Thanks that worked

Share this post


Link to post
Posted (edited)

Thanks for this - a handy little script.

 

Any reason why the below would not be excluding jpg, opf, db and json files from my Test share? 

 

/mnt/user/appdata/no_ransom/no_ransom.sh --lock-files 'yes' --media-shares 'Test' --include-extensions '*.*' --exclude-extensions '*.jpg,*.opf,*.db,*.json' --debug 'yes'

To explain the use case, I have the calibre library in my media > eBooks share which is a .db file. There are other files within this share that I also do not want to 'lock' so I had hoped to exclude them from the locking. However, when locking with the above, the files with the noted extension are also locked. 

 

The Media>eBooks folder has been copied into my Test share.

Edited by stridemat

Share this post


Link to post
5 minutes ago, stridemat said:

However, when locking with the above, the files with the noted extension are also locked. 

hmm that should work, i will do some further testing, as a possible workaround you could remove the exclude and instead use the include to only lock file types you want to have locked e.g. for ebooks *.epub, *.mobi etc.

Share this post


Link to post
Posted (edited)
24 minutes ago, binhex said:

hmm that should work, i will do some further testing, as a possible workaround you could remove the exclude and instead use the include to only lock file types you want to have locked e.g. for ebooks *.epub, *.mobi etc.

Thanks. I shall do some further investigation as well over the next couple of days. 

Edited by stridemat

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.