*** [GUIDE] *** Setup Crowdsec with SWAG


Mik3

Recommended Posts

Dear community,

 

I have been looking for a quick reference guide on how to setup crowdsec with swag but I wasn't able to find one, hence this post. Hopefully this will be useful to someone else. Please let me know in the comment if there is a mistake, if you would add something else to make it easier to follow or if the setup can be improved in anyway. Comments, suggestion are always welcome and a way for me to learn more.

 

Let's begin...

 

PREREQUISITES:

--------------------------

  • Unraid 6.11.5 or greater (might work on previous version but I didn't test)
  • Unraid community app plugin
  • Swag (linuxserver.io docker) already set up

 

SETUP:

------------

 

STEP 1 - Installing CrowdSec

---------------------------------------------

Go into the community app and search for crowdsec (IBRACORP docker maintainer), then hit install.

Before applying you need to set the following variables:

 

Network type                    => Custom: <swag>    Replace with the custom docker network type you created when you first created swag

Auth logs to analyze         =>  /mnt/user/appdata/swag/log/nginx    Replace the path with the swag log folder (the one listed is the default)

Crowdsec logs to analyze => /mnt/user/appdata/swag/log/nginx     

Collections                       => crowdsecurity/nginx crowdsecurity/http-cve  If you need more check https://hub.crowdsec.net/browse/

 

Then check that the ports 8080 and 6060 are NOT in use by another container and click apply otherwise you need to change those ports variable as well.

 

STEP 2 - Crowdsec Configuration

----------------------------------------------------

Let the docker pull finish then open the docker logs and wait for it to finishing the initial setup, then look for the following message:

 

msg="Starting processing data"

 

Once you see the message shut down the container.

Go to the crowdsec appdata folder (default is /mnt/user/appdata/crowdsec) and edit the acquis.yaml file.

Under the first "filenames:" add the following line: /var/log/crowdsec/*.log

Your file should look similar to this:

filenames:
   - /var/log/nginx/*.log
   - ./tests/nginx/nginx.log
   - /var/log/crowdsec/*.log    <------------------------------------ this is the line we added
#this is not a syslog log indicate which kind of log it is
labels:
   type: nginx  
---
[...]

 

By default crowdsec bans for 4h to modify that value we need the profiles.yaml file in the crowdsec appdata folder.

Edit profiles.yaml search for the line decisions and replace 4h with whatever vaule you want/need.

An example below:

[...]

decisions:
 - type: ban
   duration: 20h   <--------Default is 4h, replace this vaule according to your needs.

[...]

 

Now we can start the crowdsec docker, again check docker logs to make sure no errors are show and search for the line

msg="Starting processing data"
time="01-02-2023 00:03:00" level=info msg="Adding file /var/log/swag/access.log to datasources" type=file
time="01-02-2023 00:03:00" level=info msg="Adding file /var/log/swag/error.log to datasources" type=file
time="01-02-2023 00:03:00" level=info msg="Adding file /var/log/swag/unauthorized.log to datasources" type=file

If you see this, it means that everything is OK and crowdsec is now parsing the logs.

 

 

STEP 3 - SWAG <-> Crowdsec configuration

--------------------------------------------------------------------

Crowdsec is parsing the logs and making decisions but no further action is taken and the connection is not effectively blocked as swag doesn't know anything about crowdsec decisions. To make crowdsec communicate with swag we need to configure the swag container and the crowdsec bouncer.

First of all we need to install the swag bouncer inside crowdsec, to do so run in the shell the following command:

docker exec -t crowdsec cscli bouncers add swag

The command will return an API key. Copy that api key as we need it for the next step

 

Now we need to edit the swag docker container and add the following variables:

 

DOCKER_MODS: linuxserver/mods:swag-crowdsec

CROWDSEC_API_KEY: ${API}        <-------------Replace API with the API key you obtained from the cscli command in crowdsec

CROWDSEC_LAPI_URL: http://[IP]:8080   <-----Replace IP with the Ip address of the crowdsec docker. 8080 is the default                                                 port, if you changed that you need to change it here as well.

 

NOTE: Replace "${API}" with the api key you obtained. DO NOT USE THE '$' or the brackets '{' '}'. Example: ${API} becomes hj43gg061...

To create the above variable you can use the UnRAID webGui. Remember variables and values are case sensitive.

Once you have done that click apply and start the container. Open swag docker logs and check that the mod installation is successful and that the following line appears.

nginx: [alert] [lua] init_by_lua:8: [Crowdsec] Initialisation done

 

This means that the crowdsec mod is successfully installed and swag is now communicating with crowdsec docker.

If you want to double check that the communication between the two docker is successful you can run the following command

docker exec -t crowdsec cscli bouncers list

 The output is a table with the name,latest time and auth method with the bouncer.

 

CONCLUSION

---------------------

 

Now everytime a connection attempt is made, swag will communicate with crowdsec docker, check if the ip is banned and allow/reject the connection.

You can check the crowdsec decisions by running the following command:

docker exec -t crowdsec cscli decisions list

 

If you want to test the ban you can use the cscli command to ban the ip. Example:

docker exec -t crowdsec cscli decisions add --ip [IP] --type ban --duration 15m    <--- Replace IP with your IP

 

Then if you try to connect to your swag instance from that IP you should receive a 403 error.

To unban the ip just run the following:

docker exec -t crowdsec cscli decisions delete --ip [IP]

 

That's all. Hopefully you find it useful. Please let me know if something is not clear and I'll do my best to improve it.

Have fun and stay safe!

 

REFERENCES

----------------------

Blocking malicious connections with crowdsec and swag

 

SWAG docker mod

 

Crowdsec COLLECTIONS

Edited by Mik3
Fixed formatting
  • Like 1
  • Thanks 1
Link to comment
37 minutes ago, Thee_Dewd said:

Thank you for adding this. I followed this step by step and I don't see any errors but I'm also still able to connect from an IP after I manually blocked said IP. 

 

Hmm, that's strange. Is the IP a local IP or a public one? Local IP aren't blocked.

Link to comment
1 hour ago, Thee_Dewd said:

i grabbed my public IP from my office and then blocked it. i was then still able to connect from my office. my set up is pretty simple. cloudflare front end, swag proxy and apps behind that. i have the real ip docker mod added to swag for traffice coming from cloudflare. 

 

I tested that myself again and I got the same behaviour. Looks like swag is not communicating or blocking the connection. I will investigate the issue and post the solution. I'll keep you updated. Thanks for the report.

Link to comment
7 hours ago, Mik3 said:

 

I tested that myself again and I got the same behaviour. Looks like swag is not communicating or blocking the connection. I will investigate the issue and post the solution. I'll keep you updated. Thanks for the report.

Having the same issue here. Same setup as the other user, just without the docker real ip mod.

 

Thanks

Link to comment
On 2/16/2023 at 6:41 PM, Thee_Dewd said:

just checking to see if you had time to sort out the issue on this? i'd love to use it but haven't found a good source for setting it up. 

I am still working on it. I need to try and setup swag from scratch to see if there is a problem with the custom configurations. Having a busy work schedule which is not helping at all. If you could have a little bit more patience I'll try to solve the issue as soon as I can.

Link to comment
On 2/16/2023 at 6:41 PM, Thee_Dewd said:

just checking to see if you had time to sort out the issue on this? i'd love to use it but haven't found a good source for setting it up. 

Ok Solved the issue. I tested and it's working. There was an error in the guide about setting the api key. You just have to set the api key without the ${}. I updated the guide. Let me know if you have further issues.

Link to comment

EDIT: turns out the error didn't matter, followed the rest of the guide and its working just fine. Thanks so much for putting this together I've been meaning to up my security game after just using swag for my reverse proxy and this is a nice step in the right direction! May try 2fa next or something.

 

Thanks for this! I followed guide so far and got this error:

 

time="25-02-2023 07:25:55" level=warning msg="Loaded 40 scenarios"

time="25-02-2023 07:25:55" level=info msg="loading acquisition file : /etc/crowdsec/acquis.yaml"

time="25-02-2023 07:25:55" level=warning msg="No matching files for pattern /var/log/nginx/*.log" type=file

time="25-02-2023 07:25:55" level=warning msg="No matching files for pattern ./tests/nginx/nginx.log" type=file

time="25-02-2023 07:25:55" level=info msg="Adding file /var/log/crowdsec/access.log to datasources" type=file

time="25-02-2023 07:25:55" level=info msg="Adding file /var/log/crowdsec/error.log to datasources" type=file

time="25-02-2023 07:25:55" level=info msg="Adding file /var/log/auth.log to datasources" type=file

time="25-02-2023 07:25:55" level=warning msg="No matching files for pattern /var/log/syslog" type=file

time="25-02-2023 07:25:55" level=warning msg="No matching files for pattern /var/log/apache2/*.log" type=file

time="25-02-2023 07:25:55" level=info msg="Starting processing data"

time="25-02-2023 07:25:55" level=warning msg="/var/log/auth.log is a directory, ignoring it." type=file

 

I attached a screenshot of my container configuration. Please let me know what I missed. Thanks!

Screenshot from 2023-02-25 09-32-14.png

Edited by pugnobellum
New info
Link to comment
  • 2 months later...

hi there,

 

i have the same problem with "No matching files for pattern".

 

I can manually block my IP and unblock it, so I guess crowdsec is working correct.

But it seems like my nextcloud logs won't be read!

 

Nextcloud-Parser seems to be active:

 

image.png.99020049e86e2a4fbebc6fcc2f48144f.png

 

But nextcloud logs aren't displayer here:

 

image.png.ef2a91457fb954e5f2fcc0358bafc421.png

 

And in cli it says:

 

image.thumb.png.b6111fdf691a3c445734a53aed446cc2.png

 

So I guess nextcloud logfile won't be read?

 

this is my acquis.yaml:

 

image.png.ebfbea053dd8b740789ed3730ff55a80.png

 

My questsions are:

 

1. how can I check nextcloud.log with crowdsec?

2. how can I verify for sure, that crowdsec is working on nextcloud? 

3. how can I test a malicious string or something on my nextcloud to see if my IP-Adress is getting banned through crowdsec?

 

Thanks for help :)

 

:::

EDIT:

OK, I've found it out myself :)

  

You have to create a path variable on nextcloud docker and crowdsec docker:

 

grafik.png.2cf2eac9a6a0b2f5001203d9ee35b950.png

 

Then you have to tell nextcloud to change the logfile path:

 

sudo -u abc touch /log/nextcloud.log

chmod 777 /log/nextcloud.log

 

Then Edit the config.php and insert the path:

nano /mnt/user/appdata/nextcloud/www/nextcloud/config/config.php

 

'loglevel' => 1,

'logfile' => '/log/nextcloud.log',

 

:::
 

Now I have another question :)


How can we test if crowdsec is doing something at all?

As far as I know, crowdsec can only prevent bruteforce for nextcloud (which is too bad, I was hoping, that crowdsec is a real intrusion prevention system and blocks more attack szenarios like "trying SQL Injections"....

 

nethertheless I now have the problem, that I can't test crowdsec for bruteforce attacks since nextcloud anti-bruteforce tool is blocking my tests BEFORE crowdsec is blocking anything.

 

Does anybody have an idea on how to test if crowdsec is doing its job?

How can I trigger a pattern to verify?

Edited by diederich89
Link to comment
On 5/8/2023 at 12:23 AM, diederich89 said:

hi there,

 

i have the same problem with "No matching files for pattern".

 

I can manually block my IP and unblock it, so I guess crowdsec is working correct.

But it seems like my nextcloud logs won't be read!

 

Nextcloud-Parser seems to be active:

 

image.png.99020049e86e2a4fbebc6fcc2f48144f.png

 

But nextcloud logs aren't displayer here:

 

image.png.ef2a91457fb954e5f2fcc0358bafc421.png

 

And in cli it says:

 

image.thumb.png.b6111fdf691a3c445734a53aed446cc2.png

 

So I guess nextcloud logfile won't be read?

 

this is my acquis.yaml:

 

image.png.ebfbea053dd8b740789ed3730ff55a80.png

 

My questsions are:

 

1. how can I check nextcloud.log with crowdsec?

2. how can I verify for sure, that crowdsec is working on nextcloud? 

3. how can I test a malicious string or something on my nextcloud to see if my IP-Adress is getting banned through crowdsec?

 

Thanks for help :)

 

:::

EDIT:

OK, I've found it out myself :)

  

You have to create a path variable on nextcloud docker and crowdsec docker:

 

grafik.png.2cf2eac9a6a0b2f5001203d9ee35b950.png

 

Then you have to tell nextcloud to change the logfile path:

 

sudo -u abc touch /log/nextcloud.log

chmod 777 /log/nextcloud.log

 

Then Edit the config.php and insert the path:

nano /mnt/user/appdata/nextcloud/www/nextcloud/config/config.php

 

'loglevel' => 1,

'logfile' => '/log/nextcloud.log',

 

:::
 

Now I have another question :)


How can we test if crowdsec is doing something at all?

As far as I know, crowdsec can only prevent bruteforce for nextcloud (which is too bad, I was hoping, that crowdsec is a real intrusion prevention system and blocks more attack szenarios like "trying SQL Injections"....

 

nethertheless I now have the problem, that I can't test crowdsec for bruteforce attacks since nextcloud anti-bruteforce tool is blocking my tests BEFORE crowdsec is blocking anything.

 

Does anybody have an idea on how to test if crowdsec is doing its job?

How can I trigger a pattern to verify?

 

What you did with nextcloud is partially correct. Everytime you want a service protected by crowdsec, you need the correct parser and the logs. However this is only half of the equation as once crowdsec analyze the log and finds something bad it will report back through the local API that this IP should be banned. However no automatic action are taken as nextcloud doesn't know that.

Depending on how you are hosting nextcloud: If it's through a reverse proxy like swag than additional configuration might be required. If you are using the incorporated web server then you need to do some additional configuration in the nextcloud docker itself. I can't help further as I don't use nextcloud and I don't know how you host your instance.

Try to post your question/scenario and I'll try my best to help.

 

Crowdsec protection depends on the parser. For example, nextcloud collection protects against bruteforce and enumeration.

 

For how to test:

Temporary disable the nextcloud anti-bruteforce tool so to let crowdsec parser do it's job. If it works it should block your attack if it's properly configured.

You can ban your ip and try to access your nextcloud instance, if the configuration is correct you shouldn't be able to reach it.

For a pattern to verify your best bet is to look at the nextcloud crowdsec parser documentation and see which pattern is looking for and replicate that yourself.

 

Hopefully this helps.

Link to comment

Hi Mik3,

 

thank you very much for your fast reply!

 

The idea with temporarily disabling the nextclowd own bruteforce protection did the trick: BOOM...crowdsec blocked my IP :)

I have never been so happy to see a "you got blocked" page haha :)

 

grafik.thumb.png.083eb24e6ac4226fc4e70db2ac7fd27b.png

 

I didn't find out how to turn-off the bruteforce protection on vaultwarden so far, but I guess, if I can see crowdsec reading the logfile in the commandline and since I configured it the same way as i did it with nextcloud, it will work, too.

 

I use SWAG as reverse-proxy and nextcloud, vaultwarden and so on are sitting behind SWAG.

 

I was hoping, that crowdsec can also watch for SQL-Injection tryouts and so on or if a nextcloud version gets vulnerable and there are public exploits, that crowdsec would download the pattern automatically and block these exploit-attempts.

I thought that would be possible since I read that crowdsec can also block hafnium exploit attempts (Exchange Zero Day).

The attacker in this case would try to add powershell commands to a URL-Variable to gain root access.

Someone on the internet said that crowdsec can see these attacks and block the IP of the attacker.

 

But I guess that is not possible on nextcloud parser and too much of a wish :)

Edited by diederich89
Link to comment
  • 6 months later...

Hi,

 

When trying to add the Crowsec mod I get this error:

 

**** Invalid tarball, could not download crowdsec bouncer **** 
s6-rc: warning: unable to start service init-mod-swag-crowdsec: command exited 1 ... 
/run/s6/basedir/scripts/rc.init: warning: s6-rc failed to properly bring all the services up! Check your logs (in/run/uncaught-logs/current if you have in-container logging) for more information. 
/run/s6/basedir/scripts/rc.init: fatal: stopping the container.

 

I'm quite lost and I don't know what to do...

Link to comment
  • 2 weeks later...

I followed all the steps and got Crowsec running with swag.  The only difference was I had to change my port from 8080 to another as I had a conflich.  Everythign appears connected and working.  I manually banned an IP in Crowsec yet i'm not blocked.  Traffic is still flowing.  Could that be due to the port change?  

Link to comment
  • 4 weeks later...

Hi, thanks for writing up this guide. I got everything working except for automatic bans.  It seems that manual bans are working just fine but not when I am trying to brute force passwords from my work laptop (vpn connected) or from the 5G connection from my phone.  Is there anything I'm missing here? Thanks again.

 

Here is the message I get during startup for the container:

time="09-01-2024 12:37:50" level=info msg="loading acquisition file : /etc/crowdsec/acquis.yaml"
time="09-01-2024 12:37:50" level=warning msg="No matching files for pattern /var/log/nginx/*.log" type=file
time="09-01-2024 12:37:50" level=warning msg="No matching files for pattern ./tests/nginx/nginx.log" type=file
time="09-01-2024 12:37:50" level=info msg="Adding file /var/log/crowdsec/access.log to datasources" type=file
time="09-01-2024 12:37:50" level=info msg="Adding file /var/log/crowdsec/error.log to datasources" type=file
time="09-01-2024 12:37:50" level=info msg="Adding file /var/log/auth.log to datasources" type=file
time="09-01-2024 12:37:50" level=warning msg="No matching files for pattern /var/log/syslog" type=file
time="09-01-2024 12:37:50" level=warning msg="No matching files for pattern /var/log/apache2/*.log" type=file
time="09-01-2024 12:37:50" level=info msg="Starting processing data"
time="09-01-2024 12:37:50" level=warning msg="/var/log/auth.log is a directory, ignoring it." type=file

 

Edited by jdiqwd
Added logs.
Link to comment
  • 1 month later...
On 12/18/2023 at 4:52 AM, MP715 said:

I followed all the steps and got Crowsec running with swag.  The only difference was I had to change my port from 8080 to another as I had a conflich.  Everythign appears connected and working.  I manually banned an IP in Crowsec yet i'm not blocked.  Traffic is still flowing.  Could that be due to the port change?  

 

Hello, same for me, it doesn't work. I have about 80 ip that have been block by crowdsec and these IP continue to log in my access.log

I tried a manual ban from one of my IP and I still get access to my servers

 

I'm also surprised to not see the bouncer Ip adress and version

image.png.92d756cbdbf67a3a365833bbdabf239a.png

 

API pull works once at docker startup and after it never change

on crowdsec dashboard the bouncer appears offline after one day

 

help is welcome

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.