strauberry

I have discovered one last thing worth mentioning while going through the effort to get my COTURN server working on SSL. It looks like the matrix synapse Docker package that we are using already contains a COTURN server in it by default: https://github.com/AVENTER-UG/docker-matrix This is why we see a whole bunch of warning and turn related output in the logs for the matrix container. It took me a while to realize what was actually happening. We probably don't want two different turn servers running like this? They do provide a way of changing this behavior via Environment Variable of COTURN_ENABLE="false" I'm testing out if everything is stable right now after adding this variable to my Matrix Container so we don't have two COTURN services running:

Great news! I found it! This was one of those situations where I just had too much going on, too many moving parts and variables to see the simple thing right in front of me. It turns out that PFSense had automatically created it's firewall rules for the COTURN stuff under a default deny rule. I just didn't catch it right away. Once I moved the ordering of my firewall rules around, communication worked perfectly. So that brings us to the very end of this guide! Anyone who is considering implementing their own Matrix Synapse server in 2023 can be assured that this process here does work (there be dragons). It is a bit of a heavy lift in terms of implementation but ultimately worth the blood sweat and tears once you have your home server running. Matrix is amazing, Element is amazing. I couldn't be happier. Big shoutout to @Gazeley for the help and contribution of this guide. I think I'm still going to pursue working out the final console warning regarding the COTURN SSL certs just to get it out of the way. But otherwise all the basic functionality is running great now.

A lot to unpack here with the COTURN server. Here are my findings thus far: The config file in the example has some stuff in it that is not really being parsed out well: The log is throwing out: CONFIGURATION ALERT: You specified --lt-cred-mech and --use-auth-secret in the same time. Be aware that you could not mix the username/password and the shared secret based auth methods. Shared secret overrides username/password based auth method. Check your configuration! So we should probably toss the `lt-cred-mech` option in the turnserver.conf it just seems to be throwing an error. I'm also not really clear on some of the reasoning behind the attribute settings in the turnserver.conf file such as "external-ip". It seems like this would expect the WAN address or the hostname instead of the internal static binding we gave the container. But there was probably a good reason you guys set it that way. I'm just confused. The good news: We were able to make a connection over LAN inside out network. So we can assume that the very basic functionality of COTURN is working and implemented now. The bad news: NAT traversal issues. We cannot establish connections with people outside of our firewall. I have done some pretty extensive basic probe testing through nc and ntop to verify that the main listener ports are open. Those looked fine. I couldn't really get any positive reading off the range we specified (49152-49172). I have no idea what I'm doing, oh well #hacktheplanet My journey continues...

Alright! So we have arrived at the final step in the guide. I've hit a couple of problems in the COTURN section. The first one was very minor: - Guide doesn't mention that this `turnserver.conf` is actually a directory in the image (kind of odd, looked like a mistake to me). We say here to "edit the turnserver.conf" file but in my most recent run through when I created the coturn image there was no actual config file here. It was just a directory named `turnserver.conf` which I"m assuming the author intended on you creating the config file with the same name inside if that directory? - This actually kind of through me in a roundabout where I first deleted the directory because I thought it was a mistake, and then the container wouldn't restarted so I had to update the config volume mapping. I'm partially wondering if I need to just start over at this point and test again with `/turnserver.conf/turnserver.conf` which I think is what they expect by default. So right, got through that hiccup and the server starts - I am getting back ping responses from the static server address I assigned to the COTURN container. I've updated the config file with all the things. We've got some certificate related warning in the logs, but otherwise nothing major sticks out to me here. 0: : WARNING: cannot find certificate file: turn_server_cert.pem (1) 0: : WARNING: cannot start TLS and DTLS listeners because certificate file is not set properly 0: : WARNING: cannot find private key file: turn_server_pkey.pem (1) 0: : WARNING: cannot start TLS and DTLS listeners because private key file is not set properly Here's where we're stuck now: Attempted a test call and we cannot actually make connections with my other users. Right not it just hangs at "Connecting....". We get a dial tone and the system does recognize that VOIP is enabled on my Matrix home server, but I suspect there is just some more firewall issues for me to work through. Using an external port checking utility, we observe 3478 as being closed for me at the moment. So my next course of action is to try to learn how to use the https://icetest.info/ tool you mentioned and see if I can get to the bottom of the final connectivity issue here. Getting so close now!

Thank you so much @Gazeley this was the missing piece. It was an enormous relief to wake up to your message here today. EDIT: ignore the 1:1 checkbox, that was not necessary I wound up removing that setting when I learned more about it Although admittedly I do not yet really have a full comprehension of why this worked, these settings just fixed all my problems: - Set NAT Reflection mode for Port Forwards to "Pure NAT" - Enable NAT Reflection for 1:1 NAT - Enable automatic outbound NAT for Reflection I can now access synapse-admin without issue, along with the element-ui subdomain in my browser. Everything just works now as expected. I will be able to move forward in the guide now long to the COTURN section and begin making customizations to my homeserver.yaml now. So much appreciated, have a great week.

Me again, Everything was good up until this point, I got a positive test result from https://federationtester.matrix.org/ with no errors, however... The next snag I hit in the guide was when we got to the part about setting up `synapse-admin` and `element-web`. Neither of these worked for me, but luckily I got reproducible errors output in the JavaScript console for both services that indicated a CORS issue: Access to fetch at 'https://matrix.yourdomain.com/.well-known/matrix/client' from origin 'http://1.2.3.4:8075' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled. It has been a long time since I've diagnosed a CORS related issue. I'm not the greatest web host. But I *think* the right thing to do here is to amend the `matrix.subdomain.conf` file we added to our swag proxy-confs. I'm going to take a stab at this and report back. EDIT 01: - Tried all sorts of crazy things to the SWAG Nginx configs with no success yet. Just can't seem to get beyond CORS. WIll continue hacking... EDIT 02: - Tapping out for now possibly. I did all sorts of experimentation with CORS headers using `curl` requests and found that there is just something going on with my matrix subdomain specifically. CURL is reporting a (60) SSL certificate problem only on this subdomain. It is completely fine for the `element` subdomain and I can add headers there without issue in nginx. Hope to have better luck another day. EDIT 03: - So my server is working... kindof. I decided to test how things would behave outside my network on my LTE connection and I got way different results. Also just learned that according to https://github.com/vector-im/element-web/ they don't even recommend running this application on the same domain. Maybe will just kind of put my focus elsewhere for now. I was able to connect to my Matrix homeserver from my personal LTE device outside the network using the admin I created during setup. So the basic functionality is working... - It seems like (at least for me) this guide resulted in a situation where curl tests have SSL problems when they are performed inside the same network. When I curl from my LTE device for example, I have no issues. When I curl from inside the network to my matrix subdomain it complains about cert problems. At this point we are *could* be experiencing something at my firewall level. I do notice the pfsense trips a DNS Rebinding attack warning when I try to navigate directly to the matrix subdomain. I will continue to explore all these topics and hopefully make more progress tomorrow.

Hello it's me again, providing another update to a snag I hit in this guide. It appears that default permissions aren't working as expected. I have reached out to Unraid's package maintainer A75G and let him know what I found. The `matrix` package we use in Unraid's CA repo is going to need a little bit of a touchup moving forward in 2023. https://github.com/A75G/docker-templates/issues/77 Basically, if you look at the Container log you will see out of box permissions issues. The steps I took to resolve are attached inside that Github issue link. I had to manually modify some folder permissions to where the `/data` volume mapping was. And then I needed to update the `log.config` file to change the path where it attempts to create `homeserver.log` to exist in `/data/homeserver.log`. After restarting the container and making the modifications here, I was able to proceed beyond the step of running the following command: register_new_matrix_user http://localhost:8008 -c /data/homeserver.yaml Before doing this I would just get a massive traceback because the service running on 8008 was not even starting.

Hello @Gazeley and first of all, thank you for taking the time to publish this guide here. I'm following your setup here now in March of 2023 and it seems as though some new requirements may have been added to the `homeserver.yaml` specification. Upon trying to execute: register_new_matrix_user http://localhost:8008 -c /data/homeserver.yaml We get: No 'registration_shared_secret' or 'registration_shared_secret_path' defined in config. I'm new to Matrix Synapse, but I'm assuming there has been some change in requirements from the server side since the time this guide was written. In order to get beyond this error you will now need to add a fourth secret key, like how Gazeley shows us during the `homeserver.yaml` generation. # The shared secret used for the Registration of new users registration_shared_secret: "yourrandomlygeneratedstring" I just added this to the bottom of my config file, using the same shell command we used to generate the other random strings as the key. This is where I'm at in the guide now, I will continue to post any new snags or stuck points as I continue following along here in order to preserve the relevance of this article and help others with their Matrix journeys. Cheers!