Vitreous Humour

Ok, I was able to figure out how to do it on my own! It wasn't great, but it is definitely doable. I very much used https://blog.janissary.xyz/posts/tailscale-oidc-authelia-carpal as a framework for my approach. But in case anyone else wants to venture down this path I'll leave some notes here.

I purchased a domain name (I used cloudflare as the DNS provider, right now free.) and then used ddclient to effectively make it target my dynamic IP.
I already had a SWAG instance, but I did have to go through and update the various configs (nginex and subdomain configs) to updated templates in order to get Authelia to work. Oh, and use auth.domain.xyz, not the old authelia.* subdomain.
Authelia was the most PITA thing to set up. All the youtube guides and most of the documentation on authelia (especially the IBRACORP guide) are horribly out of date, and almost nothing works that way anymore. For anyone struggling with it, you can use the IBRACORP template, but then you need to turn on debug logging and then slowly but surely walkthrough the errors that it throws, correcting each one (and particularly looking at the authelia documentation to figure out what has changed and what is new). There was also a problem with the redis container they suggest not being able to properly write out some files because of permissions, which you would only know by looking at the youtube comments for that video.
Make Authelia an OIDC issuer per the janissary.xyz guide. It won't work right out of the box because authelia has broken that config in a later update, much of it is just re-writing the yml file to match authelia's new config framing. You will need to create some secrets, which authelia's guide will walk you through pretty well, though I was not able to get authelia to properly read a secrets file, so they needed to be in-line.
I pulled the Carpal docker image from https://hub.docker.com/r/peeley/carpal. Use the config options in the janissary.xyz and peeley/carpal github to help set it up, but it was pretty easy. One pitfall I ran into is that for some reason Tailscale WON'T query properly if you have capital letters in your email and the matching acct file! This was driving me crazy and throwing a 404 error, but you need to have the emails be lower case.
add the carpal webfinger server in the location section of the default.conf file in SWAG (as per the janissary.xyz guide) to allow your base domain to respond to a query at .well-known/webfinger with the proper JSON response. You should check on the webfinger website that it's properly responding when you put in the email you want Tailscale to register you at. Make sure that in the email account file, that you have the href pointing to your authelia subdomain (which should be set properly downstream of your SWAG proxy server).
Hopefully at that point the tailscale account setup will get a proper response from your carpal server that points it to your authelia instance. It will continue to do this whenever you need to login to Tailscale.
Ugh, honestly authelia was the biggest bear here, it took over a week of fiddling to get it to work right. Unfortunately there don't seem to be many "easier" OIDC setups, I considered Authentik but it seemed even MORE fiddly than authelia to get set up reliably. Good luck!