If you can't get this container secured to your satisfaction, you could do as I do and not run it unless you are actually using it. Not the greatest answer, but it certainly limits the exposure.
Another "security through obscurity" trick is to have another inconsequential container set to use the same port as this one, or even a second copy of this container, duplicated EXCEPT for the permissions of the mount point. Set the no permission container to auto start, then when you need to edit files you shut down that container and start up the dangerous one, when done restart the other.
That way the fully privileged container can't be started unless the auto run container is stopped.
The starting and stopping could be scripted, and triggered by a shortcut on your daily driver. Click an icon, do your file maintenance, click another icon, server secured.(ish)