I guess this is the solution:
Although I would say this is not exactly what I encountered, even without any container update or restart the default admin password still will not change using the webui.
Anyway thanks for the heads up! The solution in the readme totally solved the problem it seems. Next time I'll rtfm.