This issue has been reported a few times before, and I could have sworn I had fixed it.
Just tried to reproduce it, but it successfully removed the container from the folder are you on the latest version of the plugin?
edit:
Think i will add a way to remove containers from the ui even if they dont exist anymore, i really should have done that from the start