• Errors in Docker extra parameters cause container to vanish and config data to be lost


    ElectricBadger
    • Closed

    If an error is made in the Extra Parameters setting when configuring a Docker container, the container is not added to the list of containers. If an error is made when editing this setting for an existing container, the container is removed from the list of installed containers and the user configuration is lost. There is no warning or confirmation before this happens.

     

    To reproduce:

     

    • Install the GitLab CE container and start it.
    • Edit the container settings.
    • Add the following to Extra Parameters:

     

    --env GITLAB_OMNIBUS_CONFIG="external_url 'https://git.example.com';nginx['custom_gitlab_server_config'] = "location /-/plantuml/ { \n    proxy_cache off; \n    proxy_pass  http://plantuml:8080/; \n}\n";prometheus_monitoring['enable']=false;"

     

    Note that the code pasted from https://docs.gitlab.com/ee/administration/integration/plantuml.html uses double quotes which should have been escaped, as it's already inside double quotes. An easy error to miss, but one with a hefty penalty, as it causes the container creation to fail. This causes the container to vanish; when reinstalled, any existing configuration changes are lost and the container has its default settings once more.

     

    Expected behaviour: the container remains installed, in a stopped state, with its existing config. The user is free to edit that config until they come up with one that works.

     

    I'm marking this as Urgent as it is, technically, a data loss bug (while I didn't lose any data from the container's volumes, I did have to recreate all the config options. If I didn't have a copy of that Extra Parameters, which I've pasted a sanitised version of below, it would have taken hours to get the container up and running again.)

     

    --env GITLAB_OMNIBUS_CONFIG="external_url 'https://git.example.com';registry_external_url 'https://registry.example.com';gitlab_rails['usage_ping_enabled']=false;gitlab_rails['lfs_enabled']=true;gitlab_rails['gitlab_ssh_host']='git.example.com';gitlab_rails['gitlab_email_from']='[email protected]';gitlab_rails['smtp_enable']=true;gitlab_rails['smtp_address']='mail.example.com';gitlab_rails['smtp_port']=25;gitlab_rails['smtp_authentication']=false;gitlab_rails['smtp_tls']=false;gitlab_rails['smtp_openssl_verify_mode']='none';gitlab_rails['smtp_enable_starttls_auto']=false;gitlab_rails['smtp_ssl']=false;gitlab_rails['smtp_force_ssl']=false;gitlab_rails['manage_backup_path']=false;gitlab_rails['backup_path']='/backups';gitlab_rails['backup_archive_permissions']=0644;gitlab_rails['backup_pg_schema']='public';gitlab_rails['backup_keep_time']=604800;nginx['listen_port']=9080;nginx['listen_https']=false;nginx['hsts_max_age']=0;nginx['client_max_body_size']='0';nginx['custom_gitlab_server_config']='location /-/plantuml/ {\nproxy_cache off;proxy_pass http://plantuml:14480/;\n}\n';registry_nginx['listen_port']=9381;registry_nginx['listen_https']=false;registry_nginx['client_max_body_size']='0';registry_nginx['enable']=true;registry['enable']=true;gitlab_rails['pipeline_schedule_worker_cron'] = '7,37 * * * *';prometheus_monitoring['enable']=false;postgresql['auto_restart_on_version_change']=false"

     

    celestia-diagnostics-20220801-0905.zip




    User Feedback

    Recommended Comments

    Seems more like a usage issue. 

    Containers are ephemeral, they'll be deleted and recreated frequently (e.g. changing any option, each container update) so they should NOT store anything that's supposed to be persistent.

    If it's got some specific config it should be mapped to a volume and stored externally, which would in turn be mapped to appdata.

     

    Basically if deleting and recreating the container causes loss of data something's set up wrong.

    Edited by Kilrah
    Link to comment

    After reinstalling the container, it seemed to zero the databases and start afresh, even though it was pointed at the same appdata locations. It's a good job I had a backup, even though it took two tries to restore the user database and I still don't have the projects restored!

     

    Mismatching quotes in an environment variable absolutely should not cause this level of data loss.

    Edited by ElectricBadger
    Link to comment

    Couple of comments.

     

    Any error in the docker run command (and after editing and apply with your changes)

    root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='GitLab-CE' --net='bridge' -e TZ="America/New_York" -e HOST_OS="Unraid" -e HOST_HOSTNAME="unRaidB" -e HOST_CONTAINERNAME="GitLab-CE" -l net.unraid.docker.managed=dockerman -l net.unraid.docker.webui='http://[IP]:[PORT:9080]' -l net.unraid.docker.icon='https://raw.githubusercontent.com/tynor88/docker-templates/master/images/gitlab_small.png' -p '9080:9080/tcp' -p '9443:9443/tcp' -p '9022:22/tcp' -v '/mnt/cache/appdata/gitlab-ce/config':'/etc/gitlab':'rw' -v '/mnt/cache/appdata/gitlab-ce/data':'/var/opt/gitlab':'rw' -v '/mnt/cache/appdata/gitlab-ce/log':'/var/log/gitlab':'rw' --env GITLAB_OMNIBUS_CONFIG="external_url 'https://git.example.com';nginx['custom_gitlab_server_config'] = "location /-/plantuml/ { \n proxy_cache off; \n proxy_pass http://plantuml:8080/; \n}\n";prometheus_monitoring['enable']=false;" 'gitlab/gitlab-ce'
    sh: line 1: n}n;prometheus_monitoring['enable']=false;: command not found
    
    The command failed.

    Results in the container "disappearing".  This is because for all intents and purposes it does NOT exist anymore because the installation (ie: the docker run command) failed

     

    The appropriate method to get the app back installed is via Apps - Previous Apps and then it would show up the template with the changes you had made so that you can edit them appropriately to fix the error.  

     

    Doing a search in Apps for it would also give you the option to either Reinstall from Previous or Install -> Install sets the template back to defaults and depending upon what paths etc you had made would result in data loss

     

    Technically the appropriate method to add an envronment variable is via the Add Another Path / Port / Variable and enter in the appropriate entry there.  The system will automatically escape everything accordingly.  Adding variables via the Extra Parameters does work however

    Link to comment
    7 hours ago, ElectricBadger said:

    After reinstalling the container, it seemed to zero the databases and start afresh, even though it was pointed at the same appdata locations. It's a good job I had a backup, even though it took two tries to restore the user database

    That would be an issue with the container. 

    Link to comment
    22 hours ago, Squid said:

    Couple of comments.

     

    Any error in the docker run command (and after editing and apply with your changes)

    root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='GitLab-CE' --net='bridge' -e TZ="America/New_York" -e HOST_OS="Unraid" -e HOST_HOSTNAME="unRaidB" -e HOST_CONTAINERNAME="GitLab-CE" -l net.unraid.docker.managed=dockerman -l net.unraid.docker.webui='http://[IP]:[PORT:9080]' -l net.unraid.docker.icon='https://raw.githubusercontent.com/tynor88/docker-templates/master/images/gitlab_small.png' -p '9080:9080/tcp' -p '9443:9443/tcp' -p '9022:22/tcp' -v '/mnt/cache/appdata/gitlab-ce/config':'/etc/gitlab':'rw' -v '/mnt/cache/appdata/gitlab-ce/data':'/var/opt/gitlab':'rw' -v '/mnt/cache/appdata/gitlab-ce/log':'/var/log/gitlab':'rw' --env GITLAB_OMNIBUS_CONFIG="external_url 'https://git.example.com';nginx['custom_gitlab_server_config'] = "location /-/plantuml/ { \n proxy_cache off; \n proxy_pass http://plantuml:8080/; \n}\n";prometheus_monitoring['enable']=false;" 'gitlab/gitlab-ce'
    sh: line 1: n}n;prometheus_monitoring['enable']=false;: command not found
    
    The command failed.

    Results in the container "disappearing".  This is because for all intents and purposes it does NOT exist anymore because the installation (ie: the docker run command) failed

     

    The appropriate method to get the app back installed is via Apps - Previous Apps and then it would show up the template with the changes you had made so that you can edit them appropriately to fix the error.  

     

    Doing a search in Apps for it would also give you the option to either Reinstall from Previous or Install -> Install sets the template back to defaults and depending upon what paths etc you had made would result in data loss

     

    Technically the appropriate method to add an envronment variable is via the Add Another Path / Port / Variable and enter in the appropriate entry there.  The system will automatically escape everything accordingly.  Adding variables via the Extra Parameters does work however

     

    Thanks — I had forgotten about Previous Apps. That would probably have saved a lot of time 🤦‍♂️

     

    For some reason I didn't get the Reinstall from Previous option. I don't have a different version of the GitLab container in Previous Apps, so I'm guessing I did click on the same one that I was using previously, but it's probably too late to find out what happened there…

     

    I'm not even sure why that environment variable is added via Extra Parameters (I suspect I was following a set of install instructions that were incorrect) but I'll try adding it in the correct way when I have a bit more time. Not having to deal with the escaping is reason enough to make the switch 🙂

    Link to comment


    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.