gmihovics Posted January 13, 2020 Share Posted January 13, 2020 (edited) I was setting up some containers on my system and all of a sudden my docker service stopped responding and would not come back up. I deleted my docker.img and started recreating all my containers. Part way through creating containers became slow and then stopped working all together. When I checked the log from the webGui I noticed a ton of segfaults repeating endlessly. I have included a snippet below. Jan 12 21:02:33 DS5 kernel: Code: 41 c7 45 00 00 00 00 00 4c 89 e2 4c 89 f6 e8 37 fb 76 ff 49 39 c4 49 89 c7 0f 84 ab 02 00 00 e8 36 1f 00 00 41 8b 4d 00 85 c9 <89> 08 0f 85 c8 00 00 00 4d 85 ff c7 00 af 00 00 00 0f 84 89 00 00 Jan 12 21:02:34 DS5 kernel: mysqld[15565]: segfault at 0 ip 000055dd7f846200 sp 0000153f01467490 error 6 in mysqld[55dd7ea5a000+13ec000] I have also attached my diagnostic report below if it helps. I should also mention I was using the community applications (2020.01.11a) plugin to install most of these containers. UPDATE 1 I just rebooted the server and brought up all the containers without a problem. As soon as I tried to add another, the issues all started again. I managed to copy the error from my log and went to store it in a file in **appdata** and i was told by vi that the disk was full. I googled this issue and found this post. I'm going to try and follow this post and see if this resolves the issue. I also went back to the logs after this and found the endless loop of segfaults occurring again. Here is the error message I saved Jan 12 22:08:34 DS5 nginx: 2020/01/12 22:08:34 [error] 2259#2259: *546 upstream timed out (110: Connection timed out) while reading upstream, client: 127.0.0.1, server: , request: "POST /Apps/AddContainer?xmlTemplate=default:/tmp/community.applications/tempFiles/templates-community-apps/linuxserversRepository/dillinger.xml HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "localhost:8080", referrer: "http://localhost:8080/Apps/AddContainer?xmlTemplate=default:/tmp/community.applications/tempFiles/templates-community-apps/linuxserversRepository/dillinger.xml" ds5-diagnostics-20200112-2111.zip UPDATE 2 So I ran the first few commands in the link I posted and it seems one of my SSDs is way smaller than I remember and its full. # btrfs fi show /mnt/cache Label: none uuid: 1b175dd0-5756-4f33-87b6-986909856bce Total devices 2 FS bytes used 110.06GiB devid 1 size 465.76GiB used 111.79GiB path /dev/sdf1 devid 2 size 111.79GiB used 111.79GiB path /dev/sdg1 # btrfs fi df /mnt/cache Data, RAID1: total=109.76GiB, used=109.76GiB System, RAID1: total=32.00MiB, used=16.00KiB Metadata, RAID1: total=2.00GiB, used=312.55MiB GlobalReserve, single: total=111.08MiB, used=0.00B Would this be the cause of my segfaults? Edited January 14, 2020 by gmihovics Quote Link to comment
itimpi Posted January 13, 2020 Share Posted January 13, 2020 With 2 different size SSDs in the cache and configured as RAID1 (the default) for redundancy, the usable space is only that of the smaller drive. You either want to forgo the redundancy and switch to a RAID profile that allows all the space on both SSDs to be used or keep the redundancy but replace the smaller SSD with a larger one. Quote Link to comment
gmihovics Posted January 13, 2020 Author Share Posted January 13, 2020 Thanks itimpi, that's what I thought. New SSD is on the way. Thanks for the help. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.