iilied Posted May 14, 2021 Share Posted May 14, 2021 server has been running for 88 days w/o issues (version 6.8.3), but all of a sudden now it lists “not available” in the docker containers tap. tried forcing update on multiple containers, and stoping and starting the array, to no avail. first time facing this issue and not quite sure a restart would fix this, or where to start. tower-diagnostics-20210514-0713.zip Quote Link to comment
eric.frederich Posted May 14, 2021 Share Posted May 14, 2021 I'm having this issue as well. I'm also on 6.8.3... I haven't updated my images in 2 months. Now every one of them is showing "not available". I hopped on #unraid IRC channel on FreeNode and asked there too. It's pretty quiet there, but the one person who did check was also having the issue... again on 6.8.3 I wouldn't assume it's a problem with this specific version until we hear from someone on a different release saying "it works for me". That hasn't happened yet. Quote Link to comment
HyperV Posted May 14, 2021 Share Posted May 14, 2021 I had exactly the same issue and could not find any solutions on the forum or the internet. So I did some digging myself and found the cause of the issue. The docker update check script gets the remote digest of the latest tag from the docker repository via a header called 'Docker-Content-Digest'. The script checks for this header with a case-sensitive regex pattern. Manually querying the docker hub registry gives me a header called 'docker-content-digest' (mind the casing). The docker hub registry must have recently changed the casing of this header, because it broke for me in the last 24 hours. I'm running on Unraid 6.8.3 still, so I'm not 100% sure if this issue also exists in 6.9.x. If you feel up to it, you could quite easily fix this yourself until there is a real fix. I'll describe the steps below: Open file: /usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php Go to line 457. There you should look for the text: @Docker-Content-Digest:\s*(.*)@ and replace it with: @Docker-Content-Digest:\s*(.*)@i Save the file. This will make the header check case-insensitive and should make it work again. 11 29 Quote Link to comment
ephdisk Posted May 14, 2021 Share Posted May 14, 2021 7 minutes ago, HyperV said: I had exactly the same issue and could not find any solutions on the forum or the internet. So I did some digging myself and found the cause of the issue. The docker update check script gets the remote digest of the latest tag from the docker repository via a header called 'Docker-Content-Digest'. The script checks for this header with a case-sensitive regex pattern. Manually querying the docker hub registry gives me a header called 'docker-content-digest' (mind the casing). The docker hub registry must have recently changed the casing of this header, because it broke for me in the last 24 hours. I'm running on Unraid 6.8.3 still, so I'm not 100% sure if this issue also exists in 6.9.x. If you feel up to it, you could quite easily fix this yourself until there is a real fix. I'll describe the steps below: Open file: /usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php Go to line 457. There you should look for the text: @Docker-Content-Digest:\s*(.*)@ and replace it with: @Docker-Content-Digest:\s*(.*)@i Save the file. This will make the header check case-insensitive and should make it work again. Thanks for the deep dive @HyperV This fixed it for me! I was just able to refresh and update any containers that I didn't manually force. I am still on 6.8.3 because I use custom IP addresses in my docker environment and I have seen that it is still causes lock ups in 6.9.x Cheers Quote Link to comment
eric.frederich Posted May 14, 2021 Share Posted May 14, 2021 Awesome work @HyperV thanks. It worked for me. Where is that code maintained? I wanted to see if it was fixed in a newer version of if we should file a bug. All I found was this which seems to be 5 years old and completely out of date. https://github.com/limetech/dynamix The DockerClient.php in that repo doesn't even have any Docker-Content-Digest. Quote Link to comment
bluuurb Posted May 14, 2021 Share Posted May 14, 2021 2 hours ago, HyperV said: I had exactly the same issue and could not find any solutions on the forum or the internet. So I did some digging myself and found the cause of the issue. The docker update check script gets the remote digest of the latest tag from the docker repository via a header called 'Docker-Content-Digest'. The script checks for this header with a case-sensitive regex pattern. Manually querying the docker hub registry gives me a header called 'docker-content-digest' (mind the casing). The docker hub registry must have recently changed the casing of this header, because it broke for me in the last 24 hours. I'm running on Unraid 6.8.3 still, so I'm not 100% sure if this issue also exists in 6.9.x. If you feel up to it, you could quite easily fix this yourself until there is a real fix. I'll describe the steps below: Open file: /usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php Go to line 457. There you should look for the text: @Docker-Content-Digest:\s*(.*)@ and replace it with: @Docker-Content-Digest:\s*(.*)@i Save the file. This will make the header check case-insensitive and should make it work again. Works like a charm, thanks Quote Link to comment
Squid Posted May 14, 2021 Share Posted May 14, 2021 9 minutes ago, eric.frederich said: Awesome work @HyperV thanks. It worked for me. Where is that code maintained? I wanted to see if it was fixed in a newer version of if we should file a bug. All I found was this which seems to be 5 years old and completely out of date. https://github.com/limetech/dynamix The DockerClient.php in that repo doesn't even have any Docker-Content-Digest. 6.9.2 has the updated code already. But what you're looking for is https://github.com/limetech/webgui 1 1 Quote Link to comment
HyperV Posted May 14, 2021 Share Posted May 14, 2021 3 minutes ago, Squid said: 6.9.2 has the updated code already. But what you're looking for is https://github.com/limetech/webgui Good to know this has been fixed already in later versions 👍 Quote Link to comment
Unraid-arr Posted May 14, 2021 Share Posted May 14, 2021 3 hours ago, HyperV said: I had exactly the same issue and could not find any solutions on the forum or the internet. So I did some digging myself and found the cause of the issue. The docker update check script gets the remote digest of the latest tag from the docker repository via a header called 'Docker-Content-Digest'. The script checks for this header with a case-sensitive regex pattern. Manually querying the docker hub registry gives me a header called 'docker-content-digest' (mind the casing). The docker hub registry must have recently changed the casing of this header, because it broke for me in the last 24 hours. I'm running on Unraid 6.8.3 still, so I'm not 100% sure if this issue also exists in 6.9.x. If you feel up to it, you could quite easily fix this yourself until there is a real fix. I'll describe the steps below: Open file: /usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php Go to line 457. There you should look for the text: @Docker-Content-Digest:\s*(.*)@ and replace it with: @Docker-Content-Digest:\s*(.*)@i Save the file. This will make the header check case-insensitive and should make it work again. Much Thanks for this. Was going crazy with this as I am on 6.8.3 as well. Issue started yesterday for me as well. This needs to be escalated more so it is updated in the official releases. Quote Link to comment
Ford Prefect Posted May 14, 2021 Share Posted May 14, 2021 7 hours ago, HyperV said: The docker hub registry must have recently changed the casing of this header, because it broke for me in the last 24 hours. ...so, is this something under comtrol by LT or who actually did create that change and root cause of this problem? 7 hours ago, HyperV said: If you feel up to it, you could quite easily fix this yourself until there is a real fix. I'll describe the steps below: Open file: /usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php ...many thanks for that temporary Fix! But I assume this will not survive a reboot, does it? Quote Link to comment
HyperV Posted May 14, 2021 Share Posted May 14, 2021 (edited) 18 minutes ago, Ford Prefect said: ...so, is this something under comtrol by LT or who actually did create that change and root cause of this problem? ...many thanks for that temporary Fix! But I assume this will not survive a reboot, does it? Well, it's the docker hub registry that changed something. However, the HTTP RFC spec states headers are case-insensitive, so the script is at fault and this script is under the control of LT. Unfortunately this temporary fix is indeed volatile, it will not survive a reboot. The script is unpacked from the image files to ram on each boot. A permanent fix would have to be made by LT. However, as @Squid pointed out, the current source of the webgui of Unraid, which contains this script, already has updated code for this. It appears someone else at some point revisited the code and came to the conclusion as well that the code was faulty and that headers should be checked case-insentively before this ever became a real problem. Now it has actually become a problem for people still running older versions because the casing of the header changed, so this fix should probably be back-ported or we are stuck with this temporary solution. Edited May 14, 2021 by HyperV 5 Quote Link to comment
Ford Prefect Posted May 14, 2021 Share Posted May 14, 2021 6 minutes ago, HyperV said: Unfortunately this temporary fix is indeed volatile, it will not survive a reboot. The script is unpacked from the image files to ram on each boot. A permanent fix would have to be made by LT. Hmmm..OK, many thanks for your fast responce. I doubt, that LT will fix it, as 6.9.2 is available.... @ich777 do you think, one can apply a patch to the script and merge it into the image files whith your Kernel Helper plugin? Or is it even more simple and just a matter of changeing the filename in the image archive file, as (maybe) checksums are not checked here? Quote Link to comment
ich777 Posted May 14, 2021 Share Posted May 14, 2021 9 hours ago, Ford Prefect said: @ich777 do you think, one can apply a patch to the script and merge it into the image files whith your Kernel Helper plugin? Or is it even more simple and just a matter of changeing the filename in the image archive file, as (maybe) checksums are not checked here? I think the easiest solution would be to add these two lines to your go file (/boot/config/go): Use the solution 3 post below! This will make the temporary solution from @HyperV more or less permanent since this is applied on every boot of the server and no physical change to the file are needed because that's all done from the go file. Hope that also helps and is appropriate to this thread. 2 Quote Link to comment
Ford Prefect Posted May 14, 2021 Share Posted May 14, 2021 10 minutes ago, ich777 said: This will make the temporary solution from @HyperV more or less permanent since this is applied on every boot of the server and no physical changes to the files are needed. ...perfect! Thank you very much for fast response and support! 1 Quote Link to comment
JonathanM Posted May 14, 2021 Share Posted May 14, 2021 @ich777, can you please confirm the sed syntax? Quote Link to comment
Morthan Posted May 14, 2021 Share Posted May 14, 2021 14 hours ago, ich777 said: I think the easiest solution would be to add these two lines to your go file (/boot/config/go): **REMOVED** This will make the temporary solution from @HyperV more or less permanent since this is applied on every boot of the server and no physical change to the file are needed because that's all done from the go file. Hope that also helps and is appropriate to this thread. Pardon my rudeness but your sed command replaces the entire line with just that text. I adjusted it a bit and use this instead, so it only replaces the found text ( I almost always use different delimiters because the slashes get in the way of seeing what is being done with the backslashes in the way) sed -i 's#@Docker-Content-Digest:\\s*\(.*\)@#\@Docker-Content-Digest:\\s*\(.*\)@i#g' /usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php I used https://sed.js.org/ to check the syntax 1 5 Quote Link to comment
ephdisk Posted May 15, 2021 Share Posted May 15, 2021 1 hour ago, Morthan said: Pardon my rudeness but your sed command replaces the entire line with just that text. I adjusted it a bit and use this instead, so it only replaces the found text ( I almost always use different delimiters because the slashes get in the way of seeing what is being done with the backslashes in the way) sed -i 's#@Docker-Content-Digest:\\s*\(.*\)@#\@Docker-Content-Digest:\\s*\(.*\)@i#g' /usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php I used https://sed.js.org/ to check the syntax @Morthan@ich777 Thank you both for trying to make a permanent fix to apply at boot. Morthan your output is perfect: preg_match('@Docker-Content-Digest:\s*(.*)@ii', $reply, $matches); ich777 your output removes important content from the line: @Docker-Content-Digest:\s*(.*)@i 1 Quote Link to comment
ich777 Posted May 15, 2021 Share Posted May 15, 2021 10 hours ago, Morthan said: Pardon my rudeness but your sed command replaces the entire line with just that text. No problem, and yes I know, see the second answer below. 9 hours ago, ephdisk said: ich777 your output removes important content from the line: Exctly because I thought this is the full line because I have no such old system to test with. @Ford Prefect please change the line to the answer two post above from @Morthan. 1 1 Quote Link to comment
kennygunit Posted May 15, 2021 Share Posted May 15, 2021 21 hours ago, HyperV said: Open file: /usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php Pardon the newbie question, but where should i be looking for this file? I don't see it even by searching my servers root directory 1 Quote Link to comment
ephdisk Posted May 15, 2021 Share Posted May 15, 2021 (edited) 3 hours ago, kennygunit said: Pardon the newbie question, but where should i be looking for this file? I don't see it even by searching my servers root directory @kennygunit I was able to SSH in to my server. But this will only last until you reboot. sudo nano /usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php The permanent way would be to modify the boot via SSH: sudo nano /boot/config/go Paste this at the end: (Shift Insert or Right Click) # Fix Docker - Case Insensitive sed -i 's#@Docker-Content-Digest:\\s*\(.*\)@#\@Docker-Content-Digest:\\s*\(.*\)@i#g' /usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php Ctrl X and save then Reboot. All thanks goes to HyperV, Morthan, and ich777 ❤️ Edited May 15, 2021 by ephdisk 9 4 Quote Link to comment
Ford Prefect Posted May 15, 2021 Share Posted May 15, 2021 thanks again everyone, especially @HyperV, @ich777, @Morthan and @ephdisk for your help. I am glad that any easy, non-intrusive fix has been found, as I am still reluctant to move away from 6.8.3. 3 Quote Link to comment
servidude Posted May 16, 2021 Share Posted May 16, 2021 23 hours ago, Ford Prefect said: thanks again everyone, especially @HyperV, @ich777, @Morthan and @ephdisk for your help. I am glad that any easy, non-intrusive fix has been found, as I am still reluctant to move away from 6.8.3. What @Ford Prefect said! I too am still using 6.8.3 (still waiting for another point release or two before moving to 6.9) and very grateful to you all for your help with fixing this issue. Quote Link to comment
cinereus Posted May 17, 2021 Share Posted May 17, 2021 Thanks all, this fixed it. What will cause a proper fix to be made so I can remove the line from my gofile? Do I have the upgrade the unraid version entirely? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.