rodan5150 Posted April 21, 2021 Share Posted April 21, 2021 Hello all, I've been fighting a few issues since I "upgraded" to a Ryzen based system from an old dual Xeon Dell T410. New build is a Ryzen 3600, on a B450 chipset. It was getting unresponsive after a few days, and that seems to have gone away after disabling global C-states in the BIOS. The latest issue is I get a kernel panic after a week or so of uptime. I have syslog enabled, and was able to get it just before the panic. I also got a pic of the screen before I rebooted that tells the rest of the story after the syslog dropped. Looks like network related, maybe mac-vlan? Definitely a call trace happens, but not sure what can be done about it. thanks for any help in advance! Ross syslog 4-20-21.txt Quote Link to comment
John_M Posted April 21, 2021 Share Posted April 21, 2021 1 hour ago, rodan5150 said: Looks like network related, maybe mac-vlan? There are a lot of posts about macvlan call-traces related to docker containers with custom IP addresses, such as this one: 1 hour ago, rodan5150 said: disabling global C-states in the BIOS Don't do that. The only C-state that has ever been a problem with Ryzen processors is C6 and that only really affected the 1000-series. That said, it does no harm to find the BIOS setting that refers to Power Supply Idle Control and set it to Typical Current Idle instead of the default Low Current Idle. Quote Link to comment
rodan5150 Posted April 21, 2021 Author Share Posted April 21, 2021 8 hours ago, John_M said: There are a lot of posts about macvlan call-traces related to docker containers with custom IP addresses, such as this one: Don't do that. The only C-state that has ever been a problem with Ryzen processors is C6 and that only really affected the 1000-series. That said, it does no harm to find the BIOS setting that refers to Power Supply Idle Control and set it to Typical Current Idle instead of the default Low Current Idle. Awesome, thanks for letting me know. I will revert the global C-state setting, and dig around and see if I can find the idle current setting. Quote Link to comment
John_M Posted April 21, 2021 Share Posted April 21, 2021 5 hours ago, rodan5150 said: see if I can find the idle current setting I usually find it in the AMD CBS section of the menu but wherever you go to find the Global C-states setting, you might find it there too. Quote Link to comment
rodan5150 Posted April 22, 2021 Author Share Posted April 22, 2021 On 4/21/2021 at 11:55 AM, John_M said: I usually find it in the AMD CBS section of the menu but wherever you go to find the Global C-states setting, you might find it there too. That's exactly where it was. I enabled the C-states option, and then set the idle current to typical instead of low. I've also created a Docker specific vlan, and moved all of the br0 over to br0.x so hopefully that will keep my call traces and kernel panics at bay. I will update if anything changes. So far so good, but it has only been about 18 hours or so. Longest it has gone in the past was 10ish days. So if I can hit 2 weeks+ I'll consider it a win. 1 Quote Link to comment
rodan5150 Posted May 22, 2021 Author Share Posted May 22, 2021 Bad news. So kernel panic is back. I thought I had figured it out by moving everything from br0 to br0.x but it looks like there has to be another issue going on, causing the call traces that ultimately end in a kernel panic. What else could I be missing? Darkhelmet syslog 5-21.txt Quote Link to comment
JorgeB Posted May 22, 2021 Share Posted May 22, 2021 Still seems related to these: https://forums.unraid.net/topic/70529-650-call-traces-when-assigning-ip-address-to-docker-containers/ See also here: https://forums.unraid.net/bug-reports/stable-releases/690691-kernel-panic-due-to-netfilter-nf_nat_setup_info-docker-static-ip-macvlan-r1356/ Quote Link to comment
rodan5150 Posted May 22, 2021 Author Share Posted May 22, 2021 7 hours ago, JorgeB said: Still seems related to these: https://forums.unraid.net/topic/70529-650-call-traces-when-assigning-ip-address-to-docker-containers/ See also here: https://forums.unraid.net/bug-reports/stable-releases/690691-kernel-panic-due-to-netfilter-nf_nat_setup_info-docker-static-ip-macvlan-r1356/ thanks for the Reply JorgeB. I'm going to give the second NIC assignment a shot. I had been trying to do all of this through a single 10Gbe connection. I've got several 1Gbe ports open on my main switch, so its not a huge deal to just assign the containers to a second NIC. With any luck, this will solve it. Quote Link to comment
dauntouch Posted May 29, 2021 Share Posted May 29, 2021 Hey did you figure it out by any chance? I'm having the exact same issue, same cpu + mobo with a kernel panick. Tried disabling c states and the idle setting, it worked for a couple weeks then started crashing again. Quote Link to comment
John_M Posted May 30, 2021 Share Posted May 30, 2021 19 hours ago, dauntouch said: Tried disabling c states and the idle setting Don't do both. Re-enable the Global C States. See above. The kernel panic is not caused by the CPU failing to wake up from the C6 state. If that happens the sever just freezes, but the Power Supply Idle Control setting fixes that. As mentioned above (twice), the panic is likely related to docker networking. Quote Link to comment
rodan5150 Posted May 30, 2021 Author Share Posted May 30, 2021 On 5/29/2021 at 12:06 AM, dauntouch said: Hey did you figure it out by any chance? I'm having the exact same issue, same cpu + mobo with a kernel panick. Tried disabling c states and the idle setting, it worked for a couple weeks then started crashing again. 14 hours ago, John_M said: Don't do both. Re-enable the Global C States. See above. The kernel panic is not caused by the CPU failing to wake up from the C6 state. If that happens the sever just freezes, but the Power Supply Idle Control setting fixes that. As mentioned above (twice), the panic is likely related to docker networking. Yeah, I reverted the change of the C-states back to default. Only thing I have set now is the power supply idle control. So far, what has me "fixed" is I've move all of my docker containers that needed a custom network (static IP) over to a separate NIC (br2). I also disabled vlans in Unraid as well, since I wasn't using them anymore. No kernel panics or anything, yet anyway. It's been over a week now. Fingers are crossed! Quote Link to comment
dauntouch Posted May 31, 2021 Share Posted May 31, 2021 I "fixed" it by downgrading to 6.8.3 Quote Link to comment
rodan5150 Posted May 31, 2021 Author Share Posted May 31, 2021 10 hours ago, dauntouch said: I "fixed" it by downgrading to 6.8.3 I was close to doing this, but I figured I'd tough it out and give 6.9.x a shot. So far, the br2 network for my containers I want to have their own IP, has been working well. No call traces yet and certainly no kernel panics. Of course, it is barely over a week out since I made that change. If I can say this a month out, then I will consider it good to go. Quote Link to comment
dauntouch Posted June 8, 2021 Share Posted June 8, 2021 Is there an issue for this so we can track when its safe to upgrade to 6.9.x? Quote Link to comment
omartian Posted June 9, 2021 Share Posted June 9, 2021 Think i'm having the same issue. Going to try and revert to 6.8. syslog Quote Link to comment
omartian Posted July 2, 2021 Share Posted July 2, 2021 reverting helped and haven't crashed in last 2 weeks. will keep an eye out for a better upgrade. Quote Link to comment
omartian Posted August 20, 2021 Share Posted August 20, 2021 On 6/8/2021 at 12:33 PM, dauntouch said: Is there an issue for this so we can track when its safe to upgrade to 6.9.x? Would like to know this as well Quote Link to comment
pinion Posted December 20, 2021 Share Posted December 20, 2021 Hate to bring up a thread but this is the issue I am having. It's plagued me for awhile so I don't think downgrading will help. My kernel panic is just like what's in the picture. I spot checked the numbers on those codes and it's exactly the same. I'll mess around and see if I can figure out what's going on. Whenever this happens it brings down my entire network at home. Like the server start DDOSing it or something. And it's immediate. I get a text from my wife right away that the network is down. Quote Link to comment
JonathanM Posted December 20, 2021 Share Posted December 20, 2021 11 hours ago, pinion said: this is the issue I am having. Do you have any containers with custom IP addresses? Quote Link to comment
pinion Posted December 21, 2021 Share Posted December 21, 2021 11 hours ago, JonathanM said: Do you have any containers with custom IP addresses? Almost definitely but I'm not 100%. Tried to upgrade to 6.10 last night hoping that would just fix it. But the upgrade said it worked but the server wouldn't reboot saying to insert a device called UNRAID. So I put the bz(stuff) from before over on top of the usb and I was able to get back in. After that I didn't have time to mess with it and now tonight I've gotten another kernel panic. So I'm currently backing up the USB. Going to try and upgrade. And then I'll check if anything has a custom IP. By custom IP I assume you're talking about a container that I set the IP to something other than what unraid has. So if unraid = 192.168.0.5 and I setup a container with IP 192.168.0.6 then that's custom. Are there any other situations that would be considered a custom IP. Quote Link to comment
JonathanM Posted December 21, 2021 Share Posted December 21, 2021 37 minutes ago, pinion said: Tried to upgrade to 6.10 last night hoping that would just fix it. There's a second step to change from macvlan to ipvlan, I believe it's in the docker settings after you get the upgrade running. Quote Link to comment
pinion Posted December 21, 2021 Share Posted December 21, 2021 46 minutes ago, JonathanM said: There's a second step to change from macvlan to ipvlan, I believe it's in the docker settings after you get the upgrade running. You are correct that's where it's at. Need to toggle Advanced View but it's there. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.