Fail to connect to server after updates

[Mlatx]

Hi All,

 

This is the second time in the last week this has happened to me. An automatic update happens just after midnight, and I cannot connect to my server in the morning. I've had to do an unclean restart.

 

I have the log server running, and I here is a section of the log the last time it did this. It seems like a network connection issue. Can anyone help with troubleshooting?

 

If I need the whole log, let me know.

 

syslog.txt

[JorgeB]

What was updated? Unraid itself is not updated automatically.

[Mlatx]

Dockers. Last night, the following were updated: Home-Assistant-Core, Prometheus, Postgresql14, and Swag.

[Mlatx]

Here is today's log snipit. Looking at my emails, the system went unresponsive after 3am. It doesn't come backup until after I rebooted. I don't see any errors in the log.

syslog2.txt

[JorgeB]

7 minutes ago, Mlatx said:

I don't see any errors in the log.

Same.

[trurl]

11 minutes ago, Mlatx said:

system went unresponsive after 3am

Anything scheduled for that time?

[Mlatx]

Nothing was scheduled last night. I don't recall if anything was schedule the last failure. The only change I made recently was to change the spin down delay from never to 4 hours.

 

On the last unresponsive restart from the first attached log, I see this warning several times.

Sep 27 17:27:25 server kernel: WARNING: CPU: 6 PID: 7014 at fs/btrfs/delayed-inode.c:1304 btrfs_assert_delayed_root_empty+0x16/0x19
Sep 27 17:27:25 server kernel: Modules linked in: xt_mark xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle vhost_net tun vhost vhost_iotlb tap macvlan nvidia_modeset(PO) nvidia_uvm(PO) xt_nat xt_tcpudp veth xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter xfs nfsd auth_rpcgss oid_registry lockd grace sunrpc md_mod nvidia(PO) drm backlight nct6775 hwmon_vid nct6683 apex(O) gasket(O) efivarfs ip6table_filter ip6_tables iptable_filter ip_tables x_tables amd64_edac edac_mce_amd kvm_amd wmi_bmof kvm btusb btrtl btbcm crct10dif_pclmul crc32_pclmul btintel crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd mpt3sas bluetooth igb i2c_piix4 i2c_algo_bit rapl raid_class ccp k10temp igc scsi_transport_sas ecdh_generic i2c_core ahci ecc input_leds led_class libahci tpm_crb acpi_cpufreq tpm_tis tpm_tis_core tpm wmi button

 

This happened during the day not overnight.

[trurl]

 

[Mlatx]

Interesting. It has never happened to me before. Although I recently made changes to my hardware within the last 3 months or so. Maybe that caused the issues. I'll try the suggestions.

[JorgeB]

Switch to ipvlan (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right)).

[Mlatx]

Since that was the easiest option, that is what I did. Thanks for pointing it out.

[Mlatx]

I wanted to comment here in case someone has a similar issue. Going to ipvlan with cusom IP's gave me another kind of issue. My server wasn't crashing, but my dockers were randomly going offline. Using ipvlan without custom IP's solved the problem.

 

If I need custom IP's in the future, I'll put dockers on their own vlan.