JohnJay829 Posted February 20, 2020 Share Posted February 20, 2020 I have been random losing gui access to my tower on my network and i am unable to determine if the system is running normal. When i log into the system local i try running diagnostics in the terminal window but it just hangs there. i also try a ping to my router @192.168.1.1 and no response. I run the powerdown -r command which hangs at forcing shutdown. Any other commands i can use to pull data to see what is causing this issue Quote Link to comment
JohnJay829 Posted February 20, 2020 Author Share Posted February 20, 2020 7 hours ago, johnnie.black said: Try this. I started a syslog server on my windows server and sending to the flash(probably stop this soon). So far no issue with losing network. Anything i should look for in the syslogs? I see some kernel warning logs 2/20/2020 3:55 AM,Warning,192.168.1.153,RBP: ffff888329a43870 R08: 0000000000000003 R09: 0000000000019000 2/20/2020 6:48 AM,Notice,192.168.1.153,preclear_disk_4A454B3536304C4E[2 Zeroing: progress - 10% zeroed 2/20/2020 3:55 AM,Warning,192.168.1.153,---[ end trace f0b92a7f07269c1e ]--- 2/20/2020 3:55 AM,Warning,192.168.1.153,secondary_startup_64+0xa4/0xb0 2/20/2020 3:55 AM,Warning,192.168.1.153,start_secondary+0x197/0x1b2 2/20/2020 3:55 AM,Warning,192.168.1.153,cpu_startup_entry+0x6a/0x6c 2/20/2020 3:55 AM,Warning,192.168.1.153,do_idle+0x17e/0x1fc 2/20/2020 3:55 AM,Warning,192.168.1.153,? cpuidle_enter_state+0xbf/0x141 2/20/2020 3:55 AM,Warning,192.168.1.153,R13: ffffffff81e5b1e0 R14: 0000000000000000 R15: ffffffff81e5b378 2/20/2020 3:55 AM,Warning,192.168.1.153,R10: 0000000000002ff4 R11: 071c71c71c71c71c R12: 0000000000000004 2/20/2020 3:55 AM,Warning,192.168.1.153,RBP: 0000143d725bd2cf R08: 0000143d725bd2cf R09: 0000144075f51243 2/20/2020 3:55 AM,Warning,192.168.1.153,RDX: 0000000000000000 RSI: 000000002bbee902 RDI: 0000000000000000 2/20/2020 3:55 AM,Warning,192.168.1.153,RAX: ffff888329a5fac0 RBX: ffff888329a6aa00 RCX: 000000000000001f 2/20/2020 3:55 AM,Warning,192.168.1.153,RSP: 0018:ffffc900019a3e98 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffdb 2/20/2020 3:55 AM,Warning,192.168.1.153,Code: ee f2 b7 ff 45 84 f6 74 1d 9c 58 66 66 90 66 90 0f ba e0 09 73 09 0f 0b fa 66 66 90 66 66 90 31 ff e8 a8 8f bb ff fb 66 66 90 <66> 66 90 48 2b 2c 24 b8 ff ff ff 7f 48 b9 ff ff ff ff f3 01 00 00 2/20/2020 3:55 AM,Warning,192.168.1.153,RIP: 0010:cpuidle_enter_state+0xe5/0x141 2/20/2020 3:55 AM,Warning,192.168.1.153,</IRQ> 2/20/2020 3:55 AM,Warning,192.168.1.153,common_interrupt+0xf/0xf 2/20/2020 3:55 AM,Warning,192.168.1.153,do_IRQ+0xb2/0xd0 2/20/2020 3:55 AM,Warning,192.168.1.153,irq_exit+0x5e/0x9d 2/20/2020 3:55 AM,Warning,192.168.1.153,__do_softirq+0xc9/0x1d7 2/20/2020 3:55 AM,Warning,192.168.1.153,net_rx_action+0x107/0x26c 2/20/2020 3:55 AM,Warning,192.168.1.153,bnx2_poll_msix+0x2e/0xb1 [bnx2] 2/20/2020 3:55 AM,Warning,192.168.1.153,bnx2_poll_work+0xf8c/0x105f [bnx2] 2/20/2020 3:55 AM,Warning,192.168.1.153,napi_gro_receive+0x44/0x7b 2/20/2020 3:55 AM,Warning,192.168.1.153,netif_receive_skb_internal+0x9f/0xba 2/20/2020 3:55 AM,Warning,192.168.1.153,__netif_receive_skb_one_core+0x35/0x6f 2/20/2020 3:55 AM,Warning,192.168.1.153,? udp_gro_receive+0x4b/0x136 2/20/2020 3:55 AM,Warning,192.168.1.153,__netif_receive_skb_core+0x464/0x76e 2/20/2020 3:55 AM,Warning,192.168.1.153,? br_pass_frame_up+0x14a/0x14a 2/20/2020 3:55 AM,Warning,192.168.1.153,br_handle_frame+0x27e/0x2bd 2/20/2020 3:55 AM,Warning,192.168.1.153,nf_hook_slow+0x3a/0x90 2/20/2020 3:55 AM,Warning,192.168.1.153,? br_nf_forward_ip+0x362/0x362 2/20/2020 3:55 AM,Warning,192.168.1.153,br_nf_pre_routing+0x31c/0x343 2/20/2020 3:55 AM,Warning,192.168.1.153,? br_handle_local_finish+0xe/0xe 2/20/2020 3:55 AM,Warning,192.168.1.153,? nf_nat_ipv4_in+0x1e/0x62 [nf_nat_ipv4] 2/20/2020 3:55 AM,Warning,192.168.1.153,? br_handle_local_finish+0xe/0xe 2/20/2020 3:55 AM,Warning,192.168.1.153,? br_pass_frame_up+0x14a/0x14a 2/20/2020 3:55 AM,Warning,192.168.1.153,br_nf_pre_routing_finish+0x24a/0x271 2/20/2020 3:55 AM,Warning,192.168.1.153,? br_pass_frame_up+0x14a/0x14a 2/20/2020 3:55 AM,Warning,192.168.1.153,br_nf_hook_thresh+0xa3/0xc3 2/20/2020 3:55 AM,Warning,192.168.1.153,? br_pass_frame_up+0x14a/0x14a 2/20/2020 3:55 AM,Warning,192.168.1.153,br_handle_frame_finish+0x342/0x383 2/20/2020 3:55 AM,Warning,192.168.1.153,? br_port_flags_change+0x29/0x29 2/20/2020 3:55 AM,Warning,192.168.1.153,br_pass_frame_up+0x128/0x14a 2/20/2020 3:55 AM,Warning,192.168.1.153,netif_receive_skb_internal+0x9f/0xba 2/20/2020 3:55 AM,Warning,192.168.1.153,__netif_receive_skb_one_core+0x53/0x6f 2/20/2020 3:55 AM,Warning,192.168.1.153,? ip_rcv_finish_core.isra.0+0x2e1/0x2e1 2/20/2020 3:55 AM,Warning,192.168.1.153,ip_rcv+0x8e/0xbe 2/20/2020 3:55 AM,Warning,192.168.1.153,nf_hook_slow+0x3a/0x90 2/20/2020 3:55 AM,Warning,192.168.1.153,ip_sabotage_in+0x38/0x3e 2/20/2020 3:55 AM,Warning,192.168.1.153,? ip_sublist_rcv_finish+0x54/0x54 2/20/2020 3:55 AM,Warning,192.168.1.153,ip_local_deliver+0xad/0xdc 2/20/2020 3:55 AM,Warning,192.168.1.153,nf_hook_slow+0x3a/0x90 2/20/2020 3:55 AM,Warning,192.168.1.153,nf_nat_inet_fn+0x9f/0x1b9 [nf_nat] 2/20/2020 3:55 AM,Warning,192.168.1.153,nf_nat_alloc_null_binding+0x71/0x88 [nf_nat] 2/20/2020 3:55 AM,Warning,192.168.1.153,? br_fdb_offloaded_set+0x3f/0x3f 2/20/2020 3:55 AM,Warning,192.168.1.153,? fib_validate_source+0xab/0xd5 2/20/2020 3:55 AM,Warning,192.168.1.153,<IRQ> 2/20/2020 3:55 AM,Warning,192.168.1.153,Call Trace: 2/20/2020 3:55 AM,Warning,192.168.1.153,CR2: 000055c0a549e1e8 CR3: 0000000001e0a003 CR4: 00000000000206e0 2/20/2020 3:55 AM,Warning,192.168.1.153,CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 2/20/2020 3:55 AM,Warning,192.168.1.153,FS: 0000000000000000(0000) GS:ffff888329a40000(0000) knlGS:0000000000000000 2/20/2020 3:55 AM,Warning,192.168.1.153,R13: 0000000000000000 R14: ffff88815b855b80 R15: ffff888329a43884 2/20/2020 3:55 AM,Warning,192.168.1.153,R10: 0000000000000000 R11: 0000000000000044 R12: 0000000000000000 2/20/2020 3:55 AM,Warning,192.168.1.153,RBP: ffff888329a43870 R08: 0000000000000003 R09: 0000000000019000 2/20/2020 3:55 AM,Warning,192.168.1.153,RDX: 0000000000000000 RSI: 00000000000000f6 RDI: 0000000000000300 2/20/2020 3:55 AM,Warning,192.168.1.153,RAX: 0000000000000024 RBX: ffff888329a43938 RCX: 0000000000000000 2/20/2020 3:55 AM,Warning,192.168.1.153,RSP: 0018:ffff888329a43798 EFLAGS: 00010246 2/20/2020 3:55 AM,Warning,192.168.1.153,Code: 0d 8b 87 80 00 00 00 25 80 00 00 00 eb 0c 41 8b 86 80 00 00 00 25 00 01 00 00 85 c0 74 13 48 c7 c7 cc a1 09 a0 e8 bc 56 ff e0 <0f> 0b e9 49 01 00 00 4c 8d 6c 24 30 49 8d 76 58 4c 89 ef e8 73 bc 2/20/2020 3:55 AM,Warning,192.168.1.153,RIP: 0010:nf_nat_setup_info+0x94/0x666 [nf_nat] 2/20/2020 3:55 AM,Warning,192.168.1.153,Hardware name: Dell Inc. PowerEdge R710/00NH4P, BIOS 6.3.0 07/24/2012 2/20/2020 3:55 AM,Warning,192.168.1.153,CPU: 18 PID: 0 Comm: swapper/18 Tainted: G W I 4.19.98-Unraid #1 2/20/2020 3:55 AM,Warning,192.168.1.153,Modules linked in: veth rpcsec_gss_krb5 auth_rpcgss oid_registry nfsv3 nfs lockd grace sunrpc arc4 ecb md4 xt_CHECKSUM sha512_ssse3 ipt_REJECT sha512_generic cmac cifs ip6table_mangle ccm ip6table_nat nf_nat_ipv6 iptable_mangle ip6table_filter ip6_tables vhost_net tun vhost tap macvlan xt_nat ipt_MASQUERADE iptable_filter iptable_nat nf_nat_ipv4 nf_nat ip_tables xfs md_mod bonding bnx2 sr_mod cdrom intel_powerclamp coretemp kvm_intel kvm ipmi_ssif i2c_core crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd cryptd i7core_edac wmi glue_helper megaraid_sas intel_cstate intel_uncore acpi_power_meter ata_piix button pcc_cpufreq ipmi_si acpi_cpufreq [last unloaded: bnx2] 2/20/2020 3:55 AM,Warning,192.168.1.153,WARNING: CPU: 18 PID: 0 at net/netfilter/nf_nat_core.c:420 nf_nat_setup_info+0x94/0x666 [nf_nat] 2/20/2020 3:55 AM,Warning,192.168.1.153,------------[ cut here ]------------ 2/20/2020 3:39 AM,Notice,192.168.1.153,exit status 3 from user root /usr/local/sbin/mover &> /dev/null 2/20/2020 3:34 AM,Info,192.168.1.153,[origin software=""rsyslogd"" swVersion=""8.1908.0"" x-pid=""25358"" x-info=""https://www.rsyslog.com""] start 2/20/2020 3:34 AM,Info,192.168.1.153,ool www[17422]: /usr/local/emhttp/plugins/dynamix/scripts/rsyslog_config 2/20/2020 3:24 AM,Info,192.168.1.153,[origin software=""rsyslogd"" swVersion=""8.1908.0"" x-pid=""9576"" x-info=""https://www.rsyslog.com""] start Quote Link to comment
JorgeB Posted February 20, 2020 Share Posted February 20, 2020 Post the complete syslog when you lose it again, those call traces look network related. 1 Quote Link to comment
JohnJay829 Posted February 20, 2020 Author Share Posted February 20, 2020 What can i set up to get an alert when it is down. Currently i am able to tell when my plex stops working but i am not always watching a show Quote Link to comment
JorgeB Posted February 20, 2020 Share Posted February 20, 2020 13 hours ago, JohnJay829 said: I have been random losing gui access to my tower on my network When this happens again post syslog. Quote Link to comment
JohnJay829 Posted February 22, 2020 Author Share Posted February 22, 2020 (edited) On 2/20/2020 at 11:03 AM, johnnie.black said: When this happens again post syslog. Lost system again. Tried a powerdown -r just hung now no access at all 2020-02-22.txt Edited February 23, 2020 by JohnJay829 Correction Quote Link to comment
JohnJay829 Posted February 23, 2020 Author Share Posted February 23, 2020 Actually at home now in front of the system. Tried just a powerdown. Showing forcing shutdown now but doesn't actually shutdown unless i power down the sever manually Quote Link to comment
JorgeB Posted February 23, 2020 Share Posted February 23, 2020 Lots of call traces, first ones seem to be xfs related, so start by running a filesystem check on all xfs filesystems: xfs_repair -v /dev/mdX Same for cache if xfs, also a good idea to run memtest. Quote Link to comment
JohnJay829 Posted February 23, 2020 Author Share Posted February 23, 2020 5 hours ago, johnnie.black said: Lots of call traces, first ones seem to be xfs related, so start by running a filesystem check on all xfs filesystems: xfs_repair -v /dev/mdX Same for cache if xfs, also a good idea to run memtest. All of these are ran with array stopped? Quote Link to comment
JonathanM Posted February 24, 2020 Share Posted February 24, 2020 9 hours ago, JohnJay829 said: All of these are ran with array stopped? Started in maintenance mode. Quote Link to comment
JohnJay829 Posted February 29, 2020 Author Share Posted February 29, 2020 Well lost system today had been fine but i was letting sonarr run to a directory on my share. After one file was moved the system dropped 2020-02-29.txt Quote Link to comment
JohnJay829 Posted March 15, 2020 Author Share Posted March 15, 2020 Lost system again had to hard reset: syslog reading off windows 3/14/2020 9:01 PM,Notice,192.168.1.153,Downloading certs 3/14/2020 8:07 PM,Alert,192.168.1.153,php-fpm: Corrupted page table at address 5a4ae6 3/14/2020 8:07 PM,Warning,192.168.1.153,RIP: 0033:0x5a4b10 3/14/2020 8:07 PM,Warning,192.168.1.153,page_fault+0x1e/0x30 3/14/2020 8:07 PM,Warning,192.168.1.153,? page_fault+0x8/0x30 3/14/2020 8:07 PM,Warning,192.168.1.153,__do_page_fault+0x267/0x3ff 3/14/2020 8:07 PM,Warning,192.168.1.153,handle_mm_fault+0x189/0x1e3 3/14/2020 8:07 PM,Warning,192.168.1.153,__handle_mm_fault+0x6ba/0x11b7 3/14/2020 8:07 PM,Warning,192.168.1.153,_vm_normal_page+0x5b/0xc6 3/14/2020 8:07 PM,Warning,192.168.1.153,print_bad_pte+0x222/0x23f 3/14/2020 8:07 PM,Warning,192.168.1.153,dump_stack+0x67/0x83 3/14/2020 8:07 PM,Warning,192.168.1.153,Call Trace: 3/14/2020 8:07 PM,Warning,192.168.1.153,Hardware name: Dell Inc. PowerEdge R710/00NH4P, BIOS 6.3.0 07/24/2012 3/14/2020 8:07 PM,Warning,192.168.1.153,CPU: 15 PID: 20050 Comm: php-fpm Tainted: G W I 4.19.98-Unraid #1 3/14/2020 8:07 PM,Alert,192.168.1.153,file:php-fpm fault:shmem_fault mmap:shmem_mmap readpage: (null) 3/14/2020 8:07 PM,Alert,192.168.1.153,addr:00000000c4ad0535 vm_flags:00000875 anon_vma: (null) mapping:00000000f2882fbf index:1a4 3/14/2020 8:07 PM,Alert,192.168.1.153,BUG: Bad page map in process php-fpm pte:400777d9792403d pmd:55ade067 3/14/2020 6:26 PM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 5:22 PM,Notice,192.168.1.153,plugin: running: anonymous 3/14/2020 5:22 PM,Notice,192.168.1.153,plugin: running: /boot/config/plugins/community.applications/community.applications-2020.03.14-x86_64-1.txz 3/14/2020 5:22 PM,Notice,192.168.1.153,plugin: checking: /boot/config/plugins/community.applications/community.applications-2020.03.14-x86_64-1.txz - MD5 3/14/2020 5:22 PM,Notice,192.168.1.153,plugin: creating: /boot/config/plugins/community.applications/community.applications-2020.03.14-x86_64-1.txz - downloading from URL https://raw.githubusercontent.com/Squidly271/community.applications/master/archive/community.applications-2020.03.14-x86_64-1.txz 3/14/2020 5:22 PM,Notice,192.168.1.153,plugin: running: anonymous 3/14/2020 5:22 PM,Notice,192.168.1.153,plugin: running: anonymous 3/14/2020 5:22 PM,Info,192.168.1.153,cmd: /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin update community.applications.plg 3/14/2020 4:30 PM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 4:20 PM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 4:12 PM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 4:05 PM,Notice,192.168.1.153,ROOT LOGIN on '/dev/pts/0' 3/14/2020 2:37 PM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 1:51 PM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 1:50 PM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 12:51 PM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 11:57 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 10:45 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 10:34 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 9:48 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 9:44 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 8:52 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 8:50 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 8:42 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 7:42 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 7:21 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 6:56 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 6:34 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 6:24 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 5:58 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 4:47 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 4:45 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 4:19 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 4:12 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 3:40 AM,Notice,192.168.1.153,exit status 1 from user root /usr/local/sbin/mover &> /dev/null 3/14/2020 3:05 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 2:59 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 2:30 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 2:18 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci 3/14/2020 12:47 AM,Info,192.168.1.153,usb 1-1.1: reset low-speed USB device number 4 using ehci-pci Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.