October 16, 20178 yr the server is frozen. the VM screens are static no activity NIC lights blinking HD lights black just before crashing I captured the LOG bellow. any ideas ? what is this IRQ flags mismatch ? Oct 16 06:56:17 Tower upsmon[15220]: Poll UPS [ups@localhost] failed - Data staleOct 16 06:56:17 Tower upsd[15217]: UPS [ups] data is no longer staleOct 16 06:56:22 Tower upsmon[15220]: Communications with UPS ups@localhost establishedOct 16 07:00:01 Tower speedtest: Internet bandwidth test startedOct 16 07:00:28 Tower speedtest: Host: Triple C (Petah Tikva) [22.27 km]Oct 16 07:00:28 Tower speedtest: Ping (Lowest): 158.76 ms | Download (Max): 60.89 Mbit/s | Upload (Max): 1.76 Mbit/sOct 16 07:00:28 Tower speedtest: Internet bandwidth test completedOct 16 07:11:39 Tower kernel: vgaarb: device changed decodes: PCI:0000:03:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+memOct 16 07:11:39 Tower kernel: br0: port 5(vnet3) entered blocking stateOct 16 07:11:39 Tower kernel: br0: port 5(vnet3) entered disabled stateOct 16 07:11:39 Tower kernel: device vnet3 entered promiscuous modeOct 16 07:11:39 Tower kernel: br0: port 5(vnet3) entered blocking stateOct 16 07:11:39 Tower kernel: br0: port 5(vnet3) entered forwarding stateOct 16 07:11:57 Tower kernel: vfio_ecap_init: 0000:03:00.0 hiding ecap 0x19@0x900Oct 16 07:11:57 Tower kernel: vfio-pci 0000:09:00.0: enabling device (0400 -> 0402)Oct 16 07:11:58 Tower kernel: genirq: Flags mismatch irq 18. 00000080 (vfio-intx(0000:09:00.0)) vs. 00000000 (vfio-intx(0000:0a:02.0))Oct 16 07:11:59 Tower kernel: br0: port 5(vnet3) entered disabled stateOct 16 07:11:59 Tower kernel: device vnet3 left promiscuous modeOct 16 07:11:59 Tower kernel: br0: port 5(vnet3) entered disabled stateOct 16 07:12:02 Tower kernel: vgaarb: device changed decodes: PCI:0000:03:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+memOct 16 07:13:26 Tower kernel: vgaarb: device changed decodes: PCI:0000:03:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+memOct 16 07:13:26 Tower kernel: br0: port 5(vnet3) entered blocking stateOct 16 07:13:26 Tower kernel: br0: port 5(vnet3) entered disabled stateOct 16 07:13:26 Tower kernel: device vnet3 entered promiscuous modeOct 16 07:13:26 Tower kernel: br0: port 5(vnet3) entered blocking stateOct 16 07:13:26 Tower kernel: br0: port 5(vnet3) entered forwarding stateOct 16 07:13:39 Tower kernel: vfio_ecap_init: 0000:03:00.0 hiding ecap 0x19@0x900Oct 16 07:13:39 Tower kernel: vfio-pci 0000:09:00.0: enabling device (0400 -> 0402)Oct 16 07:13:40 Tower kernel: genirq: Flags mismatch irq 18. 00000080 (vfio-intx(0000:09:00.0)) vs. 00000000 (vfio-intx(0000:0a:02.0))Oct 16 07:13:41 Tower kernel: br0: port 5(vnet3) entered disabled stateOct 16 07:13:41 Tower kernel: device vnet3 left promiscuous modeOct 16 07:13:41 Tower kernel: br0: port 5(vnet3) entered disabled stateOct 16 07:13:44 Tower kernel: vgaarb: device changed decodes: PCI:0000:03:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+memOct 16 07:16:54 Tower sshd[25917]: Accepted none for root from 10.0.0.148 port 64306 ssh2Oct 16 07:18:17 Tower kernel: vgaarb: device changed decodes: PCI:0000:03:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+memOct 16 07:18:17 Tower kernel: br0: port 5(vnet3) entered blocking stateOct 16 07:18:17 Tower kernel: br0: port 5(vnet3) entered disabled stateOct 16 07:18:17 Tower kernel: device vnet3 entered promiscuous modeOct 16 07:18:17 Tower kernel: br0: port 5(vnet3) entered blocking stateOct 16 07:18:17 Tower kernel: br0: port 5(vnet3) entered forwarding stateOct 16 07:18:31 Tower kernel: vfio_ecap_init: 0000:03:00.0 hiding ecap 0x19@0x900Oct 16 07:18:31 Tower kernel: vfio-pci 0000:09:00.0: enabling device (0400 -> 0402)Oct 16 07:18:32 Tower kernel: genirq: Flags mismatch irq 18. 00000080 (vfio-intx(0000:09:00.0)) vs. 00000000 (vfio-intx(0000:0a:02.0))Oct 16 07:18:33 Tower kernel: br0: port 5(vnet3) entered disabled stateOct 16 07:18:33 Tower kernel: device vnet3 left promiscuous modeOct 16 07:18:33 Tower kernel: br0: port 5(vnet3) entered disabled state
December 6, 20178 yr Author I am having crashes every night. Please help to understand ! attached is the diagnostics "postmortem". not sure what it can show. bellow is the syslog last "dying messages". I had a syslog window open on remote pc... Dec 5 19:00:01 Tower speedtest: Internet bandwidth test started Dec 5 19:00:32 Tower speedtest: Host: Triple C (Petah Tikva) [22.27 km] Dec 5 19:00:32 Tower speedtest: Ping (Lowest): 949.874 ms | Download (Max): 94.37 Mbit/s | Upload (Max): 1.81 Mbit/s Dec 5 19:00:32 Tower speedtest: Internet bandwidth test completed Dec 5 20:00:01 Tower speedtest: Internet bandwidth test started Dec 5 20:00:12 Tower speedtest: Host: Dec 5 20:00:12 Tower speedtest: Dec 5 20:00:12 Tower speedtest: Internet bandwidth test completed Dec 5 21:00:02 Tower speedtest: Internet bandwidth test started Dec 5 21:00:37 Tower speedtest: Host: Triple C (Petah Tikva) [22.27 km] Dec 5 21:00:37 Tower speedtest: Ping (Lowest): 106.573 ms | Download (Max): 76.06 Mbit/s | Upload (Max): 1.47 Mbit/s Dec 5 21:00:37 Tower speedtest: Internet bandwidth test completed Dec 5 22:00:01 Tower speedtest: Internet bandwidth test started Dec 5 22:00:31 Tower speedtest: Host: Triple C (Petah Tikva) [22.27 km] Dec 5 22:00:31 Tower speedtest: Ping (Lowest): 133.581 ms | Download (Max): 96.27 Mbit/s | Upload (Max): 1.63 Mbit/s Dec 5 22:00:31 Tower speedtest: Internet bandwidth test completed Dec 5 23:00:01 Tower speedtest: Internet bandwidth test started Dec 5 23:00:32 Tower speedtest: Host: Triple C (Petah Tikva) [22.27 km] Dec 5 23:00:32 Tower speedtest: Ping (Lowest): 110.671 ms | Download (Max): 97.10 Mbit/s | Upload (Max): 1.72 Mbit/s Dec 5 23:00:32 Tower speedtest: Internet bandwidth test completed Dec 5 23:35:16 Tower kernel: vgaarb: device changed decodes: PCI:0000:04:00.0,olddecodes=io+mem,decodes=io+mem:owns=none Dec 5 23:35:16 Tower kernel: br0: port 5(vnet3) entered blocking state Dec 5 23:35:16 Tower kernel: br0: port 5(vnet3) entered disabled state Dec 5 23:35:16 Tower kernel: device vnet3 entered promiscuous mode Dec 5 23:35:16 Tower kernel: br0: port 5(vnet3) entered blocking state Dec 5 23:35:16 Tower kernel: br0: port 5(vnet3) entered forwarding state Dec 5 23:35:52 Tower kernel: vfio-pci 0000:04:00.0: enabling device (0140 -> 0143) Dec 5 23:35:52 Tower kernel: vfio_ecap_init: 0000:04:00.0 hiding ecap 0x19@0x270 Dec 5 23:35:52 Tower kernel: vfio_ecap_init: 0000:04:00.0 hiding ecap 0x1b@0x2d0 Dec 5 23:35:52 Tower kernel: vfio_ecap_init: 0000:04:00.0 hiding ecap 0x1e@0x370 Dec 5 23:36:07 Tower kernel: kvm: zapping shadow pages for mmio generation wraparound Dec 5 23:36:07 Tower kernel: kvm: zapping shadow pages for mmio generation wraparound Dec 6 00:00:01 Tower speedtest: Internet bandwidth test started Dec 6 00:00:27 Tower speedtest: Host: Triple C (Petah Tikva) [22.27 km] Dec 6 00:00:27 Tower speedtest: Ping (Lowest): 142.165 ms | Download (Max): 88.70 Mbit/s | Upload (Max): 1.90 Mbit/s Dec 6 00:00:27 Tower speedtest: Internet bandwidth test completed Dec 6 00:37:58 Tower afpd[19899]: afp_alarm: child timed out, entering disconnected state Dec 6 01:00:02 Tower speedtest: Internet bandwidth test started Dec 6 01:00:03 Tower root: /mnt/cache: 258.7 GiB (277808664576 bytes) trimmed Dec 6 01:00:35 Tower speedtest: Host: Triple C (Petah Tikva) [22.27 km] Dec 6 01:00:35 Tower speedtest: Ping (Lowest): 123.046 ms | Download (Max): 88.93 Mbit/s | Upload (Max): 1.72 Mbit/s Dec 6 01:00:35 Tower speedtest: Internet bandwidth test completed Dec 6 02:00:01 Tower speedtest: Internet bandwidth test started Dec 6 02:00:39 Tower speedtest: Host: Triple C (Petah Tikva) [22.27 km] Dec 6 02:00:39 Tower speedtest: Ping (Lowest): 171.166 ms | Download (Max): 98.26 Mbit/s | Upload (Max): 1.43 Mbit/s Dec 6 02:00:39 Tower speedtest: Internet bandwidth test completed Dec 6 02:07:02 Tower afpd[31875]: read: Connection reset by peer Dec 6 02:07:02 Tower afpd[20176]: read: Connection reset by peer Dec 6 02:07:02 Tower afpd[31875]: dsi_stream_read: len:-1, Connection reset by peer Dec 6 02:07:02 Tower afpd[20176]: dsi_stream_read: len:-1, Connection reset by peer Dec 6 03:00:01 Tower speedtest: Internet bandwidth test started Dec 6 03:00:01 Tower CA Backup/Restore: ####################################### Dec 6 03:00:01 Tower CA Backup/Restore: Community Applications appData Backup Dec 6 03:00:01 Tower CA Backup/Restore: Applications will be unavailable during Dec 6 03:00:01 Tower CA Backup/Restore: this process. They will automatically Dec 6 03:00:01 Tower CA Backup/Restore: be restarted upon completion. Dec 6 03:00:01 Tower CA Backup/Restore: ####################################### Dec 6 03:00:01 Tower CA Backup/Restore: Stopping binhex-delugevpn Dec 6 03:00:02 Tower kernel: docker0: port 1(vethc368c87) entered disabled state Dec 6 03:00:02 Tower kernel: vethe24010d: renamed from eth0 Dec 6 03:00:03 Tower kernel: docker0: port 1(vethc368c87) entered disabled state Dec 6 03:00:03 Tower kernel: device vethc368c87 left promiscuous mode Dec 6 03:00:03 Tower kernel: docker0: port 1(vethc368c87) entered disabled state Dec 6 03:00:03 Tower CA Backup/Restore: docker stop -t 60 binhex-delugevpn Dec 6 03:00:03 Tower CA Backup/Restore: Stopping CrashPlan Dec 6 03:00:03 Tower CA Backup/Restore: docker stop -t 60 CrashPlan Dec 6 03:00:03 Tower CA Backup/Restore: Stopping EmbyServer Dec 6 03:00:07 Tower move: skipping file in-use: ./Downloads/.AppleDB/__db.001 Dec 6 03:00:08 Tower move: skipping file in-use: ./Downloads/.AppleDB/__db.002 Dec 6 03:00:08 Tower move: skipping file in-use: ./Downloads/.AppleDB/__db.003 Dec 6 03:00:08 Tower move: skipping file in-use: ./Downloads/.AppleDB/__db.004 Dec 6 03:00:08 Tower move: skipping file in-use: ./Downloads/.AppleDB/__db.005 Dec 6 03:00:08 Tower move: skipping file in-use: ./Downloads/.AppleDB/__db.006 Dec 6 03:00:08 Tower move: rmdir: /mnt/cache/./Downloads/.AppleDB Directory not empty Dec 6 03:00:08 Tower move: rmdir: /mnt/cache/./Downloads Directory not empty Dec 6 03:00:08 Tower move: skipping file in-use: ./domains/talVM/vdisk1.img Dec 6 03:00:08 Tower move: rmdir: /mnt/cache/./domains/talVM Directory not empty Dec 6 03:00:08 Tower move: rmdir: /mnt/cache/./domains Directory not empty Dec 6 03:00:10 Tower CA Backup/Restore: docker stop -t 60 EmbyServer Dec 6 03:00:10 Tower CA Backup/Restore: Stopping PrinterCUPS Dec 6 03:00:10 Tower avahi-daemon[8708]: Files changed, reloading. Dec 6 03:00:10 Tower avahi-daemon[8708]: Service group file /services/AirPrint-HP_Deskjet_F4500_series.service vanished, removing services. Dec 6 03:00:10 Tower kernel: docker0: port 2(veth0dbb9f6) entered disabled state Dec 6 03:00:10 Tower kernel: vethd7554e3: renamed from eth0 Dec 6 03:00:10 Tower kernel: docker0: port 2(veth0dbb9f6) entered disabled state Dec 6 03:00:10 Tower kernel: device veth0dbb9f6 left promiscuous mode Dec 6 03:00:10 Tower kernel: docker0: port 2(veth0dbb9f6) entered disabled state Dec 6 03:00:10 Tower CA Backup/Restore: docker stop -t 60 PrinterCUPS Dec 6 03:00:10 Tower CA Backup/Restore: Backing up USB Flash drive config folder to Dec 6 03:00:10 Tower move: skipping file in-use: ./media/.AppleDB/__db.001 Dec 6 03:00:10 Tower move: skipping file in-use: ./media/.AppleDB/__db.002 Dec 6 03:00:10 Tower move: skipping file in-use: ./media/.AppleDB/__db.003 Dec 6 03:00:10 Tower move: skipping file in-use: ./media/.AppleDB/__db.004 Dec 6 03:00:10 Tower move: skipping file in-use: ./media/.AppleDB/__db.005 Dec 6 03:00:10 Tower move: skipping file in-use: ./media/.AppleDB/__db.006 Dec 6 03:00:11 Tower CA Backup/Restore: Using command: /usr/bin/rsync -avXHq --delete --log-file="/var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log" /boot/ "/mnt/disks/WDC_WD10EAVS-00D7B0_WD-WCAU40879013/USB FLASH backup/" > /dev/null 2>&1 Dec 6 03:00:11 Tower move: skipping file in-use: ./media/.AppleDB/log.0000001278 Dec 6 03:00:11 Tower move: rmdir: /mnt/cache/./media/.AppleDB Directory not empty Dec 6 03:00:11 Tower CA Backup/Restore: Backing Up appData from /mnt/user/appdata/ to /mnt/disks/WDC_WD10EAVS-00D7B0_WD-WCAU40879013/CommunityApplicationsAppdataBackup/[email protected] Dec 6 03:00:11 Tower CA Backup/Restore: Using command: cd '/mnt/user/appdata/' && /usr/bin/tar -cvaf '/mnt/disks/WDC_WD10EAVS-00D7B0_WD-WCAU40879013/CommunityApplicationsAppdataBackup/[email protected]/CA_backup.tar' --exclude 'docker.img' * >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/backupInProgress Dec 6 03:00:13 Tower ntpd[2282]: Deleting interface #3 docker0, 172.17.0.1#123, interface stats: received=0, sent=0, dropped=0, active_time=76771 secs Dec 6 03:00:16 Tower kernel: BTRFS info (device sde1): The free space cache file (379060224000) is invalid. skip it Dec 6 03:00:16 Tower kernel: Dec 6 03:00:17 Tower move: rmdir: /mnt/cache/./media Directory not empty Dec 6 03:00:34 Tower speedtest: Host: Triple C (Petah Tikva) [22.27 km] Dec 6 03:00:34 Tower speedtest: Ping (Lowest): 126.882 ms | Download (Max): 98.09 Mbit/s | Upload (Max): 2.08 Mbit/s Dec 6 03:00:34 Tower speedtest: Internet bandwidth test completed Dec 6 03:05:14 Tower afpd[19899]: afp_alarm: child timed out, entering disconnected state Dec 6 03:05:25 Tower afpd[20176]: afp_alarm: child timed out, entering disconnected state Dec 6 03:05:41 Tower afpd[31875]: afp_alarm: child timed out, entering disconnected state tower-diagnostics-20171206-0456.zip
December 6, 20178 yr I'm not a log interpreting guru, so I can't help you there, however, if you have Fix Common Problems installed, you can put it in "Troubleshooting Mode" (button on the far right) and it will save your complete logs to your flash drive so you'll have a complete trail after a crash.
December 8, 20178 yr Author I am having crashes every night. (didnt post the the 7th) please please help. attached is the Fix Common Problems log bellow is the syslog window I left open, that shows even more. yesterday was the first time I have successfully installed and connected the UPS. So its strange that there are related errors. The system crashed for the last few days without the UPS. I have changed the MOVER schedule from 3am to 12pm. SO it shouldn't be related. next idea is to change the backup schedule. any other ideas ? Dec 8 00:15:34 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-06 12.31.20.pngDec 8 00:15:34 Tower root: .d..t...... media/Photos/2017-12-December/Dec 8 00:15:34 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-06 19.38.25.jpgDec 8 00:15:35 Tower root: .d..t...... media/Photos/2017-12-December/Dec 8 00:15:35 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-06 19.38.48.jpgDec 8 00:15:35 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-06 19.39.17.jpgDec 8 00:15:35 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-06 19.46.32.jpgDec 8 00:15:36 Tower root: .d..t...... media/Photos/2017-12-December/Dec 8 00:15:36 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-06 19.47.05.jpgDec 8 00:15:36 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-06 19.47.45.jpgDec 8 00:15:36 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-06 19.51.09.jpgDec 8 00:15:36 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-07 09.17.04.jpgDec 8 00:15:37 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-07 09.22.12.jpgDec 8 00:15:37 Tower root: .d..t...... media/Photos/2017-12-December/Dec 8 00:15:37 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-07 09.22.17.jpgDec 8 00:15:37 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-07 09.22.20.jpgDec 8 00:15:37 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-07 09.22.25.jpgDec 8 00:15:37 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-07 09.22.27.jpgDec 8 00:15:37 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-07 09.22.34.jpgDec 8 00:15:38 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-07 11.40.33.jpgDec 8 00:15:38 Tower root: .d..t...... media/Photos/2017-12-December/Dec 8 00:15:38 Tower root: >f+++++++++ media/Photos/2017-12-December/2017-12-07 11.40.45.jpgDec 8 00:15:38 Tower root: .d..t...... media/Photos/Dec 8 00:15:38 Tower root: .d..t...... media/Dec 8 00:15:38 Tower move: rmdir: /mnt/cache/./media Directory not emptyDec 8 00:15:38 Tower root: mover finishedDec 8 01:00:01 Tower speedtest: Internet bandwidth test startedDec 8 01:00:02 Tower root: /mnt/cache: 258.7 GiB (277808664576 bytes) trimmedDec 8 01:00:31 Tower speedtest: Host: Triple C (Petah Tikva) [22.27 km]Dec 8 01:00:31 Tower speedtest: Ping (Lowest): 212.445 ms | Download (Max): 97.85 Mbit/s | Upload (Max): 1.81 Mbit/sDec 8 01:00:31 Tower speedtest: Internet bandwidth test completedDec 8 02:00:01 Tower speedtest: Internet bandwidth test startedDec 8 02:00:32 Tower speedtest: Host: Triple C (Petah Tikva) [22.27 km]Dec 8 02:00:32 Tower speedtest: Ping (Lowest): 188.11 ms | Download (Max): 98.11 Mbit/s | Upload (Max): 1.84 Mbit/sDec 8 02:00:32 Tower speedtest: Internet bandwidth test completedDec 8 02:12:34 Tower kernel: md: recovery thread: P corrected, sector=3907515136Dec 8 02:27:22 Tower afpd[8760]: Reconnect: invalidated child[10349]Dec 8 02:27:27 Tower afpd[12573]: afp_disconnect: primary reconnect failedDec 8 02:27:33 Tower kernel: mdcmd (40): spindown 2Dec 8 02:32:48 Tower root: Fix Common Problems Version 2017.11.25Dec 8 02:32:51 Tower root: Fix Common Problems: Warning: Docker Application binhex-delugevpn has an update available for itDec 8 02:32:51 Tower root: Fix Common Problems: Warning: Docker Application EmbyServer has an update available for itDec 8 02:32:51 Tower root: Fix Common Problems: Warning: Docker Application nextcloud has an update available for itDec 8 02:32:51 Tower root: Fix Common Problems: Warning: Docker Application plex has an update available for itDec 8 02:32:54 Tower root: Fix Common Problems: Other Warning: Docker application CrashPlan has moderator comments listed ** IgnoredDec 8 02:36:01 Tower root: Fix Common Problems: Troubleshooting mode activatedDec 8 02:36:01 Tower root: Fix Common Problems: Capturing diagnostics. When uploading diagnostics to the forum, also upload /logs/FCPsyslog_tail.txt on the flash driveDec 8 02:36:03 Tower root: Fix Common Problems Version 2017.11.25Dec 8 02:36:04 Tower root: Fix Common Problems: /var/log currently 3 % fullDec 8 02:36:04 Tower root: Fix Common Problems: rootfs (/) currently 2 % fullDec 8 02:40:45 Tower kernel: vgaarb: device changed decodes: PCI:0000:03:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+memDec 8 02:40:45 Tower kernel: br0: port 3(vnet1) entered blocking stateDec 8 02:40:45 Tower kernel: br0: port 3(vnet1) entered disabled stateDec 8 02:40:45 Tower kernel: device vnet1 entered promiscuous modeDec 8 02:40:45 Tower kernel: br0: port 3(vnet1) entered blocking stateDec 8 02:40:45 Tower kernel: br0: port 3(vnet1) entered forwarding stateDec 8 02:40:57 Tower kernel: vfio-pci 0000:00:1b.0: enabling device (0000 -> 0002)Dec 8 02:40:57 Tower kernel: vfio_ecap_init: 0000:00:1b.0 hiding ecap 0x5@0x130Dec 8 02:40:57 Tower kernel: vfio-pci 0000:0a:02.0: enabling device (0000 -> 0002)Dec 8 02:40:57 Tower kernel: vfio_ecap_init: 0000:03:00.0 hiding ecap 0x19@0x900Dec 8 02:40:59 Tower kernel: usb 2-1.4: reset full-speed USB device number 5 using ehci-pciDec 8 02:41:00 Tower kernel: vgaarb: device changed decodes: PCI:0000:04:00.0,olddecodes=io+mem,decodes=io+mem:owns=noneDec 8 02:41:00 Tower kernel: br0: port 4(vnet2) entered blocking stateDec 8 02:41:00 Tower kernel: br0: port 4(vnet2) entered disabled stateDec 8 02:41:00 Tower kernel: device vnet2 entered promiscuous modeDec 8 02:41:00 Tower kernel: br0: port 4(vnet2) entered blocking stateDec 8 02:41:00 Tower kernel: br0: port 4(vnet2) entered forwarding stateDec 8 02:41:01 Tower kernel: vfio-pci 0000:03:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffffDec 8 02:41:01 Tower kernel: vfio-pci 0000:03:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffffDec 8 02:41:05 Tower kernel: usb 2-1.4: reset full-speed USB device number 5 using ehci-pciDec 8 02:41:05 Tower kernel: usb 2-1.4: reset full-speed USB device number 5 using ehci-pciDec 8 02:41:05 Tower kernel: usb 2-1.4: reset full-speed USB device number 5 using ehci-pciDec 8 02:41:10 Tower kernel: kvm: zapping shadow pages for mmio generation wraparoundDec 8 02:41:10 Tower kernel: kvm: zapping shadow pages for mmio generation wraparoundDec 8 02:41:31 Tower kernel: vfio-pci 0000:04:00.0: enabling device (0140 -> 0143)Dec 8 02:41:31 Tower kernel: vfio_ecap_init: 0000:04:00.0 hiding ecap 0x19@0x270Dec 8 02:41:31 Tower kernel: vfio_ecap_init: 0000:04:00.0 hiding ecap 0x1b@0x2d0Dec 8 02:41:31 Tower kernel: vfio_ecap_init: 0000:04:00.0 hiding ecap 0x1e@0x370Dec 8 02:41:35 Tower kernel: br0: port 5(vnet3) entered blocking stateDec 8 02:41:35 Tower kernel: br0: port 5(vnet3) entered disabled stateDec 8 02:41:35 Tower kernel: device vnet3 entered promiscuous modeDec 8 02:41:35 Tower kernel: br0: port 5(vnet3) entered blocking stateDec 8 02:41:35 Tower kernel: br0: port 5(vnet3) entered forwarding stateDec 8 02:41:41 Tower kernel: kvm: zapping shadow pages for mmio generation wraparoundDec 8 02:41:41 Tower kernel: kvm: zapping shadow pages for mmio generation wraparoundDec 8 02:41:48 Tower kernel: kvm: zapping shadow pages for mmio generation wraparoundDec 8 02:41:48 Tower kernel: kvm: zapping shadow pages for mmio generation wraparoundDec 8 02:41:54 Tower kernel: usb 2-1.4: reset full-speed USB device number 5 using ehci-pciDec 8 02:42:01 Tower kernel: usb 2-1.4: reset full-speed USB device number 5 using ehci-pciDec 8 02:45:06 Tower emhttp: shcmd (8772): /usr/local/sbin/mover |& logger &Dec 8 02:45:06 Tower root: mover startedDec 8 02:45:06 Tower root: moving "Downloads" to arrayDec 8 02:45:06 Tower move: skipping file in-use: ./Downloads/.AppleDB/__db.001Dec 8 02:45:06 Tower move: skipping file in-use: ./Downloads/.AppleDB/__db.002Dec 8 02:45:06 Tower move: skipping file in-use: ./Downloads/.AppleDB/__db.003Dec 8 02:45:06 Tower move: skipping file in-use: ./Downloads/.AppleDB/__db.004Dec 8 02:45:07 Tower move: skipping file in-use: ./Downloads/.AppleDB/__db.005Dec 8 02:45:07 Tower move: skipping file in-use: ./Downloads/.AppleDB/__db.006Dec 8 02:45:07 Tower move: rmdir: /mnt/cache/./Downloads/.AppleDB Directory not emptyDec 8 02:45:07 Tower move: rmdir: /mnt/cache/./Downloads Directory not emptyDec 8 02:45:07 Tower root: moving "media" to arrayDec 8 02:45:07 Tower move: skipping file in-use: ./media/.AppleDB/log.0000001293Dec 8 02:45:07 Tower move: skipping file in-use: ./media/.AppleDB/__db.001Dec 8 02:45:07 Tower move: skipping file in-use: ./media/.AppleDB/__db.002Dec 8 02:45:07 Tower move: skipping file in-use: ./media/.AppleDB/__db.003Dec 8 02:45:07 Tower move: skipping file in-use: ./media/.AppleDB/__db.004Dec 8 02:45:07 Tower move: skipping file in-use: ./media/.AppleDB/__db.005Dec 8 02:45:07 Tower move: skipping file in-use: ./media/.AppleDB/__db.006Dec 8 02:45:07 Tower move: rmdir: /mnt/cache/./media/.AppleDB Directory not emptyDec 8 02:45:07 Tower move: rmdir: /mnt/cache/./media Directory not emptyDec 8 02:45:07 Tower root: mover finishedDec 8 02:46:04 Tower root: Fix Common Problems Version 2017.11.25Dec 8 02:46:06 Tower root: Fix Common Problems: /var/log currently 3 % fullDec 8 02:46:06 Tower root: Fix Common Problems: rootfs (/) currently 2 % fullDec 8 02:50:34 Tower kernel: usb 2-1.4: reset full-speed USB device number 5 using ehci-pciDec 8 02:50:38 Tower kernel: usb 2-1.4: reset full-speed USB device number 5 using ehci-pciDec 8 02:50:38 Tower kernel: usb 2-1.4: reset full-speed USB device number 5 using ehci-pciDec 8 02:50:39 Tower kernel: usb 2-1.4: reset full-speed USB device number 5 using ehci-pciDec 8 02:51:18 Tower kernel: usb 2-1.4: reset full-speed USB device number 5 using ehci-pciDec 8 02:51:25 Tower kernel: usb 2-1.4: reset full-speed USB device number 5 using ehci-pciDec 8 02:56:06 Tower root: Fix Common Problems Version 2017.11.25Dec 8 02:56:08 Tower root: Fix Common Problems: /var/log currently 3 % fullDec 8 02:56:08 Tower root: Fix Common Problems: rootfs (/) currently 2 % fullDec 8 03:00:01 Tower speedtest: Internet bandwidth test startedDec 8 03:00:01 Tower CA Backup/Restore: #######################################Dec 8 03:00:01 Tower CA Backup/Restore: Community Applications appData BackupDec 8 03:00:01 Tower CA Backup/Restore: Applications will be unavailable duringDec 8 03:00:01 Tower CA Backup/Restore: this process. They will automaticallyDec 8 03:00:01 Tower CA Backup/Restore: be restarted upon completion.Dec 8 03:00:01 Tower CA Backup/Restore: #######################################Dec 8 03:00:01 Tower CA Backup/Restore: Stopping binhex-delugevpnDec 8 03:00:02 Tower kernel: docker0: port 1(vethef3bc61) entered disabled stateDec 8 03:00:02 Tower kernel: vethfbed941: renamed from eth0Dec 8 03:00:02 Tower kernel: docker0: port 1(vethef3bc61) entered disabled stateDec 8 03:00:02 Tower kernel: device vethef3bc61 left promiscuous modeDec 8 03:00:02 Tower kernel: docker0: port 1(vethef3bc61) entered disabled stateDec 8 03:00:03 Tower CA Backup/Restore: docker stop -t 60 binhex-delugevpnDec 8 03:00:03 Tower CA Backup/Restore: Stopping CrashPlanDec 8 03:00:03 Tower CA Backup/Restore: docker stop -t 60 CrashPlanDec 8 03:00:03 Tower CA Backup/Restore: Stopping EmbyServerDec 8 03:00:08 Tower CA Backup/Restore: docker stop -t 60 EmbyServerDec 8 03:00:08 Tower CA Backup/Restore: Stopping NetdataDec 8 03:00:09 Tower CA Backup/Restore: docker stop -t 60 NetdataDec 8 03:00:09 Tower CA Backup/Restore: Stopping PrinterCUPSDec 8 03:00:09 Tower avahi-daemon[8767]: Files changed, reloading.Dec 8 03:00:09 Tower avahi-daemon[8767]: Service group file /services/AirPrint-HP_Deskjet_F4500_series.service vanished, removing services.Dec 8 03:00:09 Tower kernel: veth79783fd: renamed from eth0Dec 8 03:00:09 Tower kernel: docker0: port 2(vethbff74b0) entered disabled stateDec 8 03:00:10 Tower kernel: docker0: port 2(vethbff74b0) entered disabled stateDec 8 03:00:10 Tower kernel: device vethbff74b0 left promiscuous modeDec 8 03:00:10 Tower kernel: docker0: port 2(vethbff74b0) entered disabled stateDec 8 03:00:10 Tower CA Backup/Restore: docker stop -t 60 PrinterCUPSDec 8 03:00:10 Tower CA Backup/Restore: Backing up USB Flash drive config folder to Dec 8 03:00:10 Tower CA Backup/Restore: Using command: /usr/bin/rsync -avXHq --delete --log-file="/var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log" /boot/ "/mnt/disks/WDC_WD10EAVS-00D7B0_WD-WCAU40879013/USB FLASH backup/" > /dev/null 2>&1Dec 8 03:00:10 Tower CA Backup/Restore: Backing Up appData from /mnt/user/appdata/ to /mnt/disks/WDC_WD10EAVS-00D7B0_WD-WCAU40879013/CommunityApplicationsAppdataBackup/[email protected]Dec 8 03:00:10 Tower CA Backup/Restore: Using command: cd '/mnt/user/appdata/' && /usr/bin/tar -cvaf '/mnt/disks/WDC_WD10EAVS-00D7B0_WD-WCAU40879013/CommunityApplicationsAppdataBackup/[email protected]/CA_backup.tar' --exclude 'docker.img' * >> /var/lib/docker/unraid/ca.backup2.datastore/appdata_backup.log 2>&1 & echo $! > /tmp/ca.backup2/tempFiles/backupInProgressDec 8 03:00:12 Tower ntpd[2289]: Deleting interface #3 docker0, 172.17.0.1#123, interface stats: received=0, sent=0, dropped=0, active_time=21324 secsDec 8 03:00:30 Tower speedtest: Host: Triple C (Petah Tikva) [22.27 km]Dec 8 03:00:30 Tower speedtest: Ping (Lowest): 137.571 ms | Download (Max): 98.54 Mbit/s | Upload (Max): 1.92 Mbit/sDec 8 03:00:30 Tower speedtest: Internet bandwidth test completedDec 8 03:03:53 Tower upsd[7191]: Data for UPS [ups] is stale - check driverDec 8 03:03:56 Tower upsmon[7195]: Poll UPS [[email protected]] failed - Data staleDec 8 03:03:56 Tower upsmon[7195]: Communications with UPS [email protected] lostDec 8 03:04:04 Tower upsd[7191]: UPS [ups] data is no longer staleDec 8 03:04:04 Tower upsmon[7195]: Poll UPS [[email protected]] failed - Data staleDec 8 03:04:12 Tower upsmon[7195]: Communications with UPS [email protected] establishedDec 8 03:04:41 Tower upsd[7191]: Data for UPS [ups] is stale - check driverDec 8 03:05:01 Tower upsd[7191]: UPS [ups] data is no longer staleDec 8 03:05:20 Tower upsd[7191]: Data for UPS [ups] is stale - check driverDec 8 03:05:59 Tower afpd[11625]: afp_alarm: child timed out, entering disconnected stateDec 8 03:05:59 Tower afpd[11629]: afp_alarm: child timed out, entering disconnected stateDec 8 03:06:02 Tower upsmon[7195]: Poll UPS [[email protected]] failed - Data staleDec 8 03:06:02 Tower upsmon[7195]: Communications with UPS [email protected] lostDec 8 03:06:06 Tower afpd[12573]: afp_alarm: child timed out, entering disconnected stateDec 8 03:06:06 Tower afpd[8760]: child[12573]: asev_del_fd: 11Dec 8 03:07:46 Tower blazer_usb[7089]: Communications with UPS lost: status read failed!Dec 8 03:07:49 Tower upsd[7191]: UPS [ups] data is no longer staleDec 8 03:08:15 Tower upsd[7191]: Data for UPS [ups] is stale - check driverDec 8 03:08:23 Tower upsmon[7195]: Poll UPS [[email protected]] failed - Write error: Broken pipeDec 8 03:09:19 Tower upsd[7191]: User [email protected] logged into UPS [ups]Dec 8 03:09:23 Tower upsd[7191]: User [email protected] logged out from UPS [ups]Dec 8 03:09:23 Tower upsmon[7195]: Can't login to UPS [[email protected]]: Server disconnectedDec 8 03:09:31 Tower upsmon[7195]: Poll UPS [[email protected]] failed - Driver not connectedDec 8 03:09:38 Tower upsd[7191]: User [email protected] logged into UPS [ups]Dec 8 03:09:38 Tower upsmon[7195]: Poll UPS [[email protected]] failed - Data staleDec 8 03:09:38 Tower kernel: unregister_netdevice: waiting for lo to become free. Usage count = 1Dec 8 03:09:38 Tower kernel: Uhhuh. NMI received for unknown reason 31 on CPU 0.Dec 8 03:09:38 Tower kernel: Do you have a strange power saving mode enabled?Dec 8 03:09:38 Tower kernel: Dazed and confused, but trying to continueDec 8 03:09:38 Tower kernel: Uhhuh. NMI received for unknown reason 21 on CPU 0.Dec 8 03:09:38 Tower kernel: Do you have a strange power saving mode enabled?Dec 8 03:09:38 Tower kernel: Dazed and confused, but trying to continueDec 8 03:09:38 Tower kernel: Uhhuh. NMI received for unknown reason 21 on CPU 0.Dec 8 03:09:38 Tower kernel: Do you have a strange power saving mode enabled?Dec 8 03:09:38 Tower kernel: Dazed and confused, but trying to continueDec 8 03:09:38 Tower kernel: INFO: NMI handler (perf_event_nmi_handler) took too long to run: 199.444 msecsDec 8 03:09:38 Tower kernel: Uhhuh. NMI received for unknown reason 21 on CPU 0.Dec 8 03:09:38 Tower kernel: Do you have a strange power saving mode enabled?Dec 8 03:09:38 Tower kernel: Dazed and confused, but trying to continueDec 8 03:09:38 Tower kernel: Uhhuh. NMI received for unknown reason 21 on CPU 0.Dec 8 03:09:38 Tower kernel: Do you have a strange power saving mode enabled?Dec 8 03:09:38 Tower kernel: Dazed and confused, but trying to continueDec 8 03:09:38 Tower kernel: Uhhuh. NMI received for unknown reason 21 on CPU 0.Dec 8 03:09:38 Tower kernel: Do you have a strange power saving mode enabled?Dec 8 03:09:38 Tower kernel: Dazed and confused, but trying to continueDec 8 03:09:38 Tower kernel: Uhhuh. NMI received for unknown reason 21 on CPU 0.Dec 8 03:09:38 Tower kernel: Do you have a strange power saving mode enabled?Dec 8 03:09:38 Tower kernel: Dazed and confused, but trying to continueDec 8 03:09:38 Tower kernel: hrtimer: interrupt took 2219098372 nsDec 8 03:09:38 Tower kernel: Uhhuh. NMI received for unknown reason 21 on CPU 0.Dec 8 03:09:38 Tower kernel: Do you have a strange power saving mode enabled?Dec 8 03:09:38 Tower kernel: Dazed and confused, but trying to continueDec 8 03:09:38 Tower kernel: Uhhuh. NMI received for unknown reason 21 on CPU 0.Dec 8 03:09:38 Tower kernel: Do you have a strange power saving mode enabled?Dec 8 03:09:38 Tower kernel: Dazed and confused, but trying to continue tower-diagnostics-20171208-0809.zip FCPsyslog_tail.txt Edited December 8, 20178 yr by dadarara
December 8, 20178 yr Author anyone? BTW I did run MEMtest. it passed ok one pass (long one) and as it happens once a day at night around 3am my time, it doesn't look to me as a hardware problem that can occur anytime. Edited December 8, 20178 yr by dadarara
December 10, 20178 yr Author I have moved the CA backup to run at 12PM. the crashes stopped for now. what does this mean? if its only in my server, than so be it. but if its some bug than maybe need to investigate ?
Archived
This topic is now archived and is closed to further replies.