After months of running the server without a single issue, it has randomly started going down every night. After I turn it back on in the morning, Fix Common Problems plugin tells me that there was an unclean shutdown. Also, I'm running 6.4.0-rc6, so I'm not sure if this is the right forum.

 

In addition to the diagnostic logs, I have also attached screenshots from Smokeping, which is running in a docker container on the server. They show when the server stopped running, and it doesn't appear to be happening at any consistent time.

 

I just put Fix Common Problems into Troubleshooting mode, so I should have logs directly from that if it happens again, but I also have my system log sending to Papertrail, so I do have logs from that. Here's everything that was logged a few hours before last night's shutdown:

 

Aug 04 03:22:50 MediaTower kernel:  mdcmd (123): spindown 5
Aug 04 03:22:52 MediaTower kernel:  mdcmd (124): spindown 4
Aug 04 03:25:38 MediaTower kernel:  mdcmd (125): spindown 3
Aug 04 03:29:48 MediaTower kernel:  mdcmd (126): spindown 1
Aug 04 03:29:50 MediaTower kernel:  mdcmd (127): spindown 2
Aug 04 03:53:04 MediaTower kernel:  mdcmd (128): spindown 4
Aug 04 04:00:01 MediaTower speedtest:  Internet bandwidth test started
Aug 04 04:00:24 MediaTower speedtest:  Host: Quadro Communications Co-operative Inc (Kirkton, ON) [40.73 km]
Aug 04 04:00:24 MediaTower speedtest:  Ping (Lowest): 24.816 ms | Download (Max): 36.95 Mbit/s | Upload (Max): 17.58 Mbit/s
Aug 04 04:00:24 MediaTower speedtest:  Internet bandwidth test completed
Aug 04 04:22:47 MediaTower kernel:  mdcmd (129): spindown 3
Aug 04 04:29:56 MediaTower kernel:  mdcmd (130): spindown 1
Aug 04 04:33:16 MediaTower kernel:  mdcmd (131): spindown 2
Aug 04 04:53:00 MediaTower kernel:  mdcmd (132): spindown 3
Aug 04 04:56:29 MediaTower kernel:  mdcmd (133): spindown 4
Aug 04 04:57:05 MediaTower kernel:  mdcmd (134): spindown 5
Aug 04 05:00:01 MediaTower speedtest:  Internet bandwidth test started
Aug 04 05:00:25 MediaTower speedtest:  Host: Hay Communications (Zurich, ON) [59.13 km]
Aug 04 05:00:25 MediaTower speedtest:  Ping (Lowest): 29.662 ms | Download (Max): 81.40 Mbit/s | Upload (Max): 17.58 Mbit/s
Aug 04 05:00:25 MediaTower speedtest:  Internet bandwidth test completed
Aug 04 05:00:33 MediaTower kernel:  mdcmd (135): spindown 2
Aug 04 05:22:43 MediaTower kernel:  mdcmd (136): spindown 3
Aug 04 05:22:43 MediaTower kernel:  mdcmd (137): spindown 4
Aug 04 05:22:43 MediaTower kernel:  mdcmd (138): spindown 5
Aug 04 05:52:56 MediaTower kernel:  mdcmd (139): spindown 1
Aug 04 05:52:57 MediaTower kernel:  mdcmd (140): spindown 5
Aug 04 05:54:10 MediaTower kernel:  mdcmd (141): spindown 4
Aug 04 06:00:01 MediaTower speedtest:  Internet bandwidth test started
Aug 04 06:00:01 MediaTower kernel:  docker0: port 8(veth2677c60) entered blocking state
Aug 04 06:00:01 MediaTower kernel:  docker0: port 8(veth2677c60) entered disabled state
Aug 04 06:00:01 MediaTower kernel:  device veth2677c60 entered promiscuous mode
Aug 04 06:00:01 MediaTower kernel:  IPv6: ADDRCONF(NETDEV_UP): veth2677c60: link is not ready
Aug 04 06:00:01 MediaTower kernel:  docker0: port 8(veth2677c60) entered blocking state
Aug 04 06:00:01 MediaTower kernel:  docker0: port 8(veth2677c60) entered forwarding state
Aug 04 06:00:01 MediaTower kernel:  docker0: port 8(veth2677c60) entered disabled state
Aug 04 06:00:04 MediaTower kernel:  eth0: renamed from vethfedc2cf
Aug 04 06:00:04 MediaTower kernel:  IPv6: ADDRCONF(NETDEV_CHANGE): veth2677c60: link becomes ready
Aug 04 06:00:04 MediaTower kernel:  docker0: port 8(veth2677c60) entered blocking state
Aug 04 06:00:04 MediaTower kernel:  docker0: port 8(veth2677c60) entered forwarding state
Aug 04 06:00:05 MediaTower ntpd:  bind(37) AF_INET6 fe80::304a:27ff:fe9c:92%28#123 flags 0x11 failed: Cannot assign requested address
Aug 04 06:00:05 MediaTower ntpd:  unable to create socket on veth2677c60 (25) for fe80::304a:27ff:fe9c:92%28#123
Aug 04 06:00:05 MediaTower ntpd:  failed to init interface for address fe80::304a:27ff:fe9c:92%28
Aug 04 06:00:07 MediaTower ntpd:  Listen normally on 26 veth2677c60 [fe80::304a:27ff:fe9c:92%28]:123
Aug 04 06:00:07 MediaTower ntpd:  new interface(s) found: waking up resolver
Aug 04 06:00:25 MediaTower speedtest:  Host: Hay Communications (Zurich, ON) [59.13 km]
Aug 04 06:00:25 MediaTower speedtest:  Ping (Lowest): 30.758 ms | Download (Max): 85.51 Mbit/s | Upload (Max): 17.49 Mbit/s
Aug 04 06:00:25 MediaTower speedtest:  Internet bandwidth test completed

Anyone have any guesses as to what this could be? Possibly a hardware issue?

mediatower-diagnostics-20170804-1015.zip

Edited August 4, 20178 yr by wug

Is your server connected to a UPS? Any mischievous wife, children or pets in the house?

On mobile so haven't looked at the syslog though.

Sent from my LG-H815 using Tapatalk

Nope, no UPS, and I'm the only one here with my dog crated overnight. And it's not likely an external factor either, as my mobo is set to automatically boot when power resumes. So if the power was going out or something, it would be automatically booting back up.

5 minutes ago, wug said:

So if the power was going out or something, it would be automatically booting back up.

Is it actually shut off?  Or simply hung?

 

If actually shut off, then there's two primary causes:

 

Power Supply crappy

CPU overheating.

Completely shut off. As in, I wonder why I can't connect to Plex in the morning, I walk over to the server, and notice that the fans aren't running and the lights on the connected keyboard are off.

 

As for the power supply, it's a relatively high-quality 80 Plus Gold, 750W PSU that typically is only drawing around 130W on average. I've had it a little over a year without issue. And if it was just randomly cutting out, I would still probably not be finding the server in a shut-off state, because as soon as the mobo is getting power again, it would reboot.

 

CPU overheating is a possibility. It's not overclocked or anything, but it's about five years old and still has the stock fan on it. And the middle of the night is when the mover, usenet downloads, and maintenance tasks run. How would I go about testing it?

I kinda had the same problem a couple of years ago. 

I shut off every plugin and ran unRAID with nothing on it. <----- Still had a problem

Ran a memory test < ------ No problem

Turned on unRAID and basically let it idle with no use < ------ No problem

 

Come to find out my Power Supply was having issues as well as my MotherBoard because my USB Host starting dieing which really clued me in. 

Might not be your PSU but could even be your power delivery. At least that's what i had. After getting a UPS, on every month i could get up to two alerts saying power was lost and came back a few seconds later.
Without a UPS i believe the server will be turned off when that happens. I have my server configured to boot up automatically when power resumes too, but it doesnt do that whenever the above happened, before getting the UPS.
But of course, getting a UPS just to see whether it helps or not could be a costly troubleshooting step, but for what's its worth, a UPS for a NAS is never a bad idea imo

Sent from my LG-H990 using Tapatalk