Random Crashes

[zaraki1311]

Hello,

 

It seems over that past few months I have been running into random system crashes with my unraid server. I am running version 6.3.5. A few times I have been at my computer as it started to go down. When it starts going down my VMs start having application crash and/or they lose network connectivity. Shortly after the VMs start crashing I lose access to the web interface to the server at that point I am assuming that the entire system has locked up as it does not respond when I connect a monitor and keyboard to it. 

 

I have been trying to narrow down where the issue is but it appears that it dumps the logs when I turn it off unless they are hidden somewhere. I have attached the logs since the last crash that seem to have a lot of errors that I am not sure what they are related to. My first question on this is what are these errors related to, are they possibly the cause of the crashing. The second question is there a way to get to the logs around the crash or make unraid keep those logs?

System Info.txt

arlongpark-syslog-20180124-1034.zip

[Frank1940]

You could install the 'Fix Common Problems' plugin.  Then turn on its 'Troubleshooting mode'.  That will write several log files which are periodically updated several times an hour to the logs  folder/directory on your flash drive.  These can help you problem shoot the issue or you could upload the diagnostics file and the last syslog file in your next post to this thread.  

[zaraki1311]

I have attached the latest versions

 

arlongpark-diagnostics-20180124-1324.zip

arlongpark-syslog-20180124-1334.zip

[Frank1940]

I looked through the files that you uploaded and I see that there are issues with usb 3-2:  This is typical of what is being recorded in the syslog.

Jan 24 09:37:17 ArlongPark kernel: usb 3-2: new full-speed USB device number 21 using uhci_hcd
Jan 24 09:37:18 ArlongPark kernel: usb 3-2: device not accepting address 21, error -71
Jan 24 09:37:18 ArlongPark kernel: usb 3-2: new full-speed USB device number 22 using uhci_hcd
Jan 24 09:37:19 ArlongPark kernel: usb 3-2: device not accepting address 22, error -71
Jan 24 09:37:19 ArlongPark kernel: usb 3-2: new full-speed USB device number 23 using uhci_hcd
Jan 24 09:37:19 ArlongPark kernel: usb 3-2: can't set config #1, error -84
Jan 24 09:37:19 ArlongPark kernel: usb 3-2: USB disconnect, device number 23

You might want to check to see what device this port is servicing.  If it is not obvious, you might want to selectively shutdown VM's and any Docker app's one-at-a-time to see if you can figure which one is giving you the problem.  

 

I do not use any VM's and am not that familiar with them, so I can't be of much help.  Hopefully, someone who has experience with them will jump in and provide with a their thoughts.  
 

[zaraki1311]

So it is a few days later now and it has crashed again. This time i had a monitor and keyboard plugged in when it crashed. All I saw was the prompt to login. I have attached the log files from just before the crash.

arlongpark-diagnostics-20180125-1700.zip

FCPsyslog_tail.txt

[Frank1940]

I saw nothing in the files different from what I found earlier.  Let me state again,  your syslog file is being flooded with those messages that I quoted in my earlier post.  This is not a normal situation!  You will have to address that issue.  I would suspect that you have have some kind of configuration error and I would suspect that it is in a VM.  Address that issue and that may solve the crashing problem.  

 

You might want to edit your first post to edit/change the thread title (Syslog flooded with USB device errors) or you might want to start a whole new thread. 

[zaraki1311]

I know I am coming back to this a month later, but things have been somewhat stable. I managed to clear the usb errors they were related to a broken usb device. 

 

There have been 2 issues that occurred since I have fixed that issue. The first one was that suddenly one of my VMs went offline I hopped on a computer to take a look and the other VM was still running. It appeared that the folder for the VM file was gone, shortly after the other VM started acting odd and then just stopped. I rebooted the unraid machine. When it came back it would not accept the disk the VMs were on. I could not get it to take the disk so I pulled it and put it in my laptop and I could see that there was still data on the disk. even though I could get the img files off the disk KVM said they were blank. I ended up recovering from backups and moved on. 

 

The second issue happened a day or 2 ago where my VMs went down I got my PRTG alerts and I went to take a look and the web interface was down. I looked at the monitor hooked up to unraid and I had the following message on the screen. Sadly I did not have FCP in troubleshooting mode to get log files