6.7.2 Recurring Overnight System Crash


JasonM

Recommended Posts

I've been trying to isolate this issue for a while. It has survived hardware changes and complete USB rebuild. At some point overnight, the system becomes unresponsive. Dockers and VMs cease to operate. The web UI is sometimes available, and when it is, shows 2-3 CPUs pegged at 100%. Interacting with the web UI in this state works for a few navigation clicks before completely locking up. On other mornings, the web UI won't load at all. In all cases, a hard reboot is required. The server runs all day without issues.

 

In an attempt to get more data, I installed a user script that tails the syslog onto the flash drive since I'm not able to get a log at time of crash. Regular diags as well as this log are attached. Around line 460, the system seems to enter an endless loop. On one occasion, I noticed a strange message about CA Backup in my browser status bar, of which I took a screen shot. All three files are attached. Any help nailing down this issue is greatly appreciated.

Screen Shot 2019-09-10 at 6.32.20 AM.png

unraid-diagnostics-20190911-1207.zip syslog-2019-09-09_0637.txt

Edited by JasonM
Link to comment

Yes, I do have cache drives, but I only use cache-only shares. There is nothing for mover to move. I wish there were a way to disable mover completely, but there is no readily apparent way to do so. Mover is set for 1AM. I have scheduled tasks spread out enough such that they should not conflict with each other:

 

1AM Mover

2AM Auto update plugins

3AM Auto update containers

4AM CA backup

5AM SSD TRIM

 

Link to comment
10 minutes ago, JasonM said:

Yes, I do have cache drives, but I only use cache-only shares. There is nothing for mover to move. I wish there were a way to disable mover completely, but there is no readily apparent way to do so. Mover is set for 1AM. I have scheduled tasks spread out enough such that they should not conflict with each other:

 

1AM Mover

2AM Auto update plugins

3AM Auto update containers

4AM CA backup

5AM SSD TRIM

 

You could set mover to only run once a month to minimise the frequency.

 

Another technique that has been used to completely disable mover is to replace the 'mover' script with one that does nothing.

Link to comment

Frankly I did not have time yet to look into your logs (still at work), just wanted to see if your problem is connected to an open bug report for 6.7.2.

it is a known issue that reads from the array are crawling slow when a write to the array is happening at the same time. So my question on mover was to see if there is a connection to this issue. 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.