Server freezing randomly with 6.9.1 and 6.9.0


17 posts in this topic Last Reply

Recommended Posts

Hi,

 

I'm having problem with the 6.9 version of Unraid. The server freezes at random times, and can only be fixed by doing a hard reset on the server itself. It does not respond to ping when it has freezed. It happened a few days after I upgraded, then it ran for around 2 weeks before freezing again. Yesterday it froze again, restarted it and now it freezed again around 15 minutes ago. I did not have these problem with 6.8 and has not changed anything in the setup.

I tried to check the log files but it seems like it was reset when server was rebooted again. Is there any way seeing what happened from the diagnostic file?

 

Anyone has any idea of what's causing this?

 

Thanks!
 

unraid-diagnostics-20210329-1011.zip

Link to post
12 minutes ago, JorgeB said:

You can try this and then post that log.

 

Thank you very much. Great, I've enabled the syslog server and I'll post the log as soon as the server freezes again.

Link to post

@Linus, I was running into stability issues as well on my server with 6.9.1-stable, which the frequency of freezing/locking-up increased over the past few days (where I would get a few hours or so of stability after bringing the system back online).

 

For me, I may have been experiencing a bug (that others reported) regarding using Docker containers on br0 (with a static IP). However, I was never able to capture the telltale "kernel panic" error in the logs (due to my syslog server config previously not working). I attempted downgrading back to 6.8 (since using VLANs was not an option for me). Unfortunately, it did not recognize my cache drive being formatted as XFS. Therefore, I used the Unraid USB Creator tool, and downloaded 6.9.0-rc2 (which worked for me in the past). Afterward, I restored my config backup, booted up my server, and have been stable so far (I'll have to see if it remains online overnight)...

Edited by MarkRMonaco
Link to post

Spoke too soon with mine... Went down overnight.

 

Logs showed a self-reported CPU stall. From what I've seen on the forum, it's pointing to cache drive corruption. So, I ran xfs_repair and rebooted. Back to monitoring...

Edited by MarkRMonaco
Link to post

Hi,

 

My server crashed today again at 20:26, so it was just up for around 28 hours.

 

Syslog gives no clue at all from what I understand.

 

Mar 30 19:36:55 Ketsui emhttpd: read SMART /dev/sdh
Mar 30 19:37:07 Ketsui emhttpd: read SMART /dev/sdd
Mar 30 19:38:09 Ketsui sshd[31952]: Connection closed by 10.0.0.5 port 65213
Mar 30 19:38:09 Ketsui sshd[31952]: Close session: user root from 10.0.0.5 port 65213 id 0
Mar 30 19:38:09 Ketsui sshd[31952]: pam_unix(sshd:session): session closed for user root
Mar 30 19:38:09 Ketsui sshd[31952]: Transferred: sent 106292, received 10684 bytes
Mar 30 19:38:09 Ketsui sshd[31952]: Closing connection to 10.0.0.5 port 65213

-- Server crashes at around 20:26, so the following is when i rebooted it manually. --

Mar 30 20:31:24 Ketsui root: Delaying execution of fix common problems scan for 10 minutes
Mar 30 20:31:24 Ketsui unassigned.devices: Mounting 'Auto Mount' Devices...
Mar 30 20:31:25 Ketsui unassigned.devices: Adding disk '/dev/sdc1'...
Mar 30 20:31:25 Ketsui unassigned.devices: Mount drive command: /sbin/mount -t xfs -o rw,noatime,nodiratime,discard '/dev/sdc1' '/mnt/disks/INTEL_SSDSC2KW512G8_PHLA8471017M512DGN'
Mar 30 20:31:25 Ketsui kernel: XFS (sdc1): Mounting V5 Filesystem
Mar 30 20:31:25 Ketsui kernel: XFS (sdc1): Starting recovery (logdev: internal)
Mar 30 20:31:25 Ketsui kernel: XFS (sdc1): Ending recovery (logdev: internal)
Mar 30 20:31:25 Ketsui kernel: xfs filesystem being mounted at /mnt/disks/INTEL_SSDSC2KW512G8_PHLA8471017M512DGN supports timestamps until 2038 (0x7fffffff)
Mar 30 20:31:25 Ketsui unassigned.devices: Successfully mounted '/dev/sdc1' on '/mnt/disks/INTEL_SSDSC2KW512G8_PHLA8471017M512DGN'.
Mar 30 20:31:25 Ketsui unassigned.devices: Adding disk '/dev/sdb1'...
Mar 30 20:31:25 Ketsui unassigned.devices: Mount drive command: /sbin/mount -t xfs -o rw,noatime,nodiratime,discard '/dev/sdb1' '/mnt/disks/INTEL_SSDSC2KW512G8_PHLA8471031A512DGN'
Mar 30 20:31:25 Ketsui kernel: XFS (sdb1): Mounting V5 Filesystem

......

Link to post
7 hours ago, Linus said:

Gave up and downgraded to 6.8.3, hopefully it'll remain stable.

 

@Linus, let me know if you had any luck after downgrading. I think one of the reasons I was unsuccessful in my downgrade attempt, is that my cache drive required a XFS repair (as I mentioned in my previous post).

 

I did wind up going back to 6.9.1 since I was not seeing any differences in stability on 6.9.0-rc2. Unfortunately, I ran into another lock-up this morning after that & the XFS repair. Like your other post, I was unable to find anything useful in the syslog before/after I brought the system back online. Therefore, I wound up starting a separate topic so I could post logs, etc.

Edited by MarkRMonaco
Link to post
2 hours ago, MarkRMonaco said:

 

@Linus, let me know if you had any luck after downgrading. I think one of the reasons I was unsuccessful in my downgrade attempt, is that my cache drive required a XFS repair (as I mentioned in my previous post).

 

I'll keep you updated. It has been stable since the downgrade yesterday at least.

Link to post
On 3/31/2021 at 4:43 AM, MarkRMonaco said:

 

@Linus, let me know if you had any luck after downgrading. I think one of the reasons I was unsuccessful in my downgrade attempt, is that my cache drive required a XFS repair (as I mentioned in my previous post).

 

Server has been stable now since Tuesday, so it seems like downgrading "solved it", at least for me.

Link to post
Posted (edited)

Tried upgrading to 6.9.2 to see if that solved it, but server crashed again without any clue in the syslog.

 

Apr 12 15:06:40 Ketsui emhttpd: spinning down /dev/sdf
Apr 12 15:43:07 Ketsui emhttpd: read SMART /dev/sdj
Apr 12 15:44:46 Ketsui emhttpd: read SMART /dev/sdf
Apr 12 15:47:17 Ketsui emhttpd: read SMART /dev/sdg
Apr 12 16:05:24 Ketsui emhttpd: read SMART /dev/sde
Apr 12 16:41:27 Ketsui emhttpd: spinning down /dev/sdd
** Server crashes around 40 minutes later at 17:17 **

 

Edited by Linus
Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.