UNRAID locks up every few hours

[housewrecker]

The WebGUI locks up after 4-24 hours.

 

I created the user script to write the logs. I attached diagnostics and the syslog. Any help here is appreciated.

blue-diagnostics-20231119-1719.zip syslog.zip syslog-20231119-171914.txt

Edited November 21, 2023 by housewrecker

[JorgeB]

Enable the syslog server and post that after a crash.

 

[housewrecker]

Okay, trying that now. I'll post when it crashes.

[housewrecker]

Here is the syslog file. I zipped it up because unraid forums was rejecting the uncompressed file with a -200 error.

syslog.zip

[housewrecker]

And now I have no docker or cache folder. Appdata can't be found either.

[JorgeB]

Nov 21 10:00:05 Blue kernel: macvlan_broadcast+0x10a/0x150 [macvlan]
Nov 21 10:00:05 Blue kernel: ? _raw_spin_unlock+0x14/0x29
Nov 21 10:00:05 Blue kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan]

 

Macvlan call traces will usually end up crashing the server, switching to ipvlan should fix it (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right)), then reboot.

[housewrecker]

Sounds good. Thank you.

 

Do you know why the cache is showing ? and how to fix that?

[JorgeB]

19 minutes ago, housewrecker said:

why the cache is showing ?

Showing where?

[housewrecker]

2 hours ago, housewrecker said:

And now I have no docker or cache folder. Appdata can't be found either.

 

 

When I do ls -la the cache shows "?". Also in the docker settings, I'm getting errors that appdata and cache folder can't be found anymore.

[JorgeB]

Post new diags.

[housewrecker]

Here it is.

blue-diagnostics-20231123-1345.zip

[dlandon]

Remove this from your go file:

modprobe i915
chmod -R 777 /dev/dri

It's no longer needed and has been known to cause issues.

 

I'm seeing a lot of these messages that might be filling your log:

Nov 23 13:18:21 Blue nginx: 2023/11/23 13:18:21 [error] 9361#9361: *24185 limiting requests, excess: 20.016 by zone "authlimit", client: 192.168.1.221, server: hash.myunraid.net, request: "Nov 23 13:18:19 Blue nginx: 2023/11/23 13:18:19 [error] 9361#9361: *24185 limiting requests, excess: 20.216 by zone "authlimit", client: 192.168.1.221, server: hash.myunraid.net, request: GET /login HTTP/1.1", host: "hash.myunraid.net"
Nov 23 13:18:24 Blue nginx: 2023/11/23 13:18:24 [error] 9361#9361: *24424 limiting requests, excess: 20.887 by zone "authlimit", client: 192.168.1.221, server: hash.myunraid.net, request: "GET /login HTTP/2.0", host: "hash.myunraid.net", referrer: "https://hash.myunraid.net/Main/Browse?dir=%2Fboot%2Flogs"

I don't have an answer for that.

[housewrecker]

Okay, will do. Thanks.

 

Do you think someone or thing is trying to get into my server?

[dlandon]

2 minutes ago, housewrecker said:

Okay, will do. Thanks.

 

Do you think someone or thing is trying to get into my server?

Probably not.

[JorgeB]

Regarding the cache issue:

 

Nov 23 10:49:01 Blue kernel: XFS (sdc1): Metadata corruption detected at xfs_bmap_validate_extent+0x41/0x68 [xfs], inode 0x40000081 xfs_iread_extents(2)
Nov 23 10:49:01 Blue kernel: XFS (sdc1): Unmount and run xfs_repair

 

Check filesystem, run it without -n.

[housewrecker]

 

1 minute ago, JorgeB said:

Regarding the cache issue:

 

Nov 23 10:49:01 Blue kernel: XFS (sdc1): Metadata corruption detected at xfs_bmap_validate_extent+0x41/0x68 [xfs], inode 0x40000081 xfs_iread_extents(2)
Nov 23 10:49:01 Blue kernel: XFS (sdc1): Unmount and run xfs_repair

 

Check filesystem, run it without -n.

Will do.

 

 

[housewrecker]

11 minutes ago, dlandon said:

Remove this from your go file:

modprobe i915
chmod -R 777 /dev/dri

It's no longer needed and has been known to cause issues.

 

I'm seeing a lot of these messages that might be filling your log:

Nov 23 13:18:21 Blue nginx: 2023/11/23 13:18:21 [error] 9361#9361: *24185 limiting requests, excess: 20.016 by zone "authlimit", client: 192.168.1.221, server: hash.myunraid.net, request: "Nov 23 13:18:19 Blue nginx: 2023/11/23 13:18:19 [error] 9361#9361: *24185 limiting requests, excess: 20.216 by zone "authlimit", client: 192.168.1.221, server: hash.myunraid.net, request: GET /login HTTP/1.1", host: "hash.myunraid.net"
Nov 23 13:18:24 Blue nginx: 2023/11/23 13:18:24 [error] 9361#9361: *24424 limiting requests, excess: 20.887 by zone "authlimit", client: 192.168.1.221, server: hash.myunraid.net, request: "GET /login HTTP/2.0", host: "hash.myunraid.net", referrer: "https://hash.myunraid.net/Main/Browse?dir=%2Fboot%2Flogs"

I don't have an answer for that.

 

I haven't done the /dev/dri work in a bit. I think it was for the Intel GPU for Plex. Will removing that file affect that? Which one is the go file?

[housewrecker]

12 minutes ago, JorgeB said:

Regarding the cache issue:

 

Nov 23 10:49:01 Blue kernel: XFS (sdc1): Metadata corruption detected at xfs_bmap_validate_extent+0x41/0x68 [xfs], inode 0x40000081 xfs_iread_extents(2)
Nov 23 10:49:01 Blue kernel: XFS (sdc1): Unmount and run xfs_repair

 

Check filesystem, run it without -n.

 

 

Phase 1 - find and verify superblock...
Phase 2 - using internal log
        - zero log...
ERROR: The filesystem has valuable metadata changes in a log which needs to
be replayed.  Mount the filesystem to replay the log, and unmount it before
re-running xfs_repair.  If you are unable to mount the filesystem, then use
the -L option to destroy the log and attempt a repair.
Note that destroying the log may cause corruption -- please attempt a mount
of the filesystem before doing this.

 

This is what I got as a response. I'm not sure how to get the file system to replay the log.

[housewrecker]

Go file is cleaned up.

Macvlan has changed to ipvlan in docker.

Then I rebooted.

 

Just stuck on the cache issue now.

 

Below is what I see in docker.

[JorgeB]

13 hours ago, housewrecker said:

This is what I got as a response. I'm not sure how to get the file system to replay the log.

Use -L

[housewrecker]

I did that. The shares disappeared. I rebooted and they came back. I re-downloaded the containers since they were gone. I needed to change some of the network settings, maybe because of the macvlan change. But I think everything is back up. The bigger question is if the two fixes will help resolve the locking up issue.

 

Thanks for the help with everything. @JorgeB @dlandon