beasthouse Posted August 15 Share Posted August 15 Have been experiencing my server misbehaving the last few days. Things I have changed: Installed Immich (which has been pegging the CPU cores assigned to the container) Added a camera to Frigate which is having its own issues Additionally, my Zigbee USB Adapter keeps dropping and crashing Zigbee2MQTT. Also the Log Bar under the System Info section in the Dashboard has been full at times. I think this is a result of the server misbehaving. The below shows two lines from the logs that start repeating non stop. I have also uploaded my diagnostics. Not quite sure where to go from here. I have stopped Immich for now to see if that is the culprit, but would appreciate some advice if possible. I am not great at identifying stuff in logs. Aug 15 08:00:00 Odin kernel: BTRFS error (device nvme1n1p1: state EA): parent transid verify failed on logical 382166155264 mirror 1 wanted 1541924 found 1541859 Aug 15 08:00:00 Odin kernel: BTRFS error (device nvme1n1p1: state EA): parent transid verify failed on logical 382166155264 mirror 2 wanted 1541924 found 1541859 odin-diagnostics-20240815-0954.zip Quote Link to comment
JorgeB Posted August 15 Share Posted August 15 There is a macvlan call trace, so first update to 6.12.11 and then run a correcting scrub on that pool and post the results and new diags. Quote Link to comment
beasthouse Posted August 15 Author Share Posted August 15 (edited) 1 hour ago, JorgeB said: There is a macvlan call trace, so first update to 6.12.11 and then run a correcting scrub on that pool and post the results and new diags. Thanks for looking. I started the update but I think it has failed, cannot ping the server anymore so not sure where to go from here 😭 plugin: installing: unRAIDServer.plg Executing hook script: pre_plugin_checks plugin: downloading: unRAIDServer.plg ... done plugin: downloading: unRAIDServer-6.12.11-x86_64.zip ... done plugin: downloading: unRAIDServer-6.12.11-x86_64.md5 ... done Plugin-Update-Helper for 3rd party driver plugins found, updating... writing flash device - please wait... Archive: /tmp/unRAIDServer.zip inflating: /boot/unRAIDServer/bzfirmware inflating: /boot/unRAIDServer/bzfirmware.sha256 inflating: /boot/unRAIDServer/bzimage inflating: /boot/unRAIDServer/bzimage.sha256 inflating: /boot/unRAIDServer/bzmodules inflating: /boot/unRAIDServer/bzmodules.sha256 inflating: /boot/unRAIDServer/bzroot extracting: /boot/unRAIDServer/bzroot-gui inflating: /boot/unRAIDServer/bzroot-gui.sha256 inflating: /boot/unRAIDServer/bzroot.sha256 inflating: /boot/unRAIDServer/changes.txt inflating: /boot/unRAIDServer/license.txt inflating: /boot/unRAIDServer/make_bootable.bat inflating: /boot/unRAIDServer/make_bootable_linux inflating: /boot/unRAIDServer/make_bootable_mac inflating: /boot/unRAIDServer/memtest inflating: /boot/unRAIDServer/syslinux/syslinux.cfg- Did a physical off and on of the server and it booted up fine at v6.12.10. Have attached new diagnostics. Not sure whats going on (IPV6? I turned that on the other day for Matter support) but the logs are spitting this out every minute. Aug 15 19:59:49 Odin kernel: docker0: port 1(veth208483b) entered blocking state Aug 15 19:59:49 Odin kernel: docker0: port 1(veth208483b) entered disabled state Aug 15 19:59:49 Odin kernel: device veth208483b entered promiscuous mode Aug 15 19:59:49 Odin kernel: eth0: renamed from vethb9b7da3 Aug 15 19:59:49 Odin kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth208483b: link becomes ready Aug 15 19:59:49 Odin kernel: docker0: port 1(veth208483b) entered blocking state Aug 15 19:59:49 Odin kernel: docker0: port 1(veth208483b) entered forwarding state Aug 15 19:59:51 Odin dnsmasq[26371]: reading /etc/resolv.conf Aug 15 19:59:51 Odin dnsmasq[26371]: using nameserver 100.100.100.100#53 Aug 15 19:59:56 Odin kernel: docker0: port 1(veth208483b) entered disabled state Aug 15 19:59:56 Odin kernel: vethb9b7da3: renamed from eth0 Aug 15 19:59:56 Odin kernel: docker0: port 1(veth208483b) entered disabled state Aug 15 19:59:56 Odin kernel: device veth208483b left promiscuous mode Aug 15 19:59:56 Odin kernel: docker0: port 1(veth208483b) entered disabled state Aug 15 19:59:56 Odin dnsmasq[26371]: reading /etc/resolv.conf Aug 15 19:59:56 Odin dnsmasq[26371]: using nameserver 100.100.100.100#53 Aug 15 20:00:56 Odin kernel: docker0: port 1(veth2129720) entered blocking state Aug 15 20:00:56 Odin kernel: docker0: port 1(veth2129720) entered disabled state Aug 15 20:00:56 Odin kernel: device veth2129720 entered promiscuous mode Aug 15 20:00:56 Odin kernel: docker0: port 1(veth2129720) entered blocking state Aug 15 20:00:56 Odin kernel: docker0: port 1(veth2129720) entered forwarding state Aug 15 20:00:56 Odin kernel: docker0: port 1(veth2129720) entered disabled state Aug 15 20:00:56 Odin kernel: eth0: renamed from vethc6015ee Aug 15 20:00:56 Odin kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth2129720: link becomes ready Aug 15 20:00:56 Odin kernel: docker0: port 1(veth2129720) entered blocking state Aug 15 20:00:56 Odin kernel: docker0: port 1(veth2129720) entered forwarding state Aug 15 20:00:57 Odin dnsmasq[26371]: reading /etc/resolv.conf Aug 15 20:00:57 Odin dnsmasq[26371]: using nameserver 100.100.100.100#53 Aug 15 20:01:02 Odin kernel: docker0: port 1(veth2129720) entered disabled state Aug 15 20:01:02 Odin kernel: vethc6015ee: renamed from eth0 Aug 15 20:01:02 Odin kernel: docker0: port 1(veth2129720) entered disabled state Aug 15 20:01:02 Odin kernel: device veth2129720 left promiscuous mode Aug 15 20:01:02 Odin kernel: docker0: port 1(veth2129720) entered disabled state Aug 15 20:01:02 Odin dnsmasq[26371]: reading /etc/resolv.conf Aug 15 20:01:02 Odin dnsmasq[26371]: using nameserver 100.100.100.100#53 odin-diagnostics-20240815-1959.zip Edited August 15 by beasthouse Quote Link to comment
JorgeB Posted August 15 Share Posted August 15 Try rebooting and if it doesn't come back online post of photo of the monitor. Quote Link to comment
beasthouse Posted August 15 Author Share Posted August 15 Just now, JorgeB said: Try rebooting and if it doesn't come back online post of photo of the monitor. Thanks. It rebooted fine. I turned the docker and vm services off and tried updating again and it worked this time. Now to move on to the next steps you recommended. Quote Link to comment
beasthouse Posted August 15 Author Share Posted August 15 New Diagnostics attached and the Scrub results of my Appdata Cache: UUID: 14fe1259-f0eb-435c-a0e2-6f64f876d8fd Scrub started: Thu Aug 15 20:51:52 2024 Status: finished Duration: 0:01:00 Total to scrub: 267.32GiB Rate: 4.46GiB/s Error summary: csum=3 Corrected: 0 Uncorrectable: 3 Unverified: 0 odin-diagnostics-20240815-2054.zip Quote Link to comment
JorgeB Posted August 15 Share Posted August 15 Delete/restore these files from a backup and re-run the scrub: appdata/plex/Metadata/Movies/d/d0eef0b00be7de9694b79f635e67d1edacb79b1.bundle/Contents/_combined/posters/tv.plex.agents.movie_6e517bf920bfda4bd5eb12b15f37b1cbc35941b2 appdata/esphome/.esphome/platformio/packages/framework-espidf/examples/peripherals/sdio/README.md Quote Link to comment
beasthouse Posted August 15 Author Share Posted August 15 UUID: 14fe1259-f0eb-435c-a0e2-6f64f876d8fd Scrub started: Thu Aug 15 21:22:30 2024 Status: finished Duration: 0:01:01 Total to scrub: 267.32GiB Rate: 4.38GiB/s Error summary: no errors found That seems to have sorted it. Now to test the rest I guess. Quote Link to comment
beasthouse Posted August 15 Author Share Posted August 15 1 hour ago, JorgeB said: Delete/restore these files from a backup and re-run the scrub: appdata/plex/Metadata/Movies/d/d0eef0b00be7de9694b79f635e67d1edacb79b1.bundle/Contents/_combined/posters/tv.plex.agents.movie_6e517bf920bfda4bd5eb12b15f37b1cbc35941b2 appdata/esphome/.esphome/platformio/packages/framework-espidf/examples/peripherals/sdio/README.md curious how you found those errors, would like to become more self sufficient in the future Quote Link to comment
beasthouse Posted August 15 Author Share Posted August 15 Ok so went back to check the logs and now I am getting this repeating again non stop. Any ides? Aug 15 23:16:13 Odin kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethfb7cb6f: link becomes ready Aug 15 23:16:13 Odin kernel: docker0: port 5(vethfb7cb6f) entered blocking state Aug 15 23:16:13 Odin kernel: docker0: port 5(vethfb7cb6f) entered forwarding state Aug 15 23:16:20 Odin kernel: docker0: port 5(vethfb7cb6f) entered disabled state Aug 15 23:16:20 Odin kernel: veth7497558: renamed from eth0 Aug 15 23:16:20 Odin kernel: docker0: port 5(vethfb7cb6f) entered disabled state Aug 15 23:16:20 Odin kernel: device vethfb7cb6f left promiscuous mode Aug 15 23:16:20 Odin kernel: docker0: port 5(vethfb7cb6f) entered disabled state Aug 15 23:17:20 Odin kernel: docker0: port 5(vethd5d529d) entered blocking state Aug 15 23:17:20 Odin kernel: docker0: port 5(vethd5d529d) entered disabled state Aug 15 23:17:20 Odin kernel: device vethd5d529d entered promiscuous mode Aug 15 23:17:20 Odin kernel: docker0: port 5(vethd5d529d) entered blocking state Aug 15 23:17:20 Odin kernel: docker0: port 5(vethd5d529d) entered forwarding state Aug 15 23:17:20 Odin kernel: docker0: port 5(vethd5d529d) entered disabled state Aug 15 23:17:20 Odin kernel: eth0: renamed from vethe8c8ec8 Quote Link to comment
Solution JorgeB Posted August 15 Solution Share Posted August 15 1 hour ago, beasthouse said: curious how you found those errors, would like to become more self sufficient in the future They are in the syslog, and are logged during the scrub. 50 minutes ago, beasthouse said: Ok so went back to check the logs and now I am getting this repeating again non stop. Any ides? You may have a container constantly restarting, check the up times to see if you can find out which one it is. Quote Link to comment
beasthouse Posted August 15 Author Share Posted August 15 1 minute ago, JorgeB said: You may have a container constantly restarting, check the up times to see if you can find out which one it is. Bingo - Frigate container is restarting over and over. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.