February 9, 20206 yr Hello, Longtime lurker first, time asking for assistance. My system is constantly crashing hard and I am running out of steps to troubleshoot by myself Gigabyte UD3 / 4970 / 32GB 500W psu LSI 9200-8i 2x 4TB + 2x 2TB drives in xfs on array (off mobo) 8x 250GB ssd in RAID10 btfrs on cache (off hba) 1x 500GB ssd (off mobo) *4TB parity disk (currently removed from array) *Radeon 7870 (currently physically removed) It started with errors off disk1 (xfs corruption) which was a 1yr 4TB drive. Attempts to rebuild resulted in the system locking up and I was becoming unsure of the integrity of the parity disk as well. I removed the parity disk from array and ran an xfs repair on disk1 which came up ok Various errors made me doubt the integrity of sata cables so these were changed. I also moved all the cache ssd to the hba instead of mobo (was a 6/2 mix prior) Other changes/tests to try and isolate -Removed graphics card -Moved HBA to different slot (was on pcie2.0x4 slot, moved to pcie3.0x8) -Ran memtest - full test 1pass - all ok -Diskcheck in maintainance mode - all ok -VMs and Docker turned off -Leaving idle in maintainance mode -Ran extended smart test on a few drives - all ok While idling in maintanence mode it might run for up to 4-8 hours before locking up. I've been observing this all week from work. I am currently trying to preclear the parity disk to use again, however running the preclear seems to bring an error about much earlier. I managed to pull a diagnostic very close to most recent crash Any help greatly appreciated Cheers, Andy ap-ur01-diagnostics-20200208-1413.zip syslog.txt Edited February 9, 20206 yr by Andyman added version
February 9, 20206 yr Author Attached is a photo of console monitor attached to server during one crash last week. When it happens ts unresponsive to direct attached keyboard. Requires hard reset
February 9, 20206 yr Community Expert Try running in safe mode for a few hours with dockers/VMs stopped, if it crashes like that it's likely a hardware problem, if it doesn't start enabling the plugins, dockers and VMs a few at a time.
February 9, 20206 yr Author 44 minutes ago, johnnie.black said: Try running in safe mode for a few hours with dockers/VMs stopped, if it crashes like that it's likely a hardware problem, if it doesn't start enabling the plugins, dockers and VMs a few at a time. Cool thankyou will try. I have had docker and vm service turned off for this whole week while watching this. Is there any particular plugins that are common culprits? Would like to try turning on the ones most likely to error out first ca.backup2.plg - 2019.10.27 ca.cfg.editor.plg - 2019.07.07 ca.mover.tuning.plg - 2019.08.23 ca.turbo.plg - 2020.01.26 ca.update.applications.plg - 2019.10.13 community.applications.plg - 2020.02.01 dynamix.active.streams.plg - 2019.01.03 dynamix.cache.dirs.plg - 2018.11.18 dynamix.file.integrity.plg - 2018.04.20 dynamix.local.master.plg - 2016.09.13a dynamix.s3.sleep.plg - 2018.02.04 dynamix.ssd.trim.plg - 2017.04.23a dynamix.system.stats.plg - 2019.01.31c dynamix.wireguard.plg - 2020.01.27 file.activity.plg - 2019.02.10a fix.common.problems.plg - 2019.12.29 NerdPack.plg - 2019.12.31 preclear.disk.plg - 2020.01.17b tips.and.tweaks.plg - 2019.10.17 unassigned.devices.plg - 2020.02.02 unbalance.plg - v2019.10.26 unRAIDServer.plg - 6.8.1 user.scripts.plg - 2019.08.17 wakeonlan.plg - 2019.12.30
February 9, 20206 yr Author 6 minutes ago, johnnie.black said: Running is safe mode will disable them all. Yes I am aware of that. I meant for when turning them back on
February 9, 20206 yr Community Expert 13 minutes ago, Andyman said: Yes I am aware of that. I meant for when turning them back on I would think that the one most likely to cause problem is NerdPack
February 9, 20206 yr Community Expert Don't worry about that for now, first see if running is safe mode helps, if it doesn't it's not a plugin.
Archived
This topic is now archived and is closed to further replies.