toughiv

@JorgeB - what i have done is turn off tranmission for the last few days and no crashes. I think there is something there about all the file operations of moving to the "completed" folder and then that file being moved again by another docker container. Have you heard of that as an issue? I.e. i can imagine transmission not releasing the lock on the file, then the other container causing issue on the array?

@JorgeB Maybe i spoke to soon It crashed for the first time last night. The only operation that could have been occurring at the time would have been a move operation. Perhaps I should use hardlink instead. However, does that mean one of the disks is likely bad? And if so - do you have any tips on how I might diagnose which one? They all have a smart status of perfect/fine.

so almost 2 weeks - no crashes. Thankfully the NVIDIA drivers still can be passed through to my containers. What i have ended up doing is no using templates for anything, i.e. because im in Safe Mode - Community Apps arent available to me. So I have had to: curl -L "https://github.com/docker/compose/releases/download/v2.29.7/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose sudo chmod +x /usr/local/bin/docker-compose then maintain my own compose files. definitely could be worse!

So I have had it in Safe Mode (GUI) for one week with Docker turned on (not running VMs) and no crashes. Im guessing that means it is a plugin or something that is causing the crash? I am running minimal plugins. Literally the NVidia, Community and Common Problems only i think...

Sorry for late reply - i've been between houses, but should be settled now. Do you mean go into safe mode, turn on the array and do not start docker or vms ?

- Running both [Safe Mode] & [GUI w/o Array] didn't cause any crashes. - It seems turning on the array is causing crashes...and the crashes happen more frequently if i run all my docker containers. I do not run VMs at this time, only docker. It is something to do with docker i think, given it only crashes when i turn the array on. However, i did the change to IPVLan rather than MacVLan. My setup doesn't seem too unique, i have one ethernet port, that feeds into my pfSense router. The pfSense has vlan tags of: 0, 1 & 2. The networks on UnRaid match those tags and get assigned ip addresses from the router, not frmo the Docker network. That's it. Im currently running a grand total of 2 containers, one being Plex.

@JorgeB - I have replaced the RAM and UnRaid still randomly restarted. So now I have proven: - PSU is fine - Replaced the MoBo - Replaced the CPU - Replaced the RAM - Run it without the GPU All still crashes. Surely this is an UnRaid issue?

Okay dont close this thread please - ill get some RAM bought and post results (fingers crossed!)

So just run the box without the GPU plugged in and it crashed. Even though i run memtest for 36H, it could still be the RAM? If so, i'll buy a little 16GB and give it a go... Feels like such an odd issue

in fact perhaps it just means the GPU is dying?

it actually still tries to install the nvidia third party plugin even when that safe mode option selected

I just tried to boot into Safe Mode - Gui - No plugins and it still crashed... maybe red herring on the nvidia front

It has just crashed now and even when booting, it'll crash again and do so a couple times until it manages to stay up and running Given that it'll crash before the array has started in those instances, what do you think that could be? In fact it just happened and i was watching the screen and it crashed when the third party drivers (nvidia plugin) was being installed on boot ... maybe a coincidence. It is booting back up now...let's see if it happens there again. Yep it happened there twice!! Maybe it is the nvidia plugin?

Originally it was crashing with Docker running. I had one VM but it was barely anything and a new addition to the stack. This has been ongoing for quite some time, i ensured the array started automatically and just let it restart & do its thing... However, the crashes seemed to be more frequent lately, to the point where it was becoming a blocker to me doing my stuff. That's when i decided to make this thread and get real serious about trying to diagnose this issue. So after doing the checks mentioned before: 1 ) RAM memtest for 36 hours 2) PSU test 3) SMART Reporting for all drives You then said it may be a CPU/MoBo problem, but the system just halting and restarting more often than not is a hardware issue. So, I went on eBay and spent a couple hundred on those new bits. However, the issue persists. That's now why i run the series of tests: - Running both [Safe Mode] & [GUI w/o Array] didn't cause any crashes. - It seems turning on the array is causing crashes...and the crashes happen more frequently if i run all my docker containers. My gut it telling me to say two things: 1) The array, once started, cannot be stopped - it always says "retry unmounting disk shares" 2) Maybe there is a driver / storage issue somewhere (given crashes happen with the array turned on) However, nothing shows in the Persistent Syslog.

I meant both no docker and no vms. Just idling, with Gui + array started

So I did the following: Safe Mode /w no gui = 3 days with no crashes, so proceeded to next step GUI Mode with no array = 2 days with no crashes so proceeded to next step GUI Mode with array, but no Docker and VMS = 1 day but then a crash All disks are healthy. What could this be do you think?

should i run it in safe mode for a while to see if the restarts occur? and dont turn on VM/Docker Then incrementally bring services back online until crashes occur? Safe mode Not Safe mode (no docker) Not Safe mode /w docker

Okay so I am still experiencing crashes. I have done: 36 hour memtest -> 0 errors PSU test -> nothing wrong with PSU Swapped CPU for new one Swapped Mobo for new one All that expense for naught - as it crashed again 1 hour after booting up the new motherboard and cpu...

Hey @JorgeB how would you test board/CPU? I have run memtest for over 24h with no errors. I have checked my PSU with a power supply tester - no issues Run SMART report on all drives - no issues The crashes do not occur for anything particular like heavy usage of moving files or stressing the GPU. It occurred even when not running docker containers/just idling. The frequency varies, sometimes once in 24-48h window then others 5 times in a row, then again 6 hours later. Only had these issues after upgrading from UnRaid 5.X -> UnRaid 6.X

I am then assuming the most usual culprit is RAM? Then drives? I've not had a hardware issue before and struggling to diagnose then. Got any pointers on common tests to run and order of probability? Thank you

Please help, this is driving me insane. Please note I used to have a lot more frequent crashes when I was running lots of containers ... I clicked scrub in anger and wiped out all my appdata...ive got to rebuild everything :''( I have done the following: attached both the persistent syslog & the anonymised diagnostics. Run memtest for 3 passes and it come back clean...all disks pass SMART test. Noticed the system clock is off (will sort) and also noticed . Made the ipvlan change also from macvlan The only thing i can think is that 3 passes on memtest is not enough OR that the networking setup is causing issues. Just as an FYI my Unraid box is hooked up to pfSense. It has 1 x 1Gb port connecting it. I have 3 x VLANs to manage different subnets. All the subnets, outbound NAT, routing, etc. is handled by pfSense. tower-smart-20240708-0148 (2).zip tower-smart-20240708-0148 (1).zip tower-smart-20240708-0148.zip tower-smart-20240708-0147.zip tower-smart-20240708-0146.zip tower-diagnostics-20240706-1135.zip syslog-1720306751

@JorgeB - i have found that i spoke too soon! If this is hardware and no errors shown in RAM after 3 passes, and no errors in parity / no warnings about failing disks... what do you think it could be?

@JorgeB - I have turned off the Tdarr container for last 3 days and no issues. Have you ever heard of that causing an issue?

Okay @JorgeB- thank you

Crashed again this morning syslog-1703491413