Everyday at noon my server stops working.

[N385]

I use Unraid mainly for Plex and file storage.

At noon my system cache disks seem to go into read only mode (from what I've been able to gather).

To get the server working I have to reset it. Half the time the server will not reset from 'reboot' button, so has to be hard reset.

Fix common problems will work to 22% and hang.

I have a copy of the log.

workhorse-syslog-20220915-1914.zip

Edited September 15, 2022 by N385
Add system overview picture.

[trurl]

attach diagnostics to your NEXT post in this thread

[N385]

workhorse-diagnostics-20220916-1215.zip

[trurl]

Maybe not related but worth considering

 

[trurl]

Seem to be connection problems with both cache (sdf,sdg)

 

How are these connected?

[N385]

I have a PCIe card installed that has 4 mini SAS connectors with a mini SAS to 4 SATA cable on Each.

I've swapped what connectors and even replaced the SSDs to no avail.

[JorgeB]

Issue could be power related, also look for an update for the LSI firmware, or try connecting the SSDs to the onboard SATA.

[N385]

This has been going on for over a month. In that time I have switched what cable connects what drive and I have even replaced drives in the pool. I have had the machine open at least 5 times.

The SATA connectors on the motherboard go to a virtual machine that does backups, video encoding and a lot more so they are not available.

I have purposely started watching a movie close to noon many many times and it always stops at ~12:02 (when the browsers video buffer runs out). Not at 11:58 or 12:04, but always 12:02 PM

If this were a mechanical issue it wouldn't happen on an exacting schedule. We run no AC or Heat at this time of year so it's not happening due to temperature either.

After 12:02 PM Plex will only show this error, the Unraid docker button will hang, fix common problems will hang and the 'reboot' button will not work either.

After a hard reboot the machine will work for until 12:00 PM the next day.

Edited September 17, 2022 by N385
typo

[trurl]

10 minutes ago, N385 said:

started watching a movie close to noon many many times and it always stops at ~12:02

What happens if you start a movie after noon?

[N385]

[trurl]

Is that supposed to be an answer to my question? Does that mean it already crashed before noon even though you had not started a movie at all since reboot? What happens if you start a movie hours before noon? Are you sure the time of day has anything at all to do with this? Do you have anything scheduled for that time?

[N385]

I started a movie this morning at around 11 AM and it quit at 12:02 PM as if on a schedule. I have not reboot yet today so nothing much will work until I do. I did that screen grab at 12:50 PM today. 

 

I have 4 user scripts that run once a day. I just downloaded all the logs and viewed them and they show each ran successfully at 4:40 AM one after the other this morning.

 

Other than that Mover runs once a week on Sundays at 2:00 AM and parity check runs the first Sunday of every other month at 3:00 AM.

 

I don't recall having anything scheduled for 12:00 PM

User-Script.Logs.zip

Edited September 17, 2022 by N385
Added logs

[trurl]

When does CA Backup run? When does SSD Trim run?

 

Why do you have Mover Tuning plugin installed?

 

 

[N385]

CA Backup runs monthly on the 14th @ 3AM

 

Mover Tuning is there so only files that are older than 60 days will get moved to the array.

 

It looks like I overlooked SSD trim.

SSD Trim was set to run daily at 12:00 (I have just disabled it to see if that fixes the problem).

 

[trurl]

You can do these from command line to see all schedules

crontab -l

 

cat /etc/cron.d/root

 

[N385]

I ran SSD trim manually and it definitely was the cause.

I wonder if the plugin went bad or what but It's disabled and hopefully all will run well.

[trurl]

Never had a problem with it but I only run it weekly in the middle of the night