Please help. My unRAID is constantly locking up, and has become unusable.

[relink]

53 minutes ago, Vr2Io said:

Slow expected, but crash abnormal,

Well to be fair the writes didn't full on crash, they are just terribly slow. They writes are running around 50KB/s, however the parity check is running around 120MB/s, so im assuming the Parity check has a higher priority over the SMB transfer...this is all still running by the way. 

[relink]

So, my server made it through the night and it's still running right now. I only have one error in my syslog this morning;

blk_update_request: critical target error, dev sda, sector 76048 op 0x3:(DISCARD) flags 0x800 phys_seg 1 prio class 0

 

Im not sure how or what changed but let me try to layout the current status and my thoughts. 

 

Since yesterday morning the following dockers have been disabled:

-Sabnzbd

-Sonarr

-Radarr

-Lidarr

-AMD

 

Even with those being disabled that still left a lot running, such as Plex, Nextcloud, MariaDB, NPM, PiHole, etc. 

 

I am actually watching Plex while typing this and its working fine.

 

Yesterday I had a friend remotely play a 4K HDR movie and transcode it to 1080P with tone-mapping, all while 2 4K movies direct played in my house. While that was going on, I started a transfer for about 600GB to a non cache enabled share, and I also started a transfer from my unraid server of around another 600GB, this all ran fine. So I decided to double down and I started a non-correcting parity check on top of all of that. The Parity check started out kind of slow, and after a couple minutes made it up-to about 120MB/s, while the writes to the non-cache share slowed proportionately. Once the parity check hit 120MB/s the writes essentially stopped (showed 0Kb/s), when I stopped the parity check the transfers picked right back up. 

 

I did all that because the most prevalent error in my syslog pertained to "aacraid" which is my raid controller. so I stressed it to the best of my ability and I could not make it crash again. 

 

I did get one of those macvlan call traces yesterday morning, but It didn't seem to have caused any issues when it happened. I didn't really understand what the fix for that was either. 

 

Im not sure what else to check now, I find it hard to believe that any of my containers were capable if causing hardware errors, that i couldn't reproduce myself with the transfers i did. I've had the custom IPs for several years and had no problems, but I'm still willing to try the fix, I just don't know how. 

[Vr2Io]

Sense founding. But I haven't much advice, problem seems relate to some docker.

Edited April 30, 2021 by Vr2Io

[relink]

On 4/30/2021 at 8:57 AM, Vr2Io said:

problem seems relate to some docker

I was beginning to think the same thing. Specifically Sonarr, Radarr, Lidarr, and their accompanying app as they can all pull in large amounts of data, and theres nothing stopping the 3 of them from all trying to do so at the same time. So I applied the below solution to all of them and so far so good...im almost 24hours stable. 

 

First off I pinned them all to only 4 out of 12 threads. Next I added the below options to "Extra Parameters" on each container to throttle them back;

  --memory=2G --cpu-shares=100 --device-write-bps /dev/sda:10MB

 

Only time will tell if the issue is solved.

[Vr2Io]

8 minutes ago, relink said:

 

Only time will tell if the issue is solved.

Agree.

 

Greate to know those parameters for container.

 

The boot USB not /dev/sda ?

Edited May 1, 2021 by Vr2Io

[relink]

3 hours ago, Vr2Io said:

The boot USB not /dev/sda ?

This is what every example I could find said to use. It seemed a little off to me too, so I’m still looking into it, but haven’t found any different info yet.

[relink]

Hmmm, yah it looks like “/dev/sda/“ doesn’t even exist in the container. I would need to use “shfs” which apparently doesn’t work with this variable, as it expects a physical device...crap. Well the other ones work, and if the write become too much then I’ll just set the containers to write to the cache and I’ll run the mover at a lower IO priority.