Cache Pool is breaking randomly - ReadOnly Filesystem

[JoJ123]

Hello,

I'm using Unraid since more than one year now.

I've never had issue with the Cache, but in the last weeks the CachePool is making lot of problems. I can't rely anymore on my system ...

 

After 2-3 days, the Cache becomes ReadOnly and I can't do anything anymore and most of my Docker Containers are not usable anymore.

 

So I thought that it's maybe the "old" SSDs I were using, so I replaced the whole Cache Pool with 2 brand new SSD (256GB).

This morning I wanted to check some stuff on my nextcloud, but the Cache Pool is broken again --> ReadOnly Filesystem.

 

Last time a reboot helped but it seems to not be the case today.

I've also tried that with different numbers, but it's always returning "ReadOnly Filesystem"

btrfs balance start -dusage=1 /mnt/cache/

After the replacement of the SSD Cache Disks I've added the above command with `dusage = 80` as a user script that runs every 15 minutes.

 

Do you have an idea how I could fix it without loosing the data on the cache drive? I can for sure copy it to my unassigned disk and reformat and so on, but I'm scared I will end up in the same situation again in 3-4 days.

 

It's my first issue report, so I hope the diagnostics are as expected by the community.

Unraid 6.9.2

 

Best,

Johannes

 

tower-diagnostics-20220424-0750.zip

[JorgeB]

Btrfs is detecting data corruption, first thing you should do is to run memtest.

[JoJ123]

Could you tell me how to run that? Do I have to stop the array first?

[JorgeB]

It an Unraid boot option:

 

It only works with CSM boot, not UEFI

[JoJ123]

Mmh my UnRaid server is without a DIsplay sitting in the cellar

Do I have any other way / solution to go further?

 

Update:

I've taken the server to my desk, however it seems I'm running it in UEFI Mode, I was not able to change that in the BIOS. So whenever I hit the memtest, it's saying:

Loading /memtest ... ok

 

But then nothing will come up even after 2-3 Min waiting.

Edited April 24, 2022 by JoJ123

[JorgeB]

Trying to run memtest when booting UEFI generally just restarts the server, but you can try memtest86, it's UEFI only.

[JoJ123]

I read only a lot of issues with BTRFS, I think the best would be to give up the cache pool and use a single Cache Disk with XFS.

I'm just scared of my MariaDB and PostreSQL Database Data. What do you think about storing the database data on the HDD array?

[Michael_P]

If bad memory is corrupting your data, changing file systems won't stop the corruption - it'll just stop telling you about it.

[JoJ123]

It's two brand new SSDs

[Michael_P]

1 hour ago, JoJ123 said:

It's two brand new SSDs

If the system is sending corrupted data, the age of the SSDs is irrelevant  

[JoJ123]

Okay, so what I did so far now is restoring everything to one XFS Cache Disk and the server is back running well.

What analysis I could do, to check which piece or container is sending corrupted data?