Hi,

 

I had what I thought was a PCI bus die and drop a sata controller with one drive attached last week, and after futzing with the server for a few hours, I discovered that Unraid would only boot in GUI modes. No CLI modes will boot, and instead go on an endless reboot cycle.

 

Something's better than nothing. I rebooted in GUI mode, was thrilled to see that the SATA controller was recognized again, so I let the server rebuild the drive. That finished with no errors, so I'm back to an intact array.

 

BUT....After finishing, I started getting the error:

Unable to write to cache -> Drive mounted read-only or completely full. The main page shows my cache pool (2 512 GB SSDs) has 12.8 GB used/498 GB free. 

 

Also, all dockers are unresponsive. Ack.

 

I've seen a few threads about resolution on Reddit, but I want to be sure I'm asking here to get the most up-to-date information.

 

I suspect my Flash Drive needs replacing (hoping that will resolve boot issues), but the cache pool issue is a bit above my head.

 

Any guidance? 

 

 

 

Thanks for any help.

 

VS 

 

bender-diagnostics-20210322-1620.zip

Pool filesystem is corrupt, best bet is to backup and re-format it.

Thanks Jorge. Makes sense.

 

Do you know of a guide or any set of instructions for backing up and reformatting the cache pool? I've found a guide for doing so with a single cache drive, but nothing for a cache pool.

 

Thanks again.

-v 

This is a way, but basically you just need to copy everything important to the array or another disk before formatting.

Very helpful, Jorge. Many thanks. I'll get this worked out tonight and report back.

 

Thank you again.

A followup question:

Rsync appears to have made a backup of cache pool to my backup location, but it had "chown" failures the entire time - nearly every file. 

 

Is that an issue? I suspect it will cause grief on the next rsync operation, but I'm not positive.

 

Thanks!

 

Edited March 23, 20215 yr by Vonslappy
Added screen grab

35 minutes ago, Vonslappy said:

but it had "chown" failures the entire time - nearly every file. 

That suggest the destination filesystem doesn't support permissions, e.g. exFAT

Aha. OK. I'll do a little reading, and reformat in the proper filesystem, and retry. 

 

Thank you again -- really appreciate your helping me fumble through this.

Maybe the last question. I hope the last question.

 

Backup complete. No errors.

 

Stopped the array and started to change format on the Cache pool. Seems I can only change the change format to "Auto," or "BTRFS - Encrypted." Setting things to "Encrypted" naturally doesn't mount the disk because it isn't encrypted. "Auto" just spins the disk back up in BTRFS.

 

I do, however, have a giant "erase" button. Might that accomplish the same thing?

Edited March 23, 20215 yr by Vonslappy
Correcting my poor writing

Check that. I'm an idiot. Found the format button.

 

Ignore the above, please.

Up and running again. Thank you @JorgeB

Crud. It's back.

 

"Unable to write to cacheDrive mounted read-only or completely full."

 

Is it time for a scrub?

If you formatted cache and it's already corrupt it suggest an underlying hardware issue, post new diags just to see the actual issue.

I would agree. Thanks, Jorge. I'd have done a lot of damage trying to resolve this without your assistance. 

 

New diags posted. 

bender-diagnostics-20210326-1156.zip

Logs are spammed with these:

 

Mar 26 08:19:40 Bender kernel: DMAR: [DMA Read] Request device [00:02.0] PASID ffffffff fault addr 0 [fault reason 06] PTE Read access is not set
Mar 26 08:19:40 Bender kernel: DMAR: DRHD: handling fault status reg 2
Mar 26 08:19:40 Bender kernel: DMAR: [DMA Read] Request device [00:02.0] PASID ffffffff fault addr 0 [fault reason 06] PTE Read access is not set
Mar 26 08:19:40 Bender kernel: DMAR: DRHD: handling fault status reg 2
Mar 26 08:19:40 Bender kernel: DMAR: [DMA Read] Request device [00:02.0] PASID ffffffff fault addr 0 [fault reason 06] PTE Read access is not set
Mar 26 08:19:40 Bender kernel: DMAR: DRHD: handling fault status reg 2

 

Can't see anything else, please reboot and post new diags after array start.

New diags posted here. 

 

Update: After rebooting the error seems to have cleared. Naturally, I don't trust this to fix itself, but it's interesting. Also, flash drive has now been replaced, if that matters.

 

Thanks.

 

 

bender-diagnostics-20210327-0742.zip

Edited March 27, 20215 yr by Vonslappy

On 3/27/2021 at 7:44 AM, JorgeB said:

new diags after array start.

DIags are before starting the  array, you can post new ones or wait for more issues and post then.

Shoot. Got it. New diags posted here, taken after starting array.

Thanks for investing the time in this.

 

V

 

bender-diagnostics-20210328-0958.zip

Cache filesystem looks fine so far, but the docker image is corrupt, delete and recreate.

Also your system share has some files on disk1 possibly they are duplicates. See if you can get rid of those while docker is disabled.

Thanks, again, Jorge. Everything seems to be running better now. 

12 minutes ago, trurl said:

Also your system share has some files on disk1 possibly they are duplicates. See if you can get rid of those while docker is disabled.

Aw crap. Just saw this. If I re-disable docker, and root around, will a re-enable bring things back to order?

 

If not, it seems fairly trivial to delete and recreate the docker.img, so I can do that as well.

 

Thank you.

If you recreated docker.img it will be on cache and you can just delete the one on disk1.