My cache keeps going into read only - scrub aborts and balance does nothing

[jayhay548]

Hi all,

 

My dockers are saying docker service failed to start. I think my cache drive is dying and I'll replace it as soon as I can move everything off it.

 

When i look in maintenance mode it says --readonly on the cache drive and fix common problems says "Drive mounted read-only or completely full."

 

I am not an idiot but I have little to no experience in command prompt style efforts - so be gentle.

 

Thanks for the help.

[JorgeB]

Please post the diagnostics: Tools -> Diagnostics

[jayhay548]

Here?:

 

Thanks!

tower-diagnostics-20210204-1934.zip

[JorgeB]

ATA errors on the SSD:

 

Feb  4 17:44:44 Tower kernel: sd 1:0:0:0: [sdb] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
Feb  4 17:44:44 Tower kernel: sd 1:0:0:0: [sdb] tag#0 Sense Key : 0x5 [current]
Feb  4 17:44:44 Tower kernel: sd 1:0:0:0: [sdb] tag#0 ASC=0x21 ASCQ=0x4
Feb  4 17:44:44 Tower kernel: sd 1:0:0:0: [sdb] tag#0 CDB: opcode=0x2a 2a 00 00 00 cd 20 00 01 a0 00
Feb  4 17:44:44 Tower kernel: print_req_error: I/O error, dev sdb, sector 52512
Feb  4 17:44:44 Tower kernel: BTRFS error (device sdb1): bdev /dev/sdb1 errs: wr 1, rd 0, flush 0, corrupt 0, gen 0

 

SATA controller is set to IDE, try again after changing it to AHCI.

[jayhay548]

Thanks very much - I'll try later.

 

Presumably I'd do this in the BIOS? I run headless so I'd need to get a keyboard and screen.

 

Odd that this setup has been working for 3yrs and now it stops - has something changed?

 

Thank you again.

[JorgeB]

9 minutes ago, jayhay548 said:

Presumably I'd do this in the BIOS?

Yep.

 

9 minutes ago, jayhay548 said:

Odd that this setup has been working for 3yrs and now it stops -

There could be other issues, but the controller should still be set to AHCI.

[jayhay548]

changed the bios. got this error message:

 

Docker still not showing and mover not running.

 

Sorry to be so useless.

 

James

tower-diagnostics-20210205-1932.zip

[JorgeB]

Still constant ATA errors, but now easier to see which device is the problem, replace cables on the cache SSD, it can also be an SSD problem, but stat with the cables.

[jayhay548]

on it now. thanks!

[jayhay548]

Cables replaced. Unchanged sadly

 

[JorgeB]

If it's still logging constant ATA errors then it's likely a problem with the SSD, try another if available.

[jayhay548]

Ok - I don't have another - will order one - what will happen to the 137GB sitting in the cache?

 

Thanks for the ongoing help

[jayhay548]

just had a look in the cache it seems all my appdata is in there! plus a few huge movies that i won't mind if i lose them. interestingly in the mover error - it is showing errors in moving one of the movie files.

[jayhay548]

Ok I have got another ssd and added it as a second cache - still nothing is working. I thought it would mirror cache 1 and then the mover would run but it hasn't worked.

 

Sorry - can anyone help??

 

[trurl]

You should always post new diagnostics when you have made changes since the last diagnostics you posted if you still need help.

[jayhay548]

sorry thanks!

tower-diagnostics-20210207-1727.zip

[jayhay548]

So I've replaced the cache drive and fix common problems is giving me this:

 

[jayhay548]

Anyone?

[JorgeB]

Still constant ATA errors, it won't work correctly while those are going on.

On 2/5/2021 at 7:56 PM, JorgeB said:

if it's still logging constant ATA errors then it's likely a problem with the SSD, try another if available.

 

[jayhay548]

So - am I now looking at a complete rebuild?

 

[jayhay548]

I built the system 4 years ago using spare parts. Is it possible to simply move all the drives over to a new motherboard?

 

Thanks!

 

James

[JorgeB]

8 minutes ago, jayhay548 said:

. Is it possible to simply move all the drives over to a new motherboard?

Yes, and if the issues with the old cache device persist basically confirms that the SSD is the problem. 

[jayhay548]

I've already replaced the ssd

[JorgeB]

I understood you added an SSD to the pool, is that not correct?

[jayhay548]

I added one to the pool - that didn't work. So I then removed the old cache entirely and am now running on the new sdd via a new cable via a diferrent SATA port.