BTRS FS Failure

[BrandonG777]

One of my BTRS cache drives seems to have developed a problem over night. Since this drive is part of a mirrored array, can I just format it? (and rebuild it, I assume this would happen automatically?)

[JorgeB]

47 minutes ago, BrandonG777 said:

can I just format it? (and rebuild it, I assume this would happen automatically?)

No, please post your diagnostics: Tools -> Diagnostics

[BrandonG777]

veyron-diagnostics-20181019-0715.zip

[JorgeB]

Need the diagnostics after array start.

[BrandonG777]

 

incorrect offsets 8870 424057239 ERROR: cannot open file system

veyron-diagnostics-20181019-0808.zip

[JorgeB]

Pool filesystem is corrupt, see here to try and recover your data before re-formatting:

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=543490

 

[BrandonG777]

I'm hung up on even getting the FS mounted. Following those instructions it wouldn't mount anything to /x so I went with /mnt/x

 

But I get this trying to mount the first drive /dev/sdc1...

mount: /mnt/x: wrong fs type, bad option, bad superblock on /dev/sdc1, missing codepage or helper program, or other error.

 

 

This should be a mirrored array so I'm assuming that's why I can't mount it so I'm really lost on how to recovery from this.

 

EDIT: Reading further through what you sent I'm realizing the partitions are too corrupted to mount. What would cause something like this? The drives appear healthy.

Edited October 19, 2018 by BrandonG777

[BrandonG777]

Recovery in progress... but I'm still really concerned what would have caused this to begin with.

[JorgeB]

Would need the syslog from when the issue happened and before rebooting.

[BrandonG777]

So now that I've recovered what I can. Format the drives and restore data? Although it looks like I've lost a lot of important stuff due to "looping" Should be in Amazon Glacier but my Cloudberry database was another item that was lost so I'm curious to see how all that plays out.

[JorgeB]

20 minutes ago, BrandonG777 said:

So now that I've recovered what I can. Format the drives and restore data?

You can try the last option btrfs check --repair, nothing to lose now, but even if it works recover the rest of your data and reformat to make sure the filesystem starts clean.

[BrandonG777]

Now it's saying my /var/log is full. I've deleted stuff but it didn't make any space.

 

root@Veyron:/var/log# du -h
0       ./swtpm/libvirt/qemu
0       ./swtpm/libvirt
0       ./swtpm
0       ./samba/cores/winbindd
0       ./samba/cores/smbd
0       ./samba/cores/nmbd
0       ./samba/cores
0       ./samba
0       ./plugins
0       ./pkgtools/removed_scripts
0       ./pkgtools/removed_packages
0       ./pkgtools
0       ./nginx
0       ./nfsd
0       ./libvirt/qemu
4.0K    ./libvirt
108K    .

I don't understand where it's going.

[BrandonG777]

veyron-diagnostics-20181019-1445.zip

[JorgeB]

Syslog is 0 bytes, so nothing there, reboot and monitor syslog, if it keeps growing grab diags before it gets too big.