I have two cache drives in a pool. i thought that I could easy take one to go do another project with it and then add it back to the pool and keep trucking. Before I removed the cache drive from the pool, I moved all cache data to the array. Once it appeared that I was safe, I removed the cache drive. With one out, I started up the array to run on one cache drive only. Things seemed to be going fine. At some point after finishing my project, I noticed that my operating cache drive was giving an error and it was “Wrong”. What did I do and how can I recover the data on the single cache drive that I was using? The only option that I see is to format it to get it back to a working cache pool! There goes all my cache data…

 

I would appreciate your help much!

fatfarmhqserver-diagnostics-20230907-1639.zip

Post the output of:

btrfs fi show

 

Here's the output:

Quote

Label: none  uuid: 584a2486-c3bc-457c-9d4a-a61d01391679
        Total devices 1 FS bytes used 3.71GiB
        devid    1 size 20.00GiB used 8.02GiB path /dev/loop2

Label: none  uuid: 8656c3a9-8d89-49f0-93e7-b4335ca29bfd
        Total devices 1 FS bytes used 1.40MiB
        devid    1 size 1.00GiB used 126.38MiB path /dev/loop3

 

I assume the pool was btrfs? There's no btrfs filesystem on the SSDs.

Yes it was. How could it be gone? I’m not sure what I did. 

Post the output of 

fdisk -l /dev/sdd

and

fdisk -l /dev/sde

 

root@FatFarmHQServer:~# fdisk -l /dev/sdd
Disk /dev/sdd: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: CT1000MX500SSD1 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
root@FatFarmHQServer:~# fdisk -l /dev/sde
Disk /dev/sde: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: CT1000MX500SSD1 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
root@FatFarmHQServer:~# 

 

Edited September 8, 20232 yr by danbru1989

No partition exists on either device, suggesting they were wiped, depending on how it was done it may be recoverable, type

sfdisk /dev/sdd

then type

2048

and hit return, finally post a screenshot of the results.

Hit N + return to keep the signature, then type

write

and return again, then

btrfs fi show

 

I’m not familiar with everything you are instructing, but it looks promising to me. Should I try mounting sdd1 again in the cache pool?

First lets see if you can also recover the other device, in case there's an issue with that member, do:

sfdisk /dev/sde

and assuming it finds a btrfs signature follow the same steps above, then post once more

btrfs fi show

 

Here's the results. Thank you for your persistent help with this.

 

The 361gb on sdd1 is my previous cache. I think that the 2.02gb on sde1 is from a 2nd cache pool I started on that drive only during my troubleshooting and testing. If that's the case, that data is not important to save. I think that it might be best to just get sdd1 up and running the cache pool reliably, then I can just format sde1 and add it to the cache pool.

 

 

Edited September 8, 20232 yr by danbru1989
Added further info.

OK, I though both devices were members of the same pool, then make sure cache is reset by starting and stopping the array without any device assigned to cache, then assign just sdd to the first slot of a pool with two slots and start the array, then post new diags, this will only work if the pool was fully redundant.

I did as you suggested and sdd1 mounted successfully and started doing a bunch of read/writes.

fatfarmhqserver-diagnostics-20230908-1134.zip

That's a good sign, it's balancing the pool to single device, but everything looks normal, you can then add another device when that finishes if you want.

I can't thank you enough for your help! No way that I could have figured that out on my own!

When I attempt to start the array with the sde1 added to the cache pool, I get the "too many missing/wrong devices" error. Any thoughts?

Edited September 8, 20232 yr by danbru1989

Probably need to wipe wipe it first, if you're sure there's no important data there, type:

blkdiscard -f /dev/sde

 

I don't think you need to reset the pool again, but if you do, unassign sde, leave only sdd, start array, stop array, re-assign sde (after wiping it) and start array.

I did the blkdiscard -f /dev/sde and it didn't fix the error. Additionally, it's giving me the error when only sdd1 is assigned... 🤔

Reset the pool once more, unassign all pool devices, start array, stop array, re-assign only sdd, start array, post new diags to config the balance finished.

I caught what I missed. Forgot to create a new pool when starting / stopping. Looks like it's all working now. It is now balancing the pool with both drives in cache. Thanks!

Edited September 8, 20232 yr by danbru1989