Drive 2 of BTRFS Raid 0 never spins up

[Marshalleq]

Yeah, so as per title, I just realised this drive never spins up.  I checked the power on hours and it says it's only been powered on for 5 minutes, yet it has been in the cache pool running for quite some time.  You can see below it's part of the cache pool and while writing it's just spun down and greyed out.  I'm not an expert with btrfs, do I need to do some kind of rebalance to get it to work?

 

[JorgeB]

That device is likely not part of the pool, despite what the GUI shows, would need diags to confirm.

[Marshalleq]

I just noticed I got the title wrong too - supposed to be RAID1 lol.  

 

**Make sure you read Jonnie Blacks comment after this post incase it works for you (quicker)**.

 

So for the benefit of others I did a quick google and sorted it out in the command line.  Someone else who's familiar with btrfs can critique my console fu if they like:

 

# btrfs filesystem show

Label: none  uuid: 0dfcf92d-3b3b-4328-b264-0ed8641019f7

Total devices 1 FS bytes used 133.64GiB

devid    1 size 465.76GiB used 139.02GiB path /dev/sdd1

 

Label: none  uuid: a3cc2ec9-de47-4142-92ac-a9c8905ad4d0

Total devices 1 FS bytes used 3.75GiB

devid    1 size 50.00GiB used 4.52GiB path /dev/loop2

 

Label: none  uuid: 37f59ca5-8f97-4193-807c-82e4f9e81f31

Total devices 1 FS bytes used 128.00KiB

devid    1 size 465.76GiB used 20.00MiB path /dev/sdb1

 

So this shows that there are three btrfs devices on my system, though one is a loopback which I think is something to do with metadata.

 

# btrfs filesystem show /mnt/cache

Label: none  uuid: 0dfcf92d-3b3b-4328-b264-0ed8641019f7

Total devices 1 FS bytes used 145.42GiB

devid    1 size 465.76GiB used 153.02GiB path /dev/sdd1

 

This shows the cache pool does only have one device assigned to it, despite what the Unraid GUI thought (bug?)

 

#mount /dev/sdb1 /mnt/test

# ls /mnt/test

(There was no data)

#umount /mnt/test

#mount /dev/sdd1 /mnt/test

# ls /mnt/test

(There was the expected data)

So this proved my second device that was meant to be in the pool had nothing on it.

#mount

/dev/sdd1 on /mnt/cache type btrfs (rw,noatime,nodiratime)

(Yep double confirming it's the btrfs device)

 

# btrfs device add /dev/sdb /mnt/cache -f

Performing full device TRIM /dev/sdb (465.76GiB) ...

(-f forces the sdb device to be added to the pool despite it having a file system on it - be very careful as if you get this wrong you will lose your data, but that's OK for me, because have already confirmed it above)

 

# btrfs filesystem balance /mnt/cache

 

WARNING:

 

Full balance without filters requested. This operation is very

intense and takes potentially very long. It is recommended to

use the balance filters to narrow down the scope of balance.

Use 'btrfs balance start --full-balance' option to skip this

warning. The operation will start in 10 seconds.

Use Ctrl-C to stop it.

10 9 8 7 6 5 4 3 2 1

Starting balance without any filters.

Done, had to relocate 177 out of 177 chunks

 

(This is needed to turn it into a true mirror, according to the docs it will rebalance (stripe) all the extents across all the disks now in the pool.  I'm not sure about the warning though, and I always worry about doing these write intensive workloads across SSD's, especially low quality ones like these Samsungs that aren't made for heavy writes.  Someone else can probably chip in here.  The whole process on these SSD's took about 5 minutes so not too bad).

 

# btrfs filesystem show /mnt/cache

Label: none  uuid: 0dfcf92d-3b3b-4328-b264-0ed8641019f7

Total devices 2 FS bytes used 118.06GiB

devid    1 size 465.76GiB used 54.03GiB path /dev/sdd1

devid    2 size 465.76GiB used 66.00GiB path /dev/sdb

 

(I then realised I had to convert to Raid-1 (using the command below) because btrfs defaults to Raid-0 which is not what I want.  That would explain the differing sizes above.  And you can see this in the first screenshot below (it shows that it's made a 1TB drive out of two 500G drives rather than a 500G Raid-1 Mirror.  So I needn't have run the first balance command.  I will be impressed if it actually CAN convert a bigger stripe to a mirror live and on the fly).

 

# btrfs balance start -dconvert=raid1 -mconvert=raid1 /mnt/cache

Done, had to relocate 121 out of 121 chunks

 

Yep, it converted it, that's impressive.  And a screenshot showing it's now 500G

 

 

Well I'm right (write?) impressed!   A whole lot easier than mdadm though SSD's obviously make it a whole lot quicker - so that could be swaying me.  If his is what we're in for in the future with the departure of spinning disk, I'm excited.

 

And all done online without taking down / unmounting the drives.  Just wow.

 

Thanks for reading!

Edited June 24, 2019 by Marshalleq
Clarification

[JorgeB]

You just needed to stop array, unassign cache2, start array, stop array, reassign cache2 and you'd be done.

[Marshalleq]

Yeah, that's what I did when I first set this up.  Or at least what I think I did.  I remember being a bit annoyed that it didn't look right, but never coming back to it until now, because I realised it really wasn't working.  I don't think via the GUI it would be too easy to get wrong either, yet somehow it was.  A good education for me and I prefer command line anyway.  I'll edit my post to say, make sure you read Jonnie Blacks comment after this post incase it works for you (quicker).

[Marshalleq]

After a downgrade to 6.6.7 to isolate some issues, then an upgrade to 6.7.2 a lot of my configuration was reset.  One of those things is the cache and I again find myself in the situation where it's not set up correctly.  In this instance I have stopped the array to add the cache drive and reassign disk 2 etc as per @johnnie.black above, but unfortunately it's set it up as a RAID 0 for some reason.  I'm not sure why, but just noting that it seems I again need to run through the steps above, because I don't wish to format the whole cache again, which may not get different results anyway.

[JorgeB]

5 hours ago, Marshalleq said:

but unfortunately it's set it up as a RAID 0 for some reason.

Cache pool would never be configured as raid0 by Unraid, only user can do that, Unraid will only use single and raid1 profiles.

[Marshalleq]

Well you say never, but twice it did it.  The first time you could argue I did it via the console by not specifying a Raid level, but the second time, it did it all on its own.  Adding empty SSD's to the cache pool using the unraid GUI, is all I've ever done before, so far this has been unsuccessful.  I honestly don't trust it.  At some point I'll pull out one of the cables - I suspect from this experience that the mirror, doesn't work when you've got encryption enabled but could be wrong.

[JorgeB]

On 8/17/2019 at 9:39 PM, Marshalleq said:

it did it all on its own

Not possible, Unraid will never convert a pool to raid0 profile, it will keep using it if the user first convert the pool to raid0.

[Marshalleq]

OK I believe you.  It's just that it did it without my approval.  So it seems to me there's something else going on.  I think it is because one of the drives was kicked out of the mirror.  I distinctly remember when starting the array the remaining drive rebalanced as I couldn't do anything until it finished.  Once I added the second drive in, it rebalanced again and that's when it went to a stripe.  That's when it became Raid 0 and I had to manually convert it to Raid 1.  So I'd say there's some glitch around when it had only 1 drive.  I definitely did not convert the pool to raid 0.