Removing disk from Array, adding a new one and more

[trurl]

6 hours ago, piratx said:

I've increased docker.img cause it was giving me status reports that it's going to be full from time to time.. Something like this:

eleven: Alert [ELEVEN] - Docker image disk utilization of 92%
Docker utilization of image file /mnt/user/system/docker/docker.img

docker.img usage shouldn't grow. It's possible that you need more than 20G. I run 15-20 dockers and only use about half of the 20G I have configured. But even if you need more, usage shouldn't grow. The usual cause of filling docker.img is an application configured to write to a path that doesn't correspond to a container path, case-sensitive.

 

6 hours ago, piratx said:

move my appdata

You want to move your appdata and system shares to fast storage. The system share is where docker.img is, and that contains the executables of your containers, so it needs to be on fast storage also. Any maybe some of domains needs to be on fast storage as well, you can work that out as you see how you are really going to use VMs.

 

Nothing can move open files. You will have to go to Settings - Docker and disable docker. Same for Settings - VM Manager. And leave them disabled until you get your disk assignment changes completed, and the shares moved.

 

You have CA Backup plugin installed. Good idea to have a current backup of appdata and libvirt.img before making any changes.

 

New Config without that nvme and with your spinners assigned however you want. Make very sure you don't assign any data disk to any parity slot or it will be overwritten with parity. Then let it rebuild parity. You may need to take care of any include/exclude settings for each of your user shares so they agree with your assignment changes. Probably don't really want to use your server much until parity rebuild completes.

 

That's enough to get you started for most of a day and we can then deal with getting a new pool created for your system and appdata.

[piratx]

Ok, I'll try to start with the disk addition in a hour or something. 

Backuping up appdata to my array is fine right?

 

1 hour ago, trurl said:

Make very sure you don't assign any data disk to any parity slot or it will be overwritten with parity.

 

That would make things much complicated  Thanks for mentioning it!

[trurl]

12 minutes ago, piratx said:

start with the disk addition

Any disk added during new config will be accepted exactly as it is. If it has an Unraid filesystem on it then whatever data is there will become part of the array and user shares. If it doesn't have an Unraid filesystem it will be unmountable. In either case the disk can be formatted after or even during parity rebuild. Just make sure you don't agree to format anything if there are unmountable disks that are supposed to have data on them.

[piratx]

 Yes, its an empty disk. No need to worry about it. I let you know if I cannot add it on slot "8" which is missing from the list.. or I shouldn't care New config will do what should be done right?

[trurl]

1 hour ago, piratx said:

its an empty disk

I usually reserve the word "empty" for a disk which has a known filesystem with no contents. An "empty" disk in that sense is mountable (if it is an Unraid filesystem) and ready to receive new folders and files. A disk that is unmountable for Unraid array or pools must have its filesystem repaired if it is supposed to have contents, or it must be formatted by Unraid after it is added to array or pool so it becomes "empty".

 

Another useful distinction to be made is a "clear" disk. A clear disk has all of its bits set to zero. Unraid requires a clear disk when adding to a new data slot in an array with valid parity, since all zeros has no effect on parity. If you add a disk to a new data slot in an array with valid parity, Unraid will clear the disk unless it has been precleared.

 

Sometimes people will preclear a disk then format it before adding it to a new data slot in an array with valid parity. A formatted disk is "empty", but it is not "clear", so Unraid will clear it again and then let you format it in the array.

 

Many seem to have a very vague idea of what "format" does. They seem to think it just means "get a disk ready to use", but computers don't work on vague ideas.

 

Format means "write an empty filesystem to this disk". That is what it has always meant in every operating system you have ever used.

 

Another common mistake is agreeing to format an unmountable array disk that is supposed to have contents with the hope of rebuilding it from parity. Format is a write operation, and Unraid treats that write operation like any other, by updating parity. So after formatting an unmountable disk in the array, the only thing that parity can rebuild to the disk is an empty disk.

 

1 hour ago, trurl said:

make sure you don't agree to format anything if there are unmountable disks that are supposed to have data on them.

 

[piratx]

Format was always that in my mind.. or create an empty table to put data on cells.

Like I said though, the disk is an 8TB external which have some files, but will find/see it like all the others. 

 

Others had files in and I got them on unassigned devices and others I just chose them to be added on the array on start.. unRAID found them unused and it just "asked" me what I would like to do with those.

 

Still backing up.. 

 

root@eleven:/mnt/user/backup/unraid/appdata# du -h
47G    ./[email protected]
65G    ./[email protected]

 

I don't know if I need to clean this up a bit previous backup was 65G and the backup is slow  

 

Edit: Okay done just 47G this time.

Edited November 17, 2021 by piratx

[trurl]

2 hours ago, piratx said:

disk is an 8TB external

Are you planning to shuck that drive? USB connections NOT recommended for array or pool disks for many reasons.

[piratx]

Oh sorry. Yes i am opening the box to get the disk. I have many installed like that

[piratx]

It took me long enough to add that disk.. I am full

Edited November 18, 2021 by piratx

[piratx]

Ok. Parity is done without any errors. Nvme is now on unassigned list. 
 

Should I run new config option now?

[trurl]

13 minutes ago, piratx said:

Ok. Parity is done without any errors. Nvme is now on unassigned list. 
 

Should I run new config option now?

I don't understand. Didn't you already do New Config to get the nvme removed from the array?

[piratx]

No I let it finish with the Parity check, I removed the NVMe, and added the new 8TB in Disk 11 slot

Should I preserve current assignments when I use New Config? Thanks again for all the help!

 

 

[trurl]

8 hours ago, piratx said:

I let it finish with the Parity check, I removed the NVMe, and added the new 8TB in Disk 11 slot

  

Did you start the array after replacing disk11? If so then it will be rebuilding disk11 to the replacement.

 

I had some other instructions for New Config, but will wait on your answer to my question.

[piratx]

Yes, I have started the array and all is ok. I wanted to backup something so I needed to do so.

I am attaching my diags once again for a better review.

eleven-diagnostics-20211119-1627.zip

[trurl]

Things aren't making sense just based on what you have said.

 

The nvme was assigned as disk11. You replaced that with another disk and started the array. But you say you haven't done New Config to make that change, and according to your diagnostics disk11 isn't being rebuilt. Replacement disk11 is mounted and empty.

 

You are going to have to explain in more detail how you got to this point.

 

 

[piratx]

OK, I had issues when I installed the new 8tb disk, something with the hardware.. one time it restarted with the 8tb (new disk) and the 4tb (the crc error) disk and only needed to add the 8tb to a new slot in the array and when I did added the 8tb (new disk) I got an X into the slot the 4tb (crc error disk) was.

So I changed the sata cables and tried again.. finally after 3-4 restart I added a splitter that sata cable and I saw the 4tb and the 8tb disk and all well.

I've added the 8tb disk into the nvme disk place and let the nvme to unassigned disks.

 

That's all I did. In short I have hardware (cable issues) but the story is the same I only moved the nvme out of the area and added the new 8tb to that slot.

I've let the parity run and haven't touched anything else since then (a day ago)

 

Thanks for helping again! 

[trurl]

Just now, piratx said:

I've let the parity run

 

I see. What you meant by that statement is that you let it rebuild disk11.

 

Nov 18 02:51:49 eleven kernel: md: recovery thread: recon D11 ...

 

And it's not empty as I said, just mostly empty because original was small.

 

[piratx]

Original?

[trurl]

1 minute ago, piratx said:

Original?

Original disk11, the nvme

[piratx]

Ah ok, so the job is done. I need to run New config now?

[trurl]

9 hours ago, piratx said:

preserve current assignments when I use New Config?

That is the usual way. It will let you change any assignments before starting the array. Since you only have single parity, and that parity is currently in sync with all of the disks in the array, and you aren't adding or removing any data disks, then you can reorder the data disks as you want without rebuilding parity.

[piratx]

Stopped the Array and I am here now

[piratx]

I should do something like this?

 

set no device to Disk 11

and set the Disk 11 (8tb new) to Disk 8.

Edited November 19, 2021 by piratx

[trurl]

Yes. Since you don't need to rebuild parity, before starting the array, check the box saying parity is already valid.

 

[trurl]

On 11/17/2021 at 8:40 AM, trurl said:

You may need to take care of any include/exclude settings for each of your user shares so they agree with your assignment changes.