Hello,

 

I've currently got the following configuration:

2x 1TB NVME in a BTRFS Pool

 

I want to do the following, and i would love to hear some guidance if im doing this correct, and if its supported:

1. Add 1 2TB NVME and make a ZFS Pool

2. Transfer all data to the new ZFS Pool

3. Remove the 2 1TB nvme's

4. Add 2 new 2TB NVMe's

5. Add the 2 NVME's to the new ZFS pool

 

Doing so will it be possible to add the 2 and convert the pool to a ZFSZ1 pool? (1 for protection) and grant me 4TB of usable space and 2TB of protection?

 

 

2nd Question: What if i have mixed sizes? Can ZFS handle that?

Edited January 15, 20251 yr by Mirano
Running Stable now

review:
 

I don't see a problem with your setup.

It is recommended to migrate your data off and setup the zfs disk setup you want first and then bring the data back to the new setup.

Configuration Goals:

Migrate data from an existing BTRFS pool to a new ZFS pool.

Create a ZFS pool with redundancy (ZFSZ1).

Handle the potential inclusion of mixed-sized NVMe drives.

 

Additional Q/A:

1. Can ZFS Handle Mixed Sizes?

Yes, ZFS can handle mixed drive sizes. However, the pool's usable capacity is determined by the smallest drive in a RAID-Z configuration:

In your scenario, if you mix 2TB and 1TB drives, the usable space of the larger drives will be capped to match the size of the smallest drive in the RAID-Z group.

 

2. General Recommendations:

Drive Size Matching: To maximize storage efficiency, use drives of the same size.

Plan Ahead for RAID Level: If you anticipate needing more storage or redundancy in the future, consider the implications of RAID-Z levels (e.g., RAID-Z2 for dual-disk redundancy).

Test Configuration First: Since this is on a beta system, it’s wise to test the setup without critical data to ensure there are no issues.

 

Suggested Commands/WEB UI Summary:

Create ZFS Pool: zpool create zfs_pool /dev/nvmeXn1

Transfer Data: rsync -av /mnt/btrfs_pool/ /mnt/zfs_pool/

Add Drives to ZFS Pool: zpool add zfs_pool /dev/nvmeYn1 /dev/nvmeZn1

Convert to RAID-Z1: zpool attach zfs_pool /dev/nvmeXn1 /dev/nvmeYn1 /dev/nvmeZn1

Feel free to follow up for any additional clarification or troubleshooting!

 

when making zfs you need to set the # of disk and type before hand. You will not be able to make a single disk zfs (would be cosnidered a degraded mirror) and eevn then to fix that pool it would only be able to be 2 disk.

to setup the zfs with raidz1 you need 3 disks to start.

53 minutes ago, bmartino1 said:

when making zfs you need to set the # of disk and type before hand. You will not be able to make a single disk zfs (would be cosnidered a degraded mirror) and eevn then to fix that pool it would only be able to be 2 disk.

to setup the zfs with raidz1 you need 3 disks to start.

I believe you can make a single disk ZFS drive and turn it into a mirror later,  but not a mirror to a raidz config, at least not yet.

 

For OP, it'd be easier to move all of the data off to another disk like an external disk, put all of the new NVMEs in a new pool with the config you want (raidz1 or z2, etc) and then move the data back. 1TB isn't too much data to move around. 

 

 

Even in zfs circle it not recommend making a degradation zpool at start due to raid parity it does for initial resilver it be better to make the mirror pool with 2 disk and pull a disk to get the degrade 1 disk in a pool.

Correct sorry if i'm not being clear enough once you set the pool to be a mirror / raidz it is not possible to change zfs to a raidz1 with #vdev or form mirror to raidz# ... this all has to be done beforehand.

 

I'm not aware of any plans for 1 disk zfs > 3 disk raidz due to resilver and the nature of how disk, compression and encryption play a factor...

 

https://superuser.com/questions/913551/zfs-upgrade-2-mirrored-drives-to-3-drive-raidz

Quote

ZFS supports adding drives to pools without upsetting the original data but you cannot change the configuration (mirror, RAIDZ, etc.) of existing drives in the pool.

 

Edited November 25, 20241 yr by bmartino1
typo

first and foremost thank you both @bmartino1 and @foo_fighter for the responses!

 

Alright i have the following planned (Please correct me if i should not be doing a step)

 

For a complete reset i want to do the following:

1. Add a External SSD USB to the server and move all the data towards that (With mover)
2. Completely nuke the USB and start from scratch (Been experimenting for over 5 years on this single USB config)
3. Add 3x 1TB NVME in a RaidZ1 and let it sync
4. Restore my files from the External SSD USB
5. Restore my plugins that i've collected over the time
6. Restore my Dockers using the backup&restore plugin
7. Restore my VM's
8. Grab a beer and relax

 

Is there anything you guys would do differently?

 

I don’t see a problem, just to be clear. Unraid 7 doesn’t have a direct download since it’s in beta. Download the latest stable version and upgrade through the web UI.

Currently, v7 beta 2 is less buggy than some of the earlier versions, but v7 beta 4 has all the feature sets along with some bug patches. If you choose to go beta, I’d recommend starting with beta 2 for setup and recovery, then upgrading to beta 4.

 

 

1. Backup/Copy Data:
Step 1 doesn’t have to involve using Mover; it can be any method to copy/backup the existing data, such as rsync, mv/cp commands, mc, or krusader. The key is ensuring the files are accessible on the specified disk for restoration.

 

2. Flash Backup:
Please make a flash backup:

Web UI > Main > Flash

This is particularly important if you plan to reuse the same USB. Make sure to copy the config folder and the license key file into the new file to maintain your Unraid license.

 

3. RAID Setup:
Set up the RAIDZ1 configuration beforehand for the best results.

 

4. Unassigned Disk Plugin:
The Unassigned Disk plugin is useful here. Depending on your license, you may need to monitor the total number of disks attached when restoring files. As with Step 1, use whatever method works best to copy files from your backup into the array.

 

5. Restore Configurations:
Using the flash backup (e.g., old Docker settings), you can easily copy necessary files into the config folder or download fresh apps from the CA Store.

 

6. Appdata Backup Plugin:
The Appdata Backup plugin is a great tool for restoring Docker containers if you don’t copy the folder from the flash backup. However, ensure the plugin is correctly configured to capture a full backup. Some setups may not be fully backed up, which is why Step 1 is critical to ensure all data is accounted for.

 

7. VMs:
VMs can be tricky due to their large storage requirements. As long as you have the vdisk image, you should be fine.

If you’ve used snapshots: Make sure to back up the extra data in libvirt to capture the snapshot hierarchy accurately.

 

8.  Sounds like a solid plan!

Edited November 25, 20241 yr by bmartino1
typo

Sounds like a good plan. 

 

To be extra safe you could: run scrub and smart checks first on your existing BTRFS pool. If you want to be super-duper extra safe, run bunker or the dynamix file integrity tool and save all of the b3sum/md5sum hashes to file attributes on the existing pool. Don't use mover, run rsync -avX to copy the files over to the external drive and run bunker -a to re-verify on the external drive everything was copied and verified correctly. Keep the nvme pair as a backup.

 

You don't really need to nuke the whole usb, but it's fine if you want to. Keep a copy of the config/key as suggested.

 

 

 

Edited November 26, 20241 yr by foo_fighter

On 11/25/2024 at 11:25 PM, bmartino1 said:

I don’t see a problem, just to be clear. Unraid 7 doesn’t have a direct download since it’s in beta. Download the latest stable version and upgrade through the web UI.

Currently, v7 beta 2 is less buggy than some of the earlier versions, but v7 beta 4 has all the feature sets along with some bug patches. If you choose to go beta, I’d recommend starting with beta 2 for setup and recovery, then upgrading to beta 4.

 

 

1. Backup/Copy Data:
Step 1 doesn’t have to involve using Mover; it can be any method to copy/backup the existing data, such as rsync, mv/cp commands, mc, or krusader. The key is ensuring the files are accessible on the specified disk for restoration.

 

2. Flash Backup:
Please make a flash backup:

Web UI > Main > Flash

This is particularly important if you plan to reuse the same USB. Make sure to copy the config folder and the license key file into the new file to maintain your Unraid license.

 

3. RAID Setup:
Set up the RAIDZ1 configuration beforehand for the best results.

 

4. Unassigned Disk Plugin:
The Unassigned Disk plugin is useful here. Depending on your license, you may need to monitor the total number of disks attached when restoring files. As with Step 1, use whatever method works best to copy files from your backup into the array.

 

5. Restore Configurations:
Using the flash backup (e.g., old Docker settings), you can easily copy necessary files into the config folder or download fresh apps from the CA Store.

 

6. Appdata Backup Plugin:
The Appdata Backup plugin is a great tool for restoring Docker containers if you don’t copy the folder from the flash backup. However, ensure the plugin is correctly configured to capture a full backup. Some setups may not be fully backed up, which is why Step 1 is critical to ensure all data is accounted for.

 

7. VMs:
VMs can be tricky due to their large storage requirements. As long as you have the vdisk image, you should be fine.

If you’ve used snapshots: Make sure to back up the extra data in libvirt to capture the snapshot hierarchy accurately.

 

8.  Sounds like a solid plan!

Hey,

 

So the moment was there when i wanted to move files to a temp SSD which was USB mounted (T7 Samsung). I decided to use Unblanace and quite fast found out that this is not smart method as it threw me around 300.000 + permission errors. PEople told me to nuke permissions but i don't think that's a smart option since many dockers require their respective permissions.

 

What would you advice on be the smartest way to transfer it? i thought about doing each folder with 'mv' but i'm not sure if that's the smartest move.

 

Edit 1:

Since i have only 3 slots for nvme, i've just purchased a PCIE to NVME adapter so i can put the 4th NVME it so i can transfer it to a internal NVME instead of external SSD. Question still remains.

Edited January 15, 20251 yr by Mirano
added edit1

3 hours ago, Mirano said:

Hey,

 

So the moment was there when i wanted to move files to a temp SSD which was USB mounted (T7 Samsung). I decided to use Unblanace and quite fast found out that this is not smart method as it threw me around 300.000 + permission errors. PEople told me to nuke permissions but i don't think that's a smart option since many dockers require their respective permissions.

 

What would you advice on be the smartest way to transfer it? i thought about doing each folder with 'mv' but i'm not sure if that's the smartest move.

 

Edit 1:

Since i have only 3 slots for nvme, i've just purchased a PCIE to NVME adapter so i can put the 4th NVME it so i can transfer it to a internal NVME instead of external SSD. Question still remains.

 

Type in mc into the terminal window, it supports mouse input as well as using the keyboard.

 

Do not close the terminal window until you're done or else the operation will stop.

Dynamix File Manager, built-in to V7

 

 

1 hour ago, MowMdown said:

 

Type in mc into the terminal window, it supports mouse input as well as using the keyboard.

 

Do not close the terminal window until you're done or else the operation will stop.

Hey. I've just tried this but i can only seem to copy, and not move the files. or am i mistaken?

1 hour ago, trurl said:

Dynamix File Manager, built-in to V7

 

 

Good shot!

 

I will try that. I first had to figure out how to create the virtual share on the new datashare haha

 

Edit 1: i just tried that, but it cannot find a difference between the 2 shares, it only finds the shares not the disks its on

Edited January 15, 20251 yr by Mirano
edit 1

1 minute ago, Mirano said:

I first had to figure out how to create the virtual share on the new datashare haha

Not entirely sure about your use of terminology here. But normally you would use Unassigned Devices plugin to access unassigned storage devices.

10 minutes ago, trurl said:

Not entirely sure about your use of terminology here. But normally you would use Unassigned Devices plugin to access unassigned storage devices.

Hey. Looking back it indeed sounds a bit weird.

 

What i do is the following. i have a share called "Appdata" 

 

i currently have my 2 disks with BTRFS on that share with no secundairy storage.

 

What my plan is to:

1. Assign the temp storage as a 1st storage and the btrfs disks as secundairy

1. Stop the docker container

2. Move the files from the docker to the temp storage

3. Since all connections are still to "/mnt/user/Appdata/* it should all still work.

4. Start the docker

 

When looking at your command you would've moved it to a unassigned disk:

1. Assign temp storage to unassigned disk

2. Stop docker container

3. Move the files to the unassigned disk on "/mnt/disks/*" 

4. Change the storage paths to the unassigned disk on "/mnt/disks/*"

5. Start the docker

 

Could you, perhaps shed some light on which you would choose and why?

Edited January 15, 20251 yr by Mirano

28 minutes ago, Mirano said:

Hey. I've just tried this but i can only seem to copy, and not move the files. or am i mistaken?

 

RenMov option #6 moves files

unasigned disk plugin to mount the usb temp ssd. then use rsync, mc or termainl cp comands to move data.

Hey everyone!

 

So it all worked out quite well. (other than accidently deleting a database).

 

I've got one question remaining:

 

I currently have the following:

 

 

the pool with 1 disk is called; "extra" 

the pool with 3 disks is called: "main" which is a raidz1

 

I'm goign to move the data back to the main pool and remove the 1TB nvme from the system.

Here's my problem. i went over to my test server and tried to emulate it:

 

 

on my test server i have 2 pools:

When i do "New Config" under tools and i tell it to preserve the full layout:

Going back to the main page i can see that the pools stay there and i can remove the one i wanted gone:

 

What i've noticed which is my question: the File system went from "ZFS" to "AUTO".

 

My question:

What happens if i do that with my Raidz1 array on my production machine, would i need to place the disks in the same pattern as now and set it to Raidz1 again and it will just work?

 

tldr: When doing new config and retaining pool devices it doesn't keep the config set to make it RaidZ1 and instead resets to "auto"

Edited January 19, 20251 yr by Mirano

9 hours ago, Mirano said:

When doing new config and retaining pool devices it doesn't keep the config set to make it RaidZ1 and instead resets to "auto"

Leave in auto, it will automatically import any existing pool.