Possibly cool new xfs tools in the pipeline


fishface

Recommended Posts

Ha. I was just performing a quick search to make sure I wasn't double posting.

 

Any chance cp --reflink will work with XFS on the next Unraid update?

 

https://blogs.oracle.com/linux/xfs-data-block-sharing-reflink

 

Currently the array is XFS while the cache is BTRFS

 

You can reflink files within the cache, but you cannot go from the array to the cache (XFS -> BTRFS). And I just tried to reflink the VM image from within the user share (array/XFS) and it results in Operation not supported.

 

So, it would be great to be able to reflink from within user shares which, is the safer way to work with data (from my understanding) on the array versus trying to work directly from the disk "shares".

 

Link to comment
  • 4 months later...
On 4/12/2020 at 2:49 AM, Zer0Nin3r said:

Ah very cool. I guess there's no chance in getting it to work unless I were to reformat everything which means, I probably won't be able to enjoy the new functionality.

I just added a new disk to my array and found out about the reflink thing while trying to figure out exactly why it shows 66 GB used* on an empty, newly formatted 10 TB disk. All of my old disks have reflink=0 (per xfs_info command), and I don't believe it's possible to enable it without a reformat.

*Seemed high, but I'm honestly not sure what it was on other disks when I added them. Something I stumbled upon in a Google search indicated that new XFS disks have significantly more "used" space to start with when that feature is enabled.

Link to comment
1 hour ago, gilahacker said:

I just added a new disk to my array and found out about the reflink thing while trying to figure out exactly why it shows 66 GB used* on an empty, newly formatted 10 TB disk. All of my old disks have reflink=0 (per xfs_info command), and I don't believe it's possible to enable it without a reformat.

*Seemed high, but I'm honestly not sure what it was on other disks when I added them. Something I stumbled upon in a Google search indicated that new XFS disks have significantly more "used" space to start with when that feature is enabled.

That's the overhead for XFS file system and has nothing to do with this topic, as far as I understand it.

Link to comment
7 hours ago, gilahacker said:

Then one of us misunderstood what @Zer0Nin3r was asking about regarding `copy --reflink`, which requires the `reflink=1` flag to be set when the disk is formatted.

Since nobody explained what reflink was supposed to be/do I guess it could have also been a cool new pizza oven for XFS and Unraid..  it seems that reflink is mostly used for file system block sizes.  It might have some effect on overall overhead of the file system but it's not directly for that, it would seem.  Might also effect file write speed. 

 

https://blogs.oracle.com/linux/xfs-data-block-sharing-reflink

 

And here's another article talking about using it on a ZFS system.  https://blogs.oracle.com/solaris/reflink3c-what-is-it-why-do-i-care-and-how-can-i-use-it

And another, http://dashohoxha.fs.al/deduplicating-data-with-xfs-and-reflinks/

 

As far as existing overhead, this is more for what you are talking about:

 

 

Link to comment
  • 2 weeks later...
On 8/12/2020 at 7:55 PM, gilahacker said:

Then one of us misunderstood what @Zer0Nin3r was asking about regarding `copy --reflink`, which requires the `reflink=1` flag to be set when the disk is formatted.

On 8/13/2020 at 3:33 AM, Energen said:

Since nobody explained what reflink was supposed to be/do I guess it could have also been a cool new pizza oven for XFS and Unraid.. 

So, it's never a good idea to be working directly within the /mnt/disk (from what I have gathered on the forums.) What I was referring to in my original post is creating snapshots of my VMs as a means to create lightweight backups without wasting space; the other alternative would be to create an exact copy of a VM image, but you are then wasting space.

 

With the cp --reflink command, you are simply noting the changes in the blocks of data from the original image. So, if I were to make some changes to my gaming rig VM and I messed something up, I can just reverse the cp --reflink command and go back to the state that my VM was in before the changes were made.

 

My understanding with snapshots is that if the master file (disk image in this case) gets corrupted, then the reflinks will then be corrupted too. Another thing to note with the reflink flag is that Unraid will/may show that the full image of the reflink is taking up xx amount of space, when in fact, because it is a snapshot, is not actually taking up all the space.

 

Example: If I have a 100 GB VM image, I do a snapshot after installing let's say a 50 MB program, then the snapshot should only take up about 50 MB since those are the only data blocks that have changed in the VM image. However, Unraid will show that the new reflink snapshot to be taking up 100 MB when it is not really taking up that much space (you should be able to see this inside of Terminal or Midnight Commander).

 

Going back to my original point earlier. In order for me to make snapshots with my VM's and in order to use the --reflink flag for the cp command, I have to be in my /mnt/cache/domains/location_of_my_VM_image in order for the command to work. If I try the --reflink flag in my /mnt/user/domains/location_of_my_VM_image (which like I said earlier, you typically always want to be working out of the /mnt/user directory) then the command will fail because I formated my disks to XFS before the format revision that now supports the --reflink flag (reflink = reference link).

 

Furthermore, I was commenting on the fact that even though the revision to the XFS format allows us to take advantage of snapshots, I would have to reformat all of my XFS partitions if I wanted to take advantage of this new feature to which, it does not seem like a possibility for me now unless I were to have another array capable of receiving a backup of my data so I can make the changes.

 

Why XFS? Because BTRFS was not recommended in the Unraid documentation as being stable enough for the array if you don't want a possible chance of data corruption. However, BTRFS was/is fine for the cache pool.

 

So, that is what I am doing when I am making my VM snapshots. I am working directly in the cache pool when I am making my VM snapshots as my VMs reside on the cache as per the share preferences.

345235751_ScreenShot2020-08-24at13_55_27.thumb.png.de0dd2d0a3d63ba4ce693dd0cf050122.png

 

 

  • Like 1
Link to comment
  • 1 year later...

Hi,

how do I know if the reflik feature is enabled? Or how to ensure that the HDD is preformatted in this way?
Is it possible to format only one disk from the whole array and use this function?
I would like to use this feature for Veeam backups, but I still get the feature turned off.

1.PNG

Link to comment
19 minutes ago, Acu said:

how do I know if the reflik feature is enabled?

xfs_info /dev/sdX1

Look for reflink, 1 next to it means enable, 0 disable.

 

20 minutes ago, Acu said:

Is it possible to format only one disk from the whole array and use this function?

Yes, with Unraid each array device is always a separate filesystem.

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.