Transfer Between Disks + Losing Free Space

[Ketch]

I am hoping someone can help me.  I have been performing moves between disk numbers to balance my free space.

To perform these moves I used dynamix file manager app, then by going into each disk individually and moving data accordingly.

Originally, I had three 18TB drives with only 1TB available but I wanted to keep up to 20% free to prolong their healthy use over time.

Therefore, I purchased a fourth 18TB drive hoping to move 3TB off each of the original three drives and onto the new one.

Thus, my intentions was to see approx. 4TB free on the older three drives and near 9TB on the new one after all the moves.

However, after my internal moves between disks, now all four 18TB drives is showing roughly only 3TB available.

With 3TB free on the older disks plus the purchase of a 4th 18TB drive, I was estimating I should have roughly 21TB free, not 12TB.

Hence, I believe something associated with the internal moves between disk numbers is eating up a lot of space and fast.

May I kindly ask if anyone has experienced this before and know of a resolution?

Any and all assistance is truly appreciated.

[apandey]

Is the free space TB or TiB? An 18TB drive is only 16.37TiB usable space. Filesystems measure in TiB while drive manufacturers do TB

[Ketch]

Apologies, I was using generic terms based on the language of the products plus unraid's dashboard to over simplify matters in hopes to avoid confusion.

Originally, my array was actually 47.29TiB with only 2.9TiB available when I added another 16.37Tib to it after performing a preclear.

 

However, when I attempting to balance the data against the disks within the array, I now only have 10.9TiB free.

 

So by adding a new 16.37Tib drive into my array with 2.9TiB available, balancing my disks with no additional data added lost me approx. 8.37TiB.

 

Has anyone else experienced such a thing or might have an explanation on what could have occurred?

Edited January 15, 2023 by Ketch

[itimpi]

How did you do the ‘balance’?   Are you sure that you did not leave any copies of files behind so that you now have duplicates on different drives?

[Ketch]

I installed the app "dynamix file manager" from the community applications.

 

Went into each disk, selected the desired folder, hit move, then selected the location I wanted the data on.

After the move was completed, the folder was no longer at the old location and was found at the selected location.

Additionally, I used software called "Duplicate File Finder" on my laptop to review the data on my "media" share.

After my search with the duplicate file finder, it found no duplicate copies.

[Ketch]

To add onto that, I also downloaded "dupeGuru" from the community applications.

 

Then I performed a secondary review on my "media" share which resulted in no duplicates as well.

Edited January 15, 2023 by Ketch

[itimpi]

Another (although less likely) possibility is that some of the larger files you moved were something like VM disk images.   Such files are normally created as ‘sparse’ files which means they will not necessarily occupy the full amount of disk space their logical size allows.   This ‘sparseness’ can be lost during a copy and the files then expanded so that their physical size matches their logical size.   

[Ketch]

I am not running any virtual machines on this array so I am thinking VM Disk Images or VDI is not my issue.

 

At first I tried the Krusader app but I couldn't find a way to move data between disks just the ability to move data between shares.

 

By having the dynamix file manager, it allowed me to select folders on each disk to delete/copy/move.

 

For example, I went to disk 1, went into its media share, selected a folder with 10 videos and moved them into disk 4's media share.

 

After the move was completed, disk 1 no longer showed the folder I moved, it now only was shown under disk 4.

 

However, say that folder with 10 videos was showing as 10G worth of data.

When moved, disk 1 and disk 4 are not showing a simple decrease/increase of 10G.

 

Disk 1 showed a decrease but not near the 10G's in the folder that was removed while disk 4 showed it gained more than 10G worth of data on its new folder.

 

My first thought was maybe the move created a directory where it pointed where the data used to be and where it can be found now.

But even if something like that occurred, it shouldn't take away that much space.

 

It truly has me stumped.

[apandey]

Maybe try using QDirStat in a container to see how the space is being used

[Ketch]

Hi Apandey, thank you kindly for the idea.  I gave it a try but QDirStat was not able to highlight what could be causing the internal transfers to be wasting space.  I am more than open other ideas if anyone might have one?

[trurl]

FYI

 

Some of the lines in your forum posts cannot be read in light mode. 

[Ketch]

I have a working theory on what could be causing my issues but since I am new to this, I am hoping someone can confirm or refute my thoughts.  I setup my first unraid server following Trash's guide for atomic moves and hard links.  My thought is that files I internally moved associated with media that was hard linked between my torrent and media shares was performed like a copy/paste method.  The links were broken leaving the data in the torrent share and pasted within my media share. Thus, taking up more space than originally estimated. Therefore in theory, if I go into my torrent share and delete any remaining files, I can gain that storage back without affecting my Plex library. Thoughts?