Some Drives in array readjusting its utilization percentage by itself

[aquariuz23]

 

Hello all,

 

Good day to you. I'm very new to the Unraid world, no more than a few months in, and this is the first unraid setup I've cobbled up together. I have a question that potentially is a silly one but hopefully it is just a silly question and not a concern I should be worried about. 

First off, a bit of context. I migrated over from Windows to Unraid, and so I already had all these files ahead of time. Once I got my Unraid Server up and running, I transferred all the files over to Unraid in large chunks, enough to fill one or two drives in the array simultaneously. 

 

The array consists of 18 drives and my main concern is regarding Disk 1. When I started this array, Disk 1 was pegged at 88% before Unraid cascaded the files down to the next drive and so on. However, as time goes by, the utilization on Disk 1 continues to drop, and currently it is at 82%. I don't really pay attention to the rest of the Disks' utilization, but I won't be surprised if they go up since I constantly add files to the array as well.

 

My question is, does Unraid continue to move files around the array to spread the load, or does it fill up a single drive to its maximum before cascading down to the next drive? I understand that the process behind the array is pretty complicated so a detailed explanation isn't necessary. I'm just concerned that something is happening to my files in Disk 1 that I might not be aware of. I haven't been alerted to any lost of files, but I do have a lot of files, so I may not notice it yet. I don't, to the best of my knowledge, have any special setting or plugin enabled to continuously load balance the array, but I may be wrong. 

 

Thank you up front for any advice or insight provided

 

 

[itimpi]

It is difficult to answer your question without more information about your system.   You are likely to get better informed feedback if you attach your system’s diagnostics zip file to your next post in this thread so we can see more about your hardware and also how you have things configured.

[aquariuz23]

7 minutes ago, itimpi said:

It is difficult to answer your question without more information about your system.   You are likely to get better informed feedback if you attach your system’s diagnostics zip file to your next post in this thread so we can see more about your hardware and also how you have things configured.

Thanks for your reply itimpi. Let me provide the diagnostics for you to view. Anything I should do with the zip file, such as remove any private info that it might collect? I see that there's an "anonymize diagnostic" option, but I'm not sure if I should remove anything specific.

[itimpi]

1 hour ago, aquariuz23 said:

Thanks for your reply itimpi. Let me provide the diagnostics for you to view. Anything I should do with the zip file, such as remove any private info that it might collect? I see that there's an "anonymize diagnostic" option, but I'm not sure if I should remove anything specific.

In theory anything really private should be removed.    The contents of the zip file are all text files so you can decide for yourself if you are happy eith the anonymisation process.

[aquariuz23]

6 minutes ago, itimpi said:

In theory anything really private should be removed.    The contents of the zip file are all text files so you can decide for yourself if you are happy eith the anonymisation process.

I can't seem to find anything private I should be concerned about, but please let me know if I should remove something and reupload.

magi-unraid-diagnostics-20231024-1044.zip

[aquariuz23]

Hello again. Bumping this back up. So I noticed that today a few changes happened to the array. 

 

Now with a point of reference from my previous screenshot, I can see that Disk 1 utilization dropped by 1% down to 81%, and Disk 3 also dropped by 1% down to 79%. Disk 13 went up by 2% to 47%. Now I'm not sure if some files were moved to Disk 13 from the other two disks. I also failed to mention previously that I set up the Warning disk utilization threshold percentage on all my drives at 88.

 

[aquariuz23]

Bump. Anyone? 

[itimpi]

1 minute ago, aquariuz23 said:

Bump. Anyone? 

Unraid never moves files between array drives by itself.    This needs to be done manually or using something like the unbalance plugin.   Not sure therefore what you are running that might move files in this way.

[aquariuz23]

Hmmm...then that's really strange. Unless these utilization percentages are incorrect, it's still continuing, now with Disk 13 reaching 56% utilization while mutliple other drives are dropping in usage. What's interesting is that it's specifically filling up Disk 13 first over Disk 9-12. 

 

[trurl]

5 minutes ago, aquariuz23 said:

What's interesting is that it's specifically filling up Disk 13 first over Disk 9-12. 

Disk13 is larger than those so it was next chosen by High Water Allocation. Disk13 will continue to be selected (for any shares that are configured to use it) until it is half full.

 

[aquariuz23]

53 minutes ago, trurl said:

Disk13 is larger than those so it was next chosen by High Water Allocation. Disk13 will continue to be selected (for any shares that are configured to use it) until it is half full.

 

Hi Trurl, thank you for that explanation, and that totally makes sense. So does the same "High Water Allocation" explanation apply to the other drives that are shrinking in utilization? From the latest screenshot, Disk 1, 2, 4, 5, 6 and 17 shrunk in utilization. These drives are 14TB, 10TB, 10TB, 10TB, and 8TB respectively. I wonder if it reorganizes files until the larger drives all reach 79%?

[trurl]

13 minutes ago, aquariuz23 said:

drives that are shrinking in utilization? From the latest screenshot, Disk 1, 2, 4, 5, 6 and 17 shrunk in utilization. These drives are 14TB, 10TB, 10TB, 10TB, and 8TB respectively. I wonder if it reorganizes files

 

2 hours ago, itimpi said:

Unraid never moves files between array drives by itself.

Someone or something must be moving or deleting files. 'arr dockers, for example, are often configured so downloaded files will get moved to other shares

[aquariuz23]

On 11/1/2023 at 11:17 AM, trurl said:

 

Someone or something must be moving or deleting files. 'arr dockers, for example, are often configured so downloaded files will get moved to other shares

Even more interesting. I do have arr dockers, but nothing that I know off are set to move files around. But then again, this is very much possible and I may have enabled a setting that does this.  They might also be deleting and updating files in my drives and placing those newer files in a different drive in the array. This actually makes the most sense. 

 

I guess as long as no files are physically deleted and lost, I think I won't worry too much about the usage percentage moving around. Thanks more the help trurl!