running out of disk space, will unraid move files accordingly?

[takkkkkkk]

I started getting notifications that one of my drive is low on space, but I have 10 other drives which are hardly used. I'm just wondering if I should start moving files myself, or could I ignore the message and unraid would balance the drives automatically?

I would thought unraid should do it automatically, but thought I would ask...

[gundamguy]

I started getting notifications that one of my drive is low on space, but I have 10 other drives which are hardly used. I'm just wondering if I should start moving files myself, or could I ignore the message and unraid would balance the drives automatically?

I would thought unraid should do it automatically, but thought I would ask...

 

Unraid does not have a function that balances files post write.

 

The only balancing happens at the time of the write, when the user share logic decides what disk to write to.

 

There are plugins that might be able to help you balance your data.

[takkkkkkk]

I started getting notifications that one of my drive is low on space, but I have 10 other drives which are hardly used. I'm just wondering if I should start moving files myself, or could I ignore the message and unraid would balance the drives automatically?

I would thought unraid should do it automatically, but thought I would ask...

 

Unraid does not have a function that balances files post write.

 

The only balancing happens at the time of the write, when the user share logic decides what disk to write to.

 

There are plugins that might be able to help you balance your data.

 

Oh that's bummer... I would thought that would be one of the most requested feature...

Can you point me to the plug in that could help me? Meanwhile I've got to start moving files...

[trurl]

I started getting notifications that one of my drive is low on space, but I have 10 other drives which are hardly used. I'm just wondering if I should start moving files myself, or could I ignore the message and unraid would balance the drives automatically?

I would thought unraid should do it automatically, but thought I would ask...

 

Unraid does not have a function that balances files post write.

 

The only balancing happens at the time of the write, when the user share logic decides what disk to write to.

 

There are plugins that might be able to help you balance your data.

 

Oh that's bummer... I would thought that would be one of the most requested feature...

Can you point me to the plug in that could help me? Meanwhile I've got to start moving files...

In addition to moving the files, you need to examine your user share settings, because that is what determined how all those files wound up on that disk instead of any of the other disks in the first place.

 

When you are examining your user share settings, turn on the help button and it will explain what all the settings do.

 

The plugin you are looking for is unBalance. See search tips in my sig.

[itimpi]

Oh that's bummer... I would thought that would be one of the most requested feature...

Normally if you have your system set up with the correct settings on your shares for Allocation Method and Split Level then unRAID writes things to appropriate places in the first place.  Do you know why this has not happened on your system as your request implies it has not?

[gundamguy]

Super important if you are going to start moving data around never copy from a user share to a disk share or vice versa. Doing so will cause you to have data loss as the underlying FUSE (File system in userspace) will clobber data when it get's confused.... (it's hard to explain exactly what happens, but basically the FUSE doesn't know that the disk location is the same location and writes to it clobbering the data)

 

Simply put if you are doing this on command line, never mix /mnt/user/XXXX with /mnt/diskX/XXXX move from disk to disk or user share to user share.

 

If you are doing this via Windows or over SMB (or another network share) make sure you are sharing only user shares or only disk shares to completely avoid the issues (by default the settings is that disks aren't shared over SMB)

[takkkkkkk]

I started getting notifications that one of my drive is low on space, but I have 10 other drives which are hardly used. I'm just wondering if I should start moving files myself, or could I ignore the message and unraid would balance the drives automatically?

I would thought unraid should do it automatically, but thought I would ask...

 

Unraid does not have a function that balances files post write.

 

The only balancing happens at the time of the write, when the user share logic decides what disk to write to.

 

There are plugins that might be able to help you balance your data.

 

Oh that's bummer... I would thought that would be one of the most requested feature...

Can you point me to the plug in that could help me? Meanwhile I've got to start moving files...

In addition to moving the files, you need to examine your user share settings, because that is what determined how all those files wound up on that disk instead of any of the other disks in the first place.

 

When you are examining your user share settings, turn on the help button and it will explain what all the settings do.

 

The plugin you are looking for is unBalance. See search tips in my sig.

 

thanks! I'll check out the plugin.

for all the files (except for few of the cached disks that I need to be on SSD), I have selected to use all disks with the same split level. My thought was that unraid would be smart enough to balance the disks if one of the disk is low on space regardless of the level of the directory.

Can you elaborate on what I could have done??

 

[smashingtool]

I started getting notifications that one of my drive is low on space, but I have 10 other drives which are hardly used. I'm just wondering if I should start moving files myself, or could I ignore the message and unraid would balance the drives automatically?

I would thought unraid should do it automatically, but thought I would ask...

 

Unraid does not have a function that balances files post write.

 

The only balancing happens at the time of the write, when the user share logic decides what disk to write to.

 

There are plugins that might be able to help you balance your data.

 

Oh that's bummer... I would thought that would be one of the most requested feature...

Can you point me to the plug in that could help me? Meanwhile I've got to start moving files...

 

Did you add those drives recently? If not, then your share config must be wrong. unRaid by default will fill a disk to a low % and then move onto the next one. But if you set a share to only include a certain disk, that won't happen.

 

Hence why it's not a feature anyone really needs.

[smashingtool]

I started getting notifications that one of my drive is low on space, but I have 10 other drives which are hardly used. I'm just wondering if I should start moving files myself, or could I ignore the message and unraid would balance the drives automatically?

I would thought unraid should do it automatically, but thought I would ask...

 

Unraid does not have a function that balances files post write.

 

The only balancing happens at the time of the write, when the user share logic decides what disk to write to.

 

There are plugins that might be able to help you balance your data.

 

Oh that's bummer... I would thought that would be one of the most requested feature...

Can you point me to the plug in that could help me? Meanwhile I've got to start moving files...

In addition to moving the files, you need to examine your user share settings, because that is what determined how all those files wound up on that disk instead of any of the other disks in the first place.

 

When you are examining your user share settings, turn on the help button and it will explain what all the settings do.

 

The plugin you are looking for is unBalance. See search tips in my sig.

 

thanks! I'll check out the plugin.

for all the files (except for few of the cached disks that I need to be on SSD), I have selected to use all disks with the same split level. My thought was that unraid would be smart enough to balance the disks if one of the disk is low on space regardless of the level of the directory.

Can you elaborate on what I could have done??

 

You should post screenshots of your shares settings, because unRaid absolutely is smart enough to do that. It doesn't "balance the disk if it's low on space", it prevents one disk from getting way lower on space than the others by default.

[takkkkkkk]

Oh that's bummer... I would thought that would be one of the most requested feature...

Normally if you have your system set up with the correct settings on your shares for Allocation Method and Split Level then unRAID writes things to appropriate places in the first place.  Do you know why this has not happened on your system as your request implies it has not?

 

My exact thought... I have no idea... I know I created a new share for crashplan and folder for working documents(includes pictures and videos), but none of the folders have a file that is growing dramatically. So, if anything unraid should have been able to put some of the files to other drives especially since I have the split level of automatic (unless I'm misunderstanding what autiomatic does...)....

[takkkkkkk]

You should post screenshots of your shares settings, because unRaid absolutely is smart enough to do that. It doesn't "balance the disk if it's low on space", it prevents one disk from getting way lower on space than the others by default.

Here is a screenshot from one of the usershare. it's all set as automatic and to use all the drives... And I haven't added a new drive, it's been all the same.

 

 

[itimpi]

I suspect that the split level is wrong for the behaviour that you want to achieve?

 

You currently have Split Level set to a level of 1, which means that a particular share can be split across multiple drives.  However once any folder is created within that share then any further files/folders for that folder (or its contents) are constrained to the disk where the folder was created.  For example if the share was Called "Media" and within that you had a folder called "Movies" then anything that is to be under Movies would be constrained to the disk where the Movies folder was first created.

 

If you give some examples of paths you have and what you want to achieve then it might be possible to suggest a suitable split level to achieve the desired results.

 

A final point to be aware of is that if there is a potential conflict between the "Allocation Method" and the "Split Level" settings as to where a file should be placed then the Split Level setting always wins.

[takkkkkkk]

I suspect that the split level is wrong for the behaviour that you want to achieve?

 

You currently have Split Level set to a level of 1, which means that a particular share can be split across multiple drives.  However once any folder is created within that share then any further files/folders for that folder (or its contents) are constrained to the disk where the folder was created.  For example if the share was Called "Media" and within that you had a folder called "Movies" then anything that is to be under Movies would be constrained to the disk where the Movies folder was first created.

 

If you give some examples of paths you have and what you want to achieve then it might be possible to suggest a suitable split level to achieve the desired results.

 

A final point to be aware of is that if there is a potential conflict between the "Allocation Method" and the "Split Level" settings as to where a file should be placed then the Split Level setting always wins.

 

Ah thank you! I thought I had it automatically split any directory,but I guess I did not. I've changed the split level, will this solve the issue??

[itimpi]

Ah thank you! I thought I had it automatically split any directory,but I guess I did not. I've changed the split level, will this solve the issue??

Note that changing the Split Level only applies to new files - it does not affect existing files which you are likely to still have to move manually to sort out the current situation.  Whether the new split level you have chosen is suitable for the behaviour you want I have no idea.

 

Note that it is also a good idea to set the Minimum Free Space to be at least the size of the largest file you are likely to copy as otherwise you might start copying a file and then get a failure when the free space is exhausted during the copy.

[gundamguy]

I suspect that the split level is wrong for the behaviour that you want to achieve?

 

You currently have Split Level set to a level of 1, which means that a particular share can be split across multiple drives.  However once any folder is created within that share then any further files/folders for that folder (or its contents) are constrained to the disk where the folder was created.  For example if the share was Called "Media" and within that you had a folder called "Movies" then anything that is to be under Movies would be constrained to the disk where the Movies folder was first created.

 

If you give some examples of paths you have and what you want to achieve then it might be possible to suggest a suitable split level to achieve the desired results.

 

A final point to be aware of is that if there is a potential conflict between the "Allocation Method" and the "Split Level" settings as to where a file should be placed then the Split Level setting always wins.

 

Ah thank you! I thought I had it automatically split any directory,but I guess I did not. I've changed the split level, will this solve the issue??

 

Changing the Split Level will fix the issue for future writes to the share, but it won't retroactively move data. So yes and no.

 

If you don't care what disk files go to your split level should be 99, 1 means that everything written to that share will go to the same disk.

 

 

Share Level One means anything written to:

/mnt/user/share or below will be kept on the same disk. (Share is what you see via SMB)

 

Share Level 2 means anything written

/mnt/user/share can go to any disk but anything written to a folder below that say /mnt/user/share/tv will be kept together. So if you add folder like /mnt/user/share/movies it could be created on any disk that's included in the share, but if you write to /mnt/user/share/movies/bladerunner that would be kept on the same disk that /mnt/user/share/movies is on....

 

And this continues like that... so on and so on...

 

 

[takkkkkkk]

I suspect that the split level is wrong for the behaviour that you want to achieve?

 

You currently have Split Level set to a level of 1, which means that a particular share can be split across multiple drives.  However once any folder is created within that share then any further files/folders for that folder (or its contents) are constrained to the disk where the folder was created.  For example if the share was Called "Media" and within that you had a folder called "Movies" then anything that is to be under Movies would be constrained to the disk where the Movies folder was first created.

 

If you give some examples of paths you have and what you want to achieve then it might be possible to suggest a suitable split level to achieve the desired results.

 

A final point to be aware of is that if there is a potential conflict between the "Allocation Method" and the "Split Level" settings as to where a file should be placed then the Split Level setting always wins.

 

Ah thank you! I thought I had it automatically split any directory,but I guess I did not. I've changed the split level, will this solve the issue??

 

Changing the Split Level will fix the issue for future writes to the share, but it won't retroactively move data. So yes and no.

 

If you don't care what disk files go to your split level should be 99, 1 means that everything written to that share will go to the same disk.

 

 

Share Level One means anything written to:

/mnt/user/share or below will be kept on the same disk. (Share is what you see via SMB)

 

Share Level 2 means anything written

/mnt/user/share can go to any disk but anything written to a folder below that say /mnt/user/share/tv will be kept together. So if you add folder like /mnt/user/share/movies it could be created on any disk that's included in the share, but if you write to /mnt/user/share/movies/bladerunner that would be kept on the same disk that /mnt/user/share/movies is on....

 

And this continues like that... so on and so on...

 

Got it! Are there any reason I should care about the existing files in the drive? the drive is now 98% utilized, but if unraid is smart enough to know to write new files to other drives, I'm good right?

[John_M]

The existing files are fine and, yes, unRAID will store subsequent files according to your new strategy. It's really down to personal preference. Some people like to fill their disks roughly equally while others fill up first one and then the next. Still others manually allocate files to specific disks according to how they choose to classify them. Your files are equally safe however you allocate them but certain strategies are aimed at requiring the minimum number of disks to spin up when accessing your shares.

[tdallen]

If I recall correctly, ReiserFS suffered from some instability issues when the disk is near 100% capacity.  XFS is better in that regard.  You might want to move some files off the 98% disk if you are using ReiserFS.

[John_M]

I don't think there are problems reading nearly full ReiserFS format disks, are there? It's writing to them that causes problems. I agree that XFS is better in nearly every respect though.

[trurl]

Got it! Are there any reason I should care about the existing files in the drive? the drive is now 98% utilized, but if unraid is smart enough to know to write new files to other drives, I'm good right?

Entirely depends on what you have set the split level to since split level takes precedence over all other settings. If split level is telling it that any new files for a certain folder should be kept together, then it is going to write them to the same disk, which means it could write even more files to this disk if that is what the split level tells it to do.

[BRiT]

Oh that's bummer... I would thought that would be one of the most requested feature...

Can you point me to the plug in that could help me? Meanwhile I've got to start moving files...

 

I absolutely do NOT want my data being written multiple times. I want it written once and once only. Every time you're moving your actual data around is a chance for it to be corrupted.

[garycase]

Look at your Global Share settings [settings - Global Share Settings]  and be sure ALL of your disks are included in the global settings.

 

Even though your individual shares are set to use "All" disks, "All" in that case means all of the disks that are available for use in shares -- which is a function of the Global settings.

 

You do NOT need to move any files around ... you just need to get your settings correct.