Unable to copy a file to the NAS despite having lots of free space on multiple drives

[Axanar]

I am going crazy trying to copy a 1.2TB file to my NAS. After copying the file
for hours I get an error that I need 685GB more space to copy the file.
The NAS has 35.4TB free out of 96TB. My NAS has 15 drives, 2 are parity and
13 data drives.
I have copied of lots of data off of the NAS and then I ran xfs_fsr on every
data drive to defrag each drive. there are four 6TB drives that EACH have
5.96TB of 6TB free and a fifth 6TB drive with 5.3TB free. All have been defragged
and yet I still get the error that I need 685GB more space free to copy a
1.2TB file to the NAS. All shares are set to use all the data drives.

[itimpi]

You should post your system's diagnostics zip file in your next post in this thread to get more informed feedback.  It is always a good idea to post this if your question might involve us seeing how you have things set up or to look at recent logs.  It will also be worth mentioning which share you are trying to use for this?

[Axanar]

thanks.  I will start the drag and drop of the file again and get a fresh log.
 

[Axanar]

As the file is copying I see that only three drives are being written to.  The 2 parity drives and disk2 which only has 304 GB of free space instead of one of the drives that have 6 TB of free space.

 

[itimpi]

Please provide diagnostics and mention the share.

[Axanar]

godzilla-diagnostics-20240721-1216.zip

[itimpi]

Which share were you trying to copy the file that fails to?  Was it by any chance the ‘isos’ share as I noticed that you have the ‘isos’ share set to be cached on a ‘cache’ pool that you do not have.

You also have the  Mover Tuning plugin installed that does not make sense if you do not have a pool being used for caching.

[Axanar]

the share was "game images".  all shares can use all drives.  I previously had everything in one big share called "data".  I had the same problem of not being able to copy large files was happening back then too.  I made a few more shares and moved a bunch of files from the data share to the new shares hoping that would defrag the drives a bit.  it didn't help.  In fact I was getting the same problem trying to move files from the data share to one of the others. 

 

So then I moved a bunch of files from the NAS to an external drive.  when I tried to move the files back it said there wasn't enough free space.  I hadn't written any other files to the NAS in between.  I was only able to move back small files.  Now I how a bunch of directories where some of the files are on the NAS and others are on external storage and I can't move them back to the NAS even though it has 35 TB of free space..

 

I had a 500 GB SSD as a cache drive.  Thinking it could be part of the problem I changed the cache drive to "no device" yesterday.  it didn't help.

 

I just removed the mover plugin.

[itimpi]

I think that the problem is arising because 2 conditions apply:

• A disk (disk2 in this case) has more free space than is set as the Minimum Free Space for the share to be used.
• the file to be copied is larger than the free space on the drive.

the first condition will allow a drive to be selected as the selection process has no idea how large the file will be.  Once a drive is selected Unraid will not change its mind and use another drive.   The copying process will start but the file will not fit so an error is generated.  
 

The only way to avoid this would be to have disk shares enabled and explicitly copy to a disk that has enough space.

 

You would have been less likely to have encountered this condition if you had used the default High Water allocation method instead of Fill Up as the free space would then be more evenly spread across your drives.

[Axanar]

It was set to High Water when the problem first started happening.  Later I changed it to Fill Up hoping that would help.  It didn't.

Edited July 21 by Axanar

[itimpi]

Just now, Axanar said:

It was set to High Water when the problem first started happening.  Later I changed it to Fill Up hoping that would help.

The same problem could occur with High Water if the conditions I mentioned apply.

[Axanar]

Currently, I changed the share to specifically use two disk that have over 5TB of free space.  I expect that will succeed.

What would be your recommendation for me to do now?  I would hate to have to manually set a specific disk to use for transfers

and keep changing it as needed.

 

What would be your recommendation for me to do now? 

I can set it back to High Water and increase the minimum free space.

Would going back to a single "data" share help, hurt, or make no difference?  This all started happening when it was set to High Water

and all shares were set to use all drives.  I believe this all started when a large file wouldn't copy to the "data" share.  I replaced a 6 TB drive with

a 12 TB drive and it still wouldn't copy saying there wasn't enough free space.
 

[itimpi]

I think you will have to just treat very large files as special cases and explicitly copy them to a drive that is known to have enough space.

 

[Axanar]

Thank you for your help!  I've been stuck with this for quite a while - copying files back and forth, de-fraging disks, changing settings and trying again.

 

One question though.  Does the size of the cache drive have to be larger than the biggest file you will ever copy?

[itimpi]

Just now, Axanar said:

Thank you for your help!  I've been stuck with this for quite a while - copying files back and forth, de-fraging disks, changing settings and trying again.

 

One question though.  Does the size of the cache drive have to be larger than the biggest file you will ever copy?

Yes if you are trying to cache it.   Not relevant if you are copying direct to an array drive.

[Axanar]

Thanks.

[Axanar]

I still am unable to copy that file.  I have the share set to use only drives 9 and 10.  Both of which have plenty of space (4 TB and 6 TB).  I then tried a smaller 476 GB file.  That also failed.  So then instead of using Windows drag and drop of the file I used Beyond Compare to copy the file.  That also failed.

 

godzilla-diagnostics-20240722-1445.zip

Edited July 22 by Axanar

[Kilrah]

Make sure there isn't a partial file from a previous copy attempt with the same path/name on any drive, if there is it'll continue to try to put it on that drive.

 

Also to avoid "clogging" a drive the share min free space should be more than the size of the biggest file you expect to transfer to that share... which is a bit of a problem when you're often transferring such huge files.

Edited July 22 by Kilrah

[Axanar]

I checked all the drives.  No file with the same name, neither partial or complete.

 

[JorgeB]

If I understood correctly, the copy starts but then aborts with an error? If yes, to which disk is the file going?

[Axanar]

disk 9.  I changed the share to go only to disks 9 and 10 instead of all disks.  disk 9 has 4 TB of free space and disk 10 has 6 TB of free space.  both are 6 TB drives.

During the copy I watched on the main page and it was doing writes to disk 9 as it should.  The attempted copy of a 476 GB took hours before it failed. 

[JorgeB]

476GB should take about an hour and a half, assuming gigabit, but doesn't look like an allocation issue, or it should fail sooner, to you know the timecode of the copy failure in the diags above?

[Axanar]

I just tried again with Beyond Compare and it failed again.  I've tried both drag and drop with file explorer and beyond compare to copy the file.  I'm getting the diagnostics.zip now.

 

godzilla-diagnostics-20240723-0812.zip

[JorgeB]

There's nothing logged, what is the error that Windows shows?

[Kilrah]

Don't see anything specific to this but there seems to be a "cache" pool defined and used in shares but with no disks assigned to it?!