[SOLVED] Mover is being stupid, trying to move 60gb file into drive that does not have the space.

[je82]

I was wondering why my cache is filled up even though mover has been running and completeing.

 

Turns out my mover is looping and filling up a drive to 0bytes and then failing and trying again. I have no idea why the mover keeps trying to add files larger than what the remaining space on the drive is.

 

Share configuration:

Allocation Method: Fill Up

Minimum Space Free: 10GB

 

4 Drives assigned to the share.

 

Mover is trying to add 60gb blu-ray files into a drive that has 14gb free (and 10gb being the limit), so the mover should only see 4gb free?

 

Why?

 

EDIT: I've now set the minimum space free to 30gb to make the mover bypass the drive with 14gb free, and it allocates the new files to a drive that has space, i was under the assumption that the mover is smart enough to stat each file before it moves to see if it actually has the allocated space available but i guess not? That seems like something that should be fixed (if it is not already fixed, i am running Version: 6.7.2 still)

Edited August 18, 2020 by je82

[itimpi]

I do not think it is is mover that selects the target drive but lower level code within in Unraid that is applied at the time the ‘file create’ call is issued, and at that level the system does not know the size of the file.  That is why there is a recommendation that the Minimum Free Space value is set to be as large (or larger) as the biggest file you expect to copy so that the copy will complete without running out of space.

[je82]

16 minutes ago, itimpi said:

I do not think it is is mover that selects the target drive but lower level code within in Unraid that is applied at the time the ‘file create’ call is issued, and at that level the system does not know the size of the file.  That is why there is a recommendation that the Minimum Free Space value is set to be as large (or larger) as the biggest file you expect to copy so that the copy will complete without running out of space.

Thanks for explaining. Is it possible to somehow include into the mover script to check filesize and current space available to path before each copy when running the mover or would that create far to much overhead? I feel like there must be a smarter way of doing this.

 

I sometime have 500gb files (backup images) and having to set the minimum free space > 500gb seems crazy waste of space! I am surprised i have not ran into any issues of this kind until now.

[Frank1940]

11 hours ago, je82 said:

Thanks for explaining. Is it possible to somehow include into the mover script to check filesize and current space available to path before each copy when running the mover or would that create far to much overhead? I feel like there must be a smarter way of doing this.

 

This 'problem' has been known for years and the situation is still as it was back in the early 2010's.  SO I suspect that you have already surmised the answer in your rhetorical question. 

 

Observe that the Minimum Free Space is set for each share.  Could you simply setup a share for just those image files.  That would prevent an attempt to write to those disks with less than the setting for that share but the space could be available for for use by other shares with a smaller Minimum Free Space setting.

 

I suspect that I know why your Image files are so large.  You are probably backing up Windows computers where the user data files are on the same logical disk as the OS.  One thing that can help this situation is to setup multiple logical disks.  I actually use an SSD for the OS only.  I use a spinner for the Data disk.  (My WIN10 images are about 30GB.)   (I have several folders on that spinner---Documents, Downloads and Extra_stuff.  I have redirected the Windows Pointers to Documents and Downloads to keep them off of the OS disk.  If you are using the default Music and Pictures folder, you should probably change them also.   Google for how to do this.  This spinner also where I initially store all of the Images of the SSD.)

Edited August 17, 2020 by Frank1940

[je82]

19 hours ago, Frank1940 said:

This 'problem' has been known for years and the situation is still as it was back in the early 2010's.  SO I suspect that you have already surmised the answer in your rhetorical question. 

 

Observe that the Minimum Free Space is set for each share.  Could you simply setup a share for just those image files.  That would prevent an attempt to write to those disks with less than the setting for that share but the space could be available for for use by other shares with a smaller Minimum Free Space setting.

 

I suspect that I know why your Image files are so large.  You are probably backing up Windows computers where the user data files are on the same logical disk as the OS.  One thing that can help this situation is to setup multiple logical disks.  I actually use an SSD for the OS only.  I use a spinner for the Data disk.  (My WIN10 images are about 30GB.)   (I have several folders on that spinner---Documents, Downloads and Extra_stuff.  I have redirected the Windows Pointers to Documents and Downloads to keep them off of the OS disk.  If you are using the default Music and Pictures folder, you should probably change them also.   Google for how to do this.  This spinner also where I initially store all of the Images of the SSD.)

Thanks for the input, and you're right, setting up a dedicated share for these kinds of things is a good workaround.

 

I was just very surprised i had not run into any issues at all until a few days ago.

[itimpi]

The original mover was just a shell script that ran on top of the User Share sub-system.   It is now more sophisticated than that I believe, so maybe at some future time it will be improved to take into account available space on a drive when it runs.