January 10, 20188 yr Description: Mover causes IMG files to take up their full amount of storage despite actual size of vdisk How to reproduce: Take a raw img file that isn't full size (i.e. 50G but 25G on disk) and then let mover move it from cache to array or array to cache Expected results: IMG file retains all its parameters Actual results: after the move, IMG file is the full size on disk verified with ls -lsh Other information: see examples below 13G -rwxrwxrwx 1 root root 50G Jan 9 03:05 vdisk1.img* 350G -rwxrwxrwx 1 root root 350G Jan 9 03:05 vdisk2.img* The first disk was created on cache and never touched by mover. The second disk was created on my array and then I used mover to move it on to cache. Prior to move the img was only about 240G. I've replicated this a few times. I then have to use qemu-img commands to build a new img file that doesn't take up the slack space. This also may have been an issue in 6.3.5 also but it took me a while to catch on to the fact that the img was now taking up the full size since you have to use du or ls -lsh to see the real size on disk. Edited January 10, 20188 yr by nickp85
January 10, 20188 yr It isn't very easy to copy sparse files, as can be seen from the following link: https://serverfault.com/questions/665335/what-is-fastest-way-to-copy-a-sparse-file-what-method-results-in-the-smallest-f/670227
January 10, 20188 yr Author Good to know, but it looks like unraid has some of those commands that can be used to move sparse files. Perhaps mover can be enhanced to handle it properly? If it detects an img file maybe use dd or cp with the sparse options? Just thinking out loud. Would also be nice to have an easy way to load libguestfs since it has a command to sparsify an img file in place which is ideal for img on cache where you don't have the space to convert it using a second file.
May 24, 20188 yr Running into the same issue. Sparse files are created fine on the cache drive, but mover expands them to preallocated size. I am on 6.5.2-rc2.
May 24, 20188 yr 12 hours ago, MrChunky said: Running into the same issue. Sparse files are created fine on the cache drive, but mover expands them to preallocated size. I am on 6.5.2-rc2. The lseek() command in Linux supports handling of sparse files by use of SEEK_DATA and SEEK_HOLE, which means it's possible to teach mover to query the source file system for the holes in moved files. This would result in faster moves of sparse files and the file would continue to be sparse on the destination drive. If mover doesn't support sparse files, then this should probably be added as an improvement suggestion in the specific subforum.
June 4, 20188 yr On 5/25/2018 at 1:18 AM, pwm said: The lseek() command in Linux supports handling of sparse files by use of SEEK_DATA and SEEK_HOLE, which means it's possible to teach mover to query the source file system for the holes in moved files. This would result in faster moves of sparse files and the file would continue to be sparse on the destination drive. If mover doesn't support sparse files, then this should probably be added as an improvement suggestion in the specific subforum. Thank you for the explanation. I have submitted a bug report a few weeks back. We'll see what the response is, possibly I will move it to feature request
Archived
This topic is now archived and is closed to further replies.