Sparse files cause the appearance of high disk usage

[frank-unr]

I stumbled on a problem with how free disk space is presented by Unraid on the main tab. I recently added a new disk to my array. It appears as there is already 1.77 TB of data on it. However I know in reality there is much much less data.

 

 

Using rclone I accidently created a bunch of "large" sparse files that use 0 bytes of storage.

 

The actual size of all files on that drive is under 20 GB.

 

On a smb share I see the sparse files as having size zero.

 

Using ls -l   I get 0 size for the sparse files.

Using ls -s I get the large size that uses no actual disk space. 

 

From what I understand, this looks like a bug.

[JorgeB]

IIRC Unraid uses statfs() to get those values, please post the output of:

 

stat -f /dev/md#

 

Replace # with disk number.

[frank-unr]

root@Antec:/dev# stat -f /dev/md7
  File: "/dev/md7"
    ID: e508de7bfbef56ab Namelen: 255     Type: tmpfs
Block size: 4096       Fundamental block size: 4096
Blocks: Total: 2048       Free: 2048       Available: 2048
Inodes: Total: 2028594    Free: 2027960

 

All drives give the same result...

[JorgeB]

11 minutes ago, frank-unr said:

All drives give the same result...

Then it won't be that, does df report the same usage?

[frank-unr]

df reports the same usage yeah. I made more sparse files so now the reported total size is 2.2TB

 

root@Antec:~# df -h /dev/md7
Filesystem      Size  Used Avail Use% Mounted on
/dev/md7        9.1T  2.2T  7.0T  24% /mnt/disk7

 

This is what the QDirStat docker shows when I look at folders that only contain those sparse files:

 

[JorgeB]

I expect Unraid to report the same as df, you can try googlling to see if you find similar issues with xfs, sparse files and df.

[frank-unr]

For anyone reading this in the future:

 

There is a rclone option that prevents the creation of zero size sparse files when there is Google drive error when you try to sync from Google drive to local unraid array.

disable multi-thread downloads with 

--multi-thread-streams 0

stops sparse files from being created.

For me the --local-no-sparse does not fix the problem

 

https://rclone.org/docs/

 

Edited September 6, 2022 by frank-unr