Unraid disk space reporting in GUI - looks wrong

[salvdordalisdad]

HI All,

Love unraid, have 2 servers n2 is just a backup of n1

RSYNC is run regularly to copy data across.

I don't delete missing stuff during rsync but I clean it up manually from time to time (bcompare helps), so there's always a few leftovers & a small amount extra, but not 4.9TB worth.

However, despite runing bcompare to confirm they are pretty similar, n2 shows disk space used = 28.4TB whereas n1 shows it as 23.5TB

I've checked on the CLI using du, and only found around 0.6TB difference, when looking at the array share /mnt/user/filestore

 

However, checking the individual disk shows a very different story:

n1 = /mnt/disk1  = 5.3T    ./filestore

n2 = /mnt/disk1 = 6.6T    ./filestore

All 4 disks are the same sort of thing. (XFS)

I also ran xfs_repair on one of the disks, as I got an error on it, but didn't fix this though.

 

The other directories on these disks don't account for very much, system, appdata etc.

Interesting the GUI and CLI don't agree in the detailed numbers, but still generally in overall amount.

 

Not sure where to start looking now.

I know the last resort is to wipe & restart, but that's a whole week's worth & is scary.

Any pointers anyone can suggest?

[JorgeB]

Please post the output of:

 

rsync -avn --stats --human-readable --delete /path/to/source /path/to/dest

 

-n makes it a dry run, nothing will be transferred/deleted

[salvdordalisdad]

Hiya JorgeB

 

Number of files: 411,199 (reg: 380,126, dir: 31,073)
Number of created files: 1,927 (reg: 1,866, dir: 61)
Number of deleted files: 10,757 (reg: 8,620, dir: 2,137)
Number of regular files transferred: 379,786
Total file size: 22.88T bytes
Total transferred file size: 22.76T bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 524.23K
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 13.20M
Total bytes received: 2.21M

sent 13.20M bytes received 2.21M bytes 125.81K bytes/sec
total size is 22.88T speedup is 1,484,964.81 (DRY RUN)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1330) [sender=3.2.3]
Script Finished Mar 07, 2024 19:39.59

Full logs for this script are available at /tmp/user.scripts/tmpScripts/__rsync-delete-test/log.txt

 

I can see that there's a LOT of files it's trying to delete.

I will go through the list it's generated & see if they're OK to delete & then let it go ahead & do it's thing.

 

Thanks for the nudge - looks like the right direction.

I'll take a couple of days at least to go through the list & then report back.

Ta

 

 

sdd

[salvdordalisdad]

Still bashing away at this.

Have run that script a dozen times now, takes a looooong time to run, and keeps breaking for one reason or another.

It's deleting a bunch of stuff, but still recording as 27.9TB compared to 23.6TB

The server just lost all its marbles, all the shares, just a blobby mess, so I had to reboot it & restart the script (yet again).

 

THis will be the last attempt, it wil be quicker to scrap the whole thing & start again!

Update to follow.

Edited March 15 by salvdordalisdad
interrupted

[JorgeB]

Since the backup server has the same number of disks I would recommend backing up disk to disk instead, easier to manage, also if you lose a disk you know exactly what data you have now unprotected and need to backup ASAP.

[salvdordalisdad]

Cripes, that took a loooong time & got me not very far.

That was 11 days spent trying that suggestion, and not successful.

I guess it eliminates a variable, but...crikey.

 

So the rsync with delete option has finally finished, no difference in disk size.

 

n2 = 27.5TB

n1 = 23.6TB

 

So what is difference?

I've been through the whole disks.

If I don't get some useful suggestion abotu where to look, I'll have to trash the whole server & start again. Not very happy about that, really dents my appreciation of UNRAID, which has been very positive until this...

 

[salvdordalisdad]

Gave up on finding an answer, so deleted everything.

When "empty" the disks still had some data in ther, system, iso images for vms etc, but not much:

8.5G    ./disk1     64.6GB

8.2G    ./disk2    9.08GB

61G     ./disk3     120GB

0       ./disk4       55GB

CLI                      GUI

However, the GUI still reported somewhat less than empty...when compared to the "du" command on CLI.

I don't care all that much about such a small amount, but do I need to do a "disk check" to clear them down ??

 

 

[salvdordalisdad]

Ah spit.

 

I ran xfs-repair on all the disks & disk3 complained.

So did waht it say, & mounted & dismounted & re-ran, no errors now.

But the disk used sizes didn't change. so Didn't achieve very much at all.

 

But lots of system gone now. No community apps, no user scripts, ahh spit.

Can't download cummunity - unraid version too low.

Really screwed it. Well so be it, it's only hours.

grr

Next time - LEAVE IT ALONE !!!!

 

Edited March 20 by salvdordalisdad
results.

[trurl]

There will always be some space used even on a newly formatted disk due to filesystem overhead.