I am currently using restic to backup about 9TB of data which is my entire array + appdata + cache files. Basically everything under `/mnt/user`. I have been noticing that my backups have taken a gigantic amount of time to backup (14hrs+ with no end in sight). I have this thread going on the restic forums: https://forum.restic.net/t/restic-takes-a-long-time/8409/11

 

They are leading me to realize that using a fuse-based filesystem is the cause and specifically due to shfs which is painfully slow. I want to understand how does Unraid handle inodes on its mount points? Seems my issue is due to inodes not being stable on any fuse-based filesystem which means every file is seen as "new" and has to be re-scanned.

Edited September 5, 20241 yr by Ustrombase
Updated title to better explain the issue at hand.

Anyone?

I doubt this is anything forum members will know the answer to.   You should contact Limetech directly to get anything other than a guess at the answer.

On 9/5/2024 at 10:42 AM, Ustrombase said:

I am currently using restic to backup about 9TB of data which is my entire array + appdata + cache files. Basically everything under `/mnt/user`. I have been noticing that my backups have taken a gigantic amount of time to backup (14hrs+ with no end in sight). I have this thread going on the restic forums: https://forum.restic.net/t/restic-takes-a-long-time/8409/11

 

They are leading me to realize that using a fuse-based filesystem is the cause and specifically due to shfs which is painfully slow. I want to understand how does Unraid handle inodes on its mount points? Seems my issue is due to inodes not being stable on any fuse-based filesystem which means every file is seen as "new" and has to be re-scanned.

I believe with fuse mounts, which is what unRAID uses for the array by default, the inode ids are generated on start/mount. So every time you reboot unraid, the inode ids of the files will change. 

 

Restic uses multiple things to determine if a file is unchanged and one of those things is the inode id.

 

On unRAID, every time you reboot, most if not all files will be recognized as changed by restic, leading it to update metadata for every single file in the backup, which causes really long backup times.

 

Restic recommends using the arg "--ignore-inode" on fuse mounts so it won't incorrectly recognize the files as modified when inode ids change. It will continue relying on other things like mtime.

 

https://restic.readthedocs.io/en/latest/040_backup.html#file-change-detection

On 1/25/2025 at 11:52 PM, dumurluk said:

...the inode ids are generated on start/mount.  

Restic recommends using the arg "--ignore-inode" on fuse mounts so it won't incorrectly recognize the files as modified when inode ids change. It will continue relying on other things like mtime.

 

https://restic.readthedocs.io/en/latest/040_backup.html#file-change-detection

Wanted to add my experience here. I was not sure if --ignore-inode was going to work because of this statement the inode ids are generated on start/mount. I was not re-starting or re-mounting anything and backups were slow between runs.

I believe this statement is true from the Restic documentation that using inodes on FUSE systems are simply unreliable: The option --ignore-inode exists to support FUSE-based filesystems and pCloud, which do not assign stable inodes to files.

---

Here are my stats that prove, --ignore-inode is required for using restic for any Unraid file shares using the mount /mnt/user . I went from almost 2 hours to for subsequent backups from 178,000 files (~1TB), to 4 minutes

Command:

restic --verbose --tag vault --exclude-file $excludefile backup "/mnt/user/vault"

Results

restic thinks 90k files were changed, but running a restic diff proved that they were not.

start backup on [/mnt/user/vault]
scan finished in 82.535s: 178532 files, 973.427 GiB

Files:           0 new, 90296 changed, 88236 unmodified
Dirs:            0 new,  1530 changed,  9316 unmodified
Data Blobs:      0 new
Tree Blobs:   1504 new
Added to the repository: 89.162 MiB (28.621 MiB stored)

processed 178532 files, 973.427 GiB in 1:46:22

Command:

restic --ignore-inode --verbose --tag vault --exclude-file $excludefile backup "/mnt/user/vault"

Results

start backup on [/mnt/user/vault]
scan finished in 49.176s: 178533 files, 973.427 GiB

Files:           1 new,     1 changed, 178531 unmodified
Dirs:            0 new,  1533 changed,  9313 unmodified
Data Blobs:      2 new
Tree Blobs:   1507 new
Added to the repository: 91.126 MiB (29.021 MiB stored)

processed 178533 files, 973.427 GiB in 4:06

Thank you both! I have been running restic for months now and just forgot to report back but I agree --ignore-inodes does indeed work here and has made a world of difference!I need to pull the latest stats on this but it hides in the background for the most part.