An index of files on each disk, in case of data loss

[-Daedalus]

Something that occurred to me the other day. If an array loses n+1 drives to failure before the array can be rebuilt, then you've lost at least 1 drive worth of data.

At the moment, we're trusting unRAID with what to put what file, and that's great, but it could make things very tricky in the event of disaster recovery.

 

Some of my shares are split logically - seasons of a TV show, for example - but even then, that's looking through hundreds of TV shows to see what, if any, seasons are missing. In the case of shares where the split is handled entirely by unRAID, any random assortment of files could be gone.

 

I imagine it's wouldn't be too high on the list of priorities, but if you do happen to need it one day, it would be a life-saver.

 

So, tl;dr - I'd like to request some sort of index of each file, stored on each disk (cache, array, and UD).

Bare minimum could be a text file per disk. Ideally, it would be tree-based with some sort of simple UI, and unRAID would check against files actually on the disks, and highlight missing items in the event of data loss.

[Addy90]

You can fake this over File Integrity plugin. There you get txt files with each file from each drive with a hash sum. Your plus here is that when you have drive failure and rebuild a drive, you can check the rebuilt files for validity with the previously exported hash sums. You can also automate the process of calculating hash sums of new files and export them as txt file. It is no tree, but it has each file in it. If not automatically, you export it manually (I do it manually, because automatically drastically drops write speed).

Not perfect, but maybe good enough for you?

Edited March 1, 2020 by Addy90

[hawihoney]

I'm using the User Scripts plugin to issue this command for every disk on a daily base.

 

tree /mnt/disk1 > /mnt/<your Location>/disk01.txt

 

[-Daedalus]

10 hours ago, Addy90 said:

You can fake this over File Integrity plugin. There you get txt files with each file from each drive with a hash sum. Your plus here is that when you have drive failure and rebuild a drive, you can check the rebuilt files for validity with the previously exported hash sums. You can also automate the process of calculating hash sums of new files and export them as txt file. It is no tree, but it has each file in it. If not automatically, you export it manually (I do it manually, because automatically drastically drops write speed).

Not perfect, but maybe good enough for you?

 

Good shout, I meant to look at this a while back, forgot to. I'll absolutely be using this.

 

8 hours ago, hawihoney said:

I'm using the User Scripts plugin to issue this command for every disk on a daily base.

 

tree /mnt/disk1 > /mnt/<your Location>/disk01.txt

 

 

 

I cannot believe I did not think of this.

That is enough of a solve for me personally to be happy, but I stand by the feature request for something native in the UI.

 

Thanks to you both!