Given that unRAID allows the loss of multiple drives with only the loss of the content on those particular drives, it would be a nice feature to be able to read off a catalog of exactly which files have been lost in such an event.

 

I use my 5 disk 8TB unRAID array to store DVD ISO files, which have been ripped from my own collection. Given that this data could be easily (if somewhat time-consumingly) retrieved without needing a backup, it would be nice to know which DVDs I need to rip to replace the data on the lost disk.

 

I'd imagine that storing a full directory tree structure in a text file replicated on each disk in the array (or on the Flash drive) would be a fairly trivial task?

 

Maybe this could be set up as a script which runs on a Cron job once a day?

 

I'd look into it myself, but I'm afraid my Linux skills are not as good as they might be.

 

Thanks.

I wrote a cron script to do this a while ago. I'm not using it now, but I'll see if I can find it.

That would be great if it's no great effort, thanks.

I had an idea I was working on like this a while back. Maybe I'll resurect it.

In any case it worked like the updatedb/locate faciltiy only it used an internal sqlite database.

 

My first goal was a duplication of the locate function so you can find files across your whole server.

 

It also kept other information about the file such as mtime. permissions and I was adding the ability for a md5sum.

This would allow you to detect changes/corruptions and/or identify a file if your hard drive was corrupt/repaired via a fsck with lost+found files.

 

I'll restart on this once the summer winds down.

 

 

I can't find it, but it was very simple. I think it was just using the "ls" command redirecting to a text file. I did use command line switches to drill down through subdirectories.

I have a Windows Batchfile that I use to query against the SMB drive shares. I don't have access to it right now but it's something like:

dir /s \\nas\disk1 > d:\disk1.txt

dir /s \\nas\disk2 > d:\disk2.txt

... etc ...

 

Then I merge them together:

copy /a d:\disk1.txt + d:\disk2.txt + etc + d:\disk7.txt d:\nas.txt

 

Then I delete the diskx.txt files.

del d:\disk1.txt

del d:\disk2.txt

... etc ...

I have a script I run that does this... should really get it set up in cron so that it can be run once a week or something like that.

 

This is not overly hard to do.  An ls -r on all folders under /mnt/user/ would return the content for each user share.

Yes, I can see that it should be very simple. Piping the output of a tree command to a file once a day was what I had in mind. Unfortunately, Being a total Linux novice, I don't even know which folders to use to store things like scripts. I haven't worked at the command line for many years, and find it all a bit tedious. Especially the default text editors.

Here's my /etc/cron.daily/updatefilelist cron job.

 

the -print can be changed to be -ls on the second find below.

The output directory must already exist.

 

mkdir -p /mnt/cache/.local/var/db/filelist

I suppose I could update the script, but this is what I have in place right now.

 

 

root@atlas /etc/cron.daily #more /etc/cron.daily/updatefilelist

 

#!/bin/bash

 

find /mnt -name 'disk*' -type d -maxdepth 1  | sort | while read DIR

do    find ${DIR} -type f -o -type d -print

done > /mnt/cache/.local/var/db/filelist

 

Here's what I use, just copy into the cron weekly folder, catalogs each disk into its own txt file then packs them all up with that weeks date.  Also, I am not a programmer so please be kind

 

#!/bin/bash

############################################
#                                          #
# Script to take a listing of all files    #
# on data disks in /mnt                    #
#                                          #
############################################


SAVEIFS=$IFS
IFS=$(echo -en "\n\b")

DATE=`date +%Y-%m-%d`

for (( i=1; $i <= 20; i++))

do

    for file in $(find /mnt/disk$i -type f)

    do

    #Will output checksum on each file but takes a long time, not practical
    #echo -ne $(md5sum "$file" | cut -f 1 -d " ") " "

    ls -l -g -G -h $file | cut -d ' ' -f3-
    
    # output to file and screen for testing
    #done 2>&1 | tee -a /boot/logs/filelist/filelist-$DATE-disk$i.txt
    
    done >> /boot/logs/filelist/filelist-$DATE-disk$i.txt

done

zip /boot/logs/filelist/filelist-$DATE /boot/logs/filelist/filelist-$DATE*

rm /boot/logs/filelist/*.txt

IFS=$SAVEIFS