Data missing, mover running

[Unthred]

I have no idea when it happened sometime in the last week or so but I have lost access to about 12TB of movies, the folders are empty but when I do a du or a df the both show the space is still used.

I am really hoping the data is still there

I am quite new to unraid and struggling to find any issue that points to what is wrong. There does not seem to be anything in the logs that I can find, no errors in the gui

When I noticed the issue I did a quick google and someone suggested a reboot, my unraid server hung during the reboot, had to go hang a monitor on it to see what was wrong

Stopping md/unraid driver:
/usr/local/sbin/mdcmd: line 11: echo: write error: Deviceor resource busy

After waiting an hour and a hard reset the server came back up no problem, data still missing and automatically started a parity check. The parity check came back fine so I went to do a check on the filesystem for which the array needs to be stopped this is when I noticed the mover is running and will not let me stop the array. There was an error in the log about mover

Apr 27 14:04:20 Heart-Of-Gold move: error: move, 396: No such file or directory (2): lstat: /mnt/cache/FTP/TV Shows/Sleepy Hollow/Season 2/Sleepy Hollow - S02E01 - This Is War Bluray-1080p.nfo
Apr 27 14:04:20 Heart-Of-Gold move: error: move, 396: No such file or directory (2): lstat: /mnt/cache/FTP/TV Shows/Sleepy Hollow/Season 2/Sleepy Hollow - S02E01 - This Is War Bluray-1080p-thumb.jpg
Apr 27 14:04:21 Heart-Of-Gold move: error: move, 396: No such file or directory (2): lstat: /mnt/cache/FTP/TV Shows/Sleepy Hollow/Season 2/Sleepy Hollow - S02E02 - The Kindred Bluray-1080p.nfo
Apr 27 14:04:21 Heart-Of-Gold move: error: move, 396: No such file or directory (2): lstat: /mnt/cache/FTP/TV Shows/Sleepy Hollow/Season 2/Sleepy Hollow - S02E02 - The Kindred Bluray-1080p-thumb.jpg
Apr 2

 

I think the data is still there because of the following

df shows (ignoring the messy docker bits)

Filesystem      Size  Used Avail Use% Mounted on
rootfs           12G  893M   11G   8% /
tmpfs            32M  948K   32M   3% /run
devtmpfs         12G     0   12G   0% /dev
tmpfs            12G     0   12G   0% /dev/shm
cgroup_root     8.0M     0  8.0M   0% /sys/fs/cgroup
tmpfs           128M  5.4M  123M   5% /var/log
/dev/sda1       7.3G  526M  6.7G   8% /boot
/dev/loop0      7.5M  7.5M     0 100% /lib/modules
/dev/loop1      4.5M  4.5M     0 100% /lib/firmware
/dev/md1        3.7T  2.8T  931G  76% /mnt/disk1
/dev/md2        3.7T  2.8T  882G  77% /mnt/disk2
/dev/md3        3.7T  2.4T  1.3T  66% /mnt/disk3
/dev/md4        7.3T  5.5T  1.9T  76% /mnt/disk4
/dev/sdf1       699G  156G  542G  23% /mnt/cache
shfs             19T   14T  4.9T  74% /mnt/user0
shfs             19T   14T  5.4T  72% /mnt/user
/dev/loop2       50G   27G   21G  57% /var/lib/docker

if I choose one of the drives

du --max-depth=1 -h

17G	./TV Shows
5.1G	./downloads
0	./src
39G	./Movies
0	./pictures
2.8T	./FTP
80K	./Useful Docs
2.8T	.

Now that movie folder should have around 350 HD movies in it. The total 2.8T at the bottom really does not add up to what is in those folders

 

Anyone got any suggestions on how to diagnose this further? and hopefully recover my data?

 

Thanks

[JorgeB]

Please post your diagnostics: Tools -> Diagnostics

[Unthred]

Sorry should have thought to do that 

 

heart-of-gold-diagnostics-20180428-1433.zip

[JorgeB]

Don't see nothing out of the ordinary on the logs.

 

Why is this folder 2.8TB, maybe you move your data there?

59 minutes ago, Unthred said:

2.8T ./FTP

 

[John_M]

1 hour ago, Unthred said:

if I choose one of the drives

du --max-depth=1 -h

17G	./TV Shows
5.1G	./downloads
0	./src
39G	./Movies
0	./pictures
2.8T	./FTP
80K	./Useful Docs
2.8T	.

Now that movie folder should have around 350 HD movies in it. The total 2.8T at the bottom really does not add up to what is in those folders

 

Yes it does. You said you are looking at just one of the drives. The total of 2.8 TB used corresponds to that being either disk1 or disk2. Your "missing" files are on the other disks. You're looking in the wrong place. Instead of looking under /mnt/diskX look under /mnt/user and run your du command from there.

 

Do you understand how user shares work? The contents of, say, /mnt/user/Movies is the union of /mnt/disk1/Movies, /mnt/disk2/Movies, etc.

[Unthred]

Thanks for the responses guys...... when I looked before it crashed there wasnt that in FTP folder.......

So that was a change I made a week or two ago the fix common problems plugin was complaining that the FTP folder was not on the cache drive....so I put it on the cache drive......seems the mover has been moving my movies and TV shows into the FTP folder...... I dont understand why.

 

But At least I have found my data and have a direction to investigate!

 

Thanks again

[Unthred]

Thanks for the responses guys...... when I looked before it crashed there wasnt that in FTP folder.......

So that was a change I made a week or two ago the fix common problems plugin was complaining that the FTP folder was not on the cache drive....so I put it on the cache drive......seems the mover has been moving my movies and TV shows into the FTP folder...... I dont understand why.

 

But At least I have found my data and have a direction to investigate!

 

How can I stop the mover?

 

Thanks again

[itimpi]

The mover will never put items into a different folder.   All it will do is move a top level folder (and contents) corresponding to a share name to/from the cache drive according to the Share settings into the same top level folder on the destination.