Cache Mover - symbolic links

[bah1976]

Ok - here's my situation.  I've got a typical unRaid setup, with a cache drive.  I have a share exported via NFS to my mythtv server to store TV recordings.  (mythtv is DVR software) No problems there - myth writes/reads without any issue over my GB network. 

After a TV show records, a script runs to move the randomly named *.mpg file into an episode-based file. 

For example, the file that used to reside at:

/mnt/beast/mythtv/recordings/1051_20131030235900.mpg is renamed and moved to

/mnt/beast/mythtv/recordings/Episodes/Revolution/Season 2/Revolution S02E06 (Dead Man Waking).mpg

 

I do this so my plex server can index the /Episodes folder and do what Plex does.

Then, because myth isn't that smart and will continue to look for the original .mpg, it creates a symbolic link: 

1051_20131030235900.mpg -> /mnt/beast/mythtv/recordings/Episodes/Revolution/Season 2/Revolution S02E06 (Dead Man Waking).mpg

 

All this works fine - this is the setup to my question, which is:

These symbolic links, which are ending up on my cache drive as they should when they're created, will not move to the protected array.  I suspect it's because of the link target - if I look at /mnt/cache/mythtv/recordings, I see all the links that should reside at /mnt/user/mythtv/recordings - but they have the naming scheme from /mnt/beast/mythtv.  Is there any way to make the mover actually move these links?

 

My user share is 'mythtv'.  my unRaid is 'beast', hence me mounting /mnt/user/mythv on my myth box as /mnt/beast/mythtv.  Is that my problem?  If I were to remount on my myth box as /mnt/user/mythtv - would it work?

 

I didn't realize all this was going on until I had to replace my cache drive due to smart errors - and I had to manually re-mount the old cache drive and copy the links over to the new cache drive to get my mythtv setup to not freak out.

I'd look forward to any thoughts here - thanks in advance..

 

[bah1976]

Am I really the only one with symbolic links on my array?  I figured somebody would know the answer..

[bah1976]

Ok, so I've looked into the rsync command that's used in the mover script.  It seems it only copies regular files or directories.  I'm working on syntax to sync the links as well - will update here once I've gotten it - or if anybody has input on rsync/symlink syntax?

Original code was this:

(cd /mnt/cache ; find -depth -print \
\( -type f -regex '[.]/[^.].*/.*' ! -exec fuser -s {} \; \
    -exec rsync -i -dIWRpEAXogt --numeric-ids --inplace --remove-source-files {} /mnt/user0/ \; \) -o \
\( -type d -regex '[.]/[^.].*' \
    -exec rsync -i -dIWRpEAXogt --numeric-ids --inplace {} /mnt/user0/ \; -empty -delete \) \
)

 

I've modified it to this:

(cd /mnt/cache ; find -depth -print \
\( -type f -regex '[.]/[^.].*/.*' ! -exec fuser -s {} \; \
    -exec rsync -i -dIWRpEAXogt --numeric-ids --inplace --remove-source-files {} /mnt/user0/ \; \) -o \
\( -type l -regex '[.]/[^.].*/.*' ! -exec fuser -s {} \; \
    -exec rsync -i -dIWRpEAXogtlH --numeric-ids --inplace --remove-source-files {} /mnt/user0/ \; \) -o \
\( -type d -regex '[.]/[^.].*' \
    -exec rsync -i -dIWRpEAXogt --numeric-ids --inplace {} /mnt/user0/ \; -empty -delete \) \
)

which has resulted in copying of all the symlinks to the array, but they remain on the cache drive as well.

 

if I run

rsync -i -dIWRpEAXogtlH --numeric-ids --inplace --remove-source-files /mnt/cache/folder/linkname /mnt/user0/folder

manually, it copies the link, removes it from the cache, and I get a couple errors in syslog:

Nov  5 13:09:04 BEAST shfs0: shfs_utimens: utimes: /mnt/disk4/mythtv/recordings/1091_20130917235900.mpg (2) No such file or directory
Nov  5 13:09:04 BEAST shfs0: shfs_listxattr: listxattr: /mnt/disk4/mythtv/recordings/1091_20130917235900.mpg (2) No such file or directory

 

When I run the mover script, which I think does the same thing I did manually, I get those same errors plus a couple extra, and the file stays on the cache drive, which now generates a bunch of "duplicate object" errors on the log.

 

sigh..  any thoughts or do I continue to be on my own?

[bah1976]

Update - day #3..  It looks like even though the links remain on the cache drive initially after the mover runs - some other process is coming in and deleting the duplicate objects.  Is that by design?

[Zan]

Thanks bah1976,

 

I noticed this issue too and will try your 'lH' change to the rsync command on unRAID 6 beta5a. Will update here on how it goes.