Jump to content
Sign in to follow this  
lewispm

command line file moving

12 posts in this topic Last Reply

Recommended Posts

I created a user script that invokes a "mv" command in the command line to move files from a share with a "use cache: yes" preference to a share with "use cache: no"  preference.

 

The mv command syntax is this:  

mv /mnt/user/shareUsingCache/folder/* /mnt/user/shareNotUsingCache/folder/

 

I thought using the /mnt/user would allow the software to place the files in their correct location based on the share rules (use cache:no).  However, after the move, the files were on the cache drive.  I thought the "mover" would fix this, but after the move they are still on the cache.

 

As a temporary fix, I have changed the setting on the destination share to "use cache:yes" so now the mover should move them off the next time it runs.  I would like to know what I am doing wrong, and a way to move these files via a command line script that will honor the share preferences.

 

Thanks.

Share this post


Link to post

When moving between paths that are at the same mount (/mnt/user) linux simply renames them so the directories are changed on the same disk instead of copying to the destination disk and then deleting from the source disk. That explains why the files were still on cache after the mv.

 

And because the mv resulted in them now being in a cache-no user share, mover won't touch them, since it only moves cache-yes shares from cache to array, and cache-prefer share from array to cache.

 

Also it might be worth noting that "honoring" user share settings only applies to writing of new files. unRAID never automatically does anything to files that have already been written except as the result of the already explained actions of the mover.

 

You would have to copy from source user share to destination user share to get it to honor the (write) settings of the destination user share, then delete them from the source user share.

Share this post


Link to post

And since you are working at this level, you should also be aware that you must never mix disks and user shares when moving/copying since that can result in data loss when it tries to overwrite the file at the same physical disk location it is trying to copy it from. Linux doesn't know that /mnt/user/somepath might actually be /mnt/disk1/somepath, for example.

Share this post


Link to post
Quote

When moving between paths that are at the same mount (/mnt/user) linux simply renames them so the directories are changed on the same disk instead of copying to the destination disk and then deleting from the source disk. That explains why the files were still on cache after the mv. 

 

And because the mv resulted in them now being in a cache-no user share, mover won't touch them, since it only moves cache-yes shares from cache to array, and cache-prefer share from array to cache.

Makes sense.

 

Quote

Also it might be worth noting that "honoring" user share settings only applies to writing of new files. unRAID never automatically does anything to files that have already been written except as the result of the already explained actions of the mover.

 

You would have to copy from source user share to destination user share to get it to honor the (write) settings of the destination user share, then delete them from the source user share.

 

So the "mv" command doesn't write the files to the disk (in the sense that you are describing), that occurred when they were placed in the original share?

 

But a "cp" command would cause a write action on the share, "honoring" the preferences, correct?

 

Quote

And since you are working at this level, you should also be aware that you must never mix disks and user shares when moving/copying since that can result in data loss when it tries to overwrite the file at the same physical disk location it is trying to copy it from. Linux doesn't know that /mnt/user/somepath might actually be /mnt/disk1/somepath, for example.

My /mnt/user in both source and destination is the safe and correct way to do this task, then, right?

 

Thanks for all the info.  It is very helpful!

Share this post


Link to post
Quote

And because the mv resulted in them now being in a cache-no user share, mover won't touch them, since it only moves cache-yes shares from cache to array, and cache-prefer share from array to cache.

 

Since I get a warning from the "fix common problems" plugin about files on the cache disk in a share that has "use cache:no" wouldn't it make sense for the mover to move files from "use cache:no" shares to the array?   Would it be harmful for the mover to do this?

 

Share this post


Link to post

Not harmful per se, but there are valid reasons to have files on the cache drive when the share is set to not use the cache. The link is the FCP warning about it

Sent from my LG-D852 using Tapatalk

Share this post


Link to post
4 hours ago, lewispm said:

My /mnt/user in both source and destination is the safe and correct way to do this task, then, right?

Yes

 

Here is a link to the FAQ that goes into a lot more detail about how the various cache settings work. Some of these details aren't necessarily obvious without some study:

 

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?page=2#comment-537383

 

Share this post


Link to post

Ok, now I need help fixing this problem.  

 

The data is in the correct share, but orphaned, and won't be moved. 

 

Is it safe to move the files disk share to disk share?  So I copy from the correct share on the cache drive to the correct share on one of the array drives?

 

Then I can delete the orphaned directory on the cache drive, right?

 

 That seems like my only option.

Edited by lewispm

Share this post


Link to post
2 hours ago, lewispm said:

Ok, now I need help fixing this problem.  

 

The data is in the correct share, but orphaned, and won't be moved. 

 

Is it safe to move the files disk share to disk share?  So I copy from the correct share on the cache drive to the correct share on one of the array drives?

 

Then I can delete the orphaned directory on the cache drive, right?

 

 That seems like my only option.

It is safe to do this.    However it might be easier to temporarily set the orphaned share to Use Cache=Yes and then run mover which will achieve the same effect.   When mover completes the move you can set the share back to setting you want.

Share this post


Link to post
7 hours ago, itimpi said:

It is safe to do this.    However it might be easier to temporarily set the orphaned share to Use Cache=Yes and then run mover which will achieve the same effect.   When mover completes the move you can set the share back to setting you want.

That was my plan, but a reply above says mover won't do this move.  And I can confirm this, as I already set cache =yes and run the mover,and it's still orphaned

Share this post


Link to post
2 hours ago, lewispm said:

That was my plan, but a reply above says mover won't do this move.  And I can confirm this, as I already set cache =yes and run the mover,and it's still orphaned

I have frequently done this without problems!  I also see no reply that suggests this would not work (since despite its name mover does not use the ‘mv’ command to move files).  If it not working this suggests some other issue is coming into play.   It might be worth activating mover logging and trying again to see if useful messages saying why it is not working appear in the syslog.

Edited by itimpi

Share this post


Link to post
44 minutes ago, itimpi said:

I have frequently done this without problems!  I also see no reply that suggests this would not work (since despite its name mover does not use the ‘mv’ command to move files).  If it not working this suggests some other issue is coming into play.   It might be worth activating mover logging and trying again to see if useful messages saying why it is not working appear in the syslog.

You are right, it worked.  Now I switched it back to use cache:no.

 

Thanks, I misunderstood a reply earlier.

 

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this