Solved : Unable to get mover to actually move 400 GB plus from cache to array


dr_dre
Go to solution Solved by trurl,

Recommended Posts

Hello all,

 

im unable to get the mover to move all of my appdata ( dockerstuf etc) to the array.

Cache drive is about 480 GB and 460 GB is filled . I know my docker image is on the cache and that only 80 Gb so thats not a prob.

But i want to get to the array with only new files on the cache drive so it still will be quick and responsive while old data may be on the array.

 

image.thumb.png.7c47359d40e77c316ee4ff0a53e93ac2.png

 

i tried several share settings for the appdata  but mover wont move those files. I did move files from other shares

image.png.3fc8642786abd785fa30cab4d4176253.png

 

Any idea where to start as ive tried all the different setting the data just wont move.

The dockers i installed all point to /mnt/user/appdata share and not the /mnt/cache/appdata

 

 

foxie4-diagnostics-20220218-1235.zip

Edited by dr_dre
Link to comment
3 hours ago, dr_dre said:

docker image is on the cache and that only 80 Gb so thats not a prob

Might be an indication of a different problem. 20GB is often more than enough. Why do you have it so large? A common reason for filling docker.img is an application writing to a path that isn't mapped to host storage.

 

And the fact that you even want to move appdata off cache might also be an indication of a problem. If your appdata is very large you probably have something writing into it. Your downloads, for example, should be mapped somewhere besides appdata.

 

3 hours ago, dr_dre said:

only new files on the cache drive so it still will be quick and responsive

Your dockers/VMs won't perform as well if appdata and system shares are not on fast pool (cache), and they will keep array disks spunup since these files are always open.

Link to comment

the dockers are crypto related and the databases are on the cache. They tend to be very large.

ive got no idea on the speed impact and want to test that.

But as the data isnt moving from cache to array im not able to test that.

 

docker service and VM's are off. Open files cant be the problem. 

 

Your saying it might be some other problem... like ?

 

Link to comment

2 things 

a: it tells me ALL the files from share appdata  already exists from the appdata subdir.

image.thumb.png.78a529b6f43adbb0bcbff2df9097ebab.png

the mover also moves other shares from cache. and does a splendid job

image.thumb.png.ab8afee2654dbf4716c72bdcb3822c72.png

 

I tried to rename a file in the destination from the appdata share  ( before the mover got to that file)  to look in the logfile what he was going to do with it.

 

( /mnt/user/appdata) i renamed 1 file .fanart.jpg to fanart1.jpg. 

as you can see the right (cache) is renamed aswell. so removing files from /mnt/user/appdata also renames it in cache. Therefor im not renaming/deleting anything . as i could lose it all

image.thumb.png.f6eabbc30fbc3e453204d4da7dfdc991.png

 

 

 

 

 

 

 

Edited by dr_dre
Link to comment
30 minutes ago, dr_dre said:

(cache) is renamed aswell

You have a basic misunderstanding of user shares. User shares are simply the combined top level folders on all pools and array disks. So, /mnt/cache/somefolder/somefile and /mnt/user/somefolder/somefile ARE the same file.

 

Probably you already have that same file on some array disk and that is what mover is complaining about.

 

Also, don't try to work with user shares and disks at the same time, linux doesn't understand user shares and move/copy could actually cause data loss if you specify a source and destination that are actually the same file.

Link to comment
27 minutes ago, trurl said:

 

Probably you already have that same file on some array disk and that is what mover is complaining about.

 

 

its all the files on the cache drive from appdata its complaining about !

regarding the mention about disks i looked at the /mnt/disk1 to 12 and the appdata files are not there. (ive found 4 from today  ( the new files that were created as ive set the share to use cache no to test things out.) There were no other files on any of the disks 

 

SO: mover thinks the files  ( all files from appdata share)  it needs to move, already exists on the array.

Which is not correct as ive checked every disk(1-12) for the files and they arent there .

 

How can i convince the mover that he actually needs to copy the files to the array

 

p.s. ive looked at /mnt aswell and ive noticed there is  a /mnt/user and a /mnt/user0 with different contents  is this normal ?

 

 

Link to comment

/mnt/user is the top level folders on all pool (cache) and array disks. /mnt/user0 is the top level folders on only the array disks.

 

Mover moves files from /mnt/cache (or whatever pool is specified for the share) to /mnt/user0 for cache-yes shares.

 

Mover moves files from /mnt/user0 to /mnt/cache (or whatever pool is specified for the share) for cache-prefer shares.

 

Go to Shares - User Shares, click Compute... for the appdata share, wait for the result, then post a screenshot.

Link to comment

Like the past 5 times. nothing..

everything is up and running , mover is busy for 15 mins and says its done.. but dint move anything and log files tells me that it already exists on array but it isnt.

 

the contents from /mnt/user and /mnt/user0 is my witness as there is a difference aswell

 /mnt/user tells me it exists on cache and

/mnt/user0 tells me it doesnt exist on the array

Edited by dr_dre
Link to comment
  • dr_dre changed the title to Solved : Unable to get mover to actually move 400 GB plus from cache to array

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.