August 21, 20223 yr Hello I'm having an issue moving files from cache to the array. I have mover set to hourly, and it was working great with my data migration until it didn't. After it wrote the initial amount to the array, I had 330gb on the array. Now when it ran I have 178gb on my cache selected and 256 on the array. I have the share setup with Cache: Yes Device: vmstorage which is my larger ssd until I move all then my cache will be a 500gb and the 1tb will go to VM storage. My understanding is that yes prefer will keep it on the cache whole yes will initially write to it then let mover move it to the array.
August 21, 20223 yr Author So it's working now after I downloaded CA Mover Tuning. Not sure why but everything is being moved from cache to the array. Would love is someone knows why it was working on reverse without that plugin.
August 21, 20223 yr Community Expert Probably some misunderstanding. A few things to consider about mover: Nothing can move open files Mover won't move duplicates Mover moves cache:yes shares from the designated pool to the array Mover moves cache:prefer shares from the array to the designated pool Mover ignores cache:only and cache:no shares Mover ignores files on pools other than the designated pool Ideally, you want appdata, domains, system shares all on a fast pool and set to stay on a fast pool. If these files are on the array, docker/VM performance is impacted by slower array, and array disks can't spin down since these files are always open. Some things about your current configuration that are less than ideal: Your appdata share has files on the array, and it is set to move all files from your cache pool to the array. Probably some can't be moved because they are open. You want it all on a fast pool. Your domains share has all its files on your cache pool, but it is set cache:no, so mover will ignore it, and any new files will go to the array. You want it all on a fast pool. Your system share has files on the array, and it is cache:no, so mover will ignore it and any new files will go to the array. You want it all on a fast pool. You have another share, anonymized as d--a, it is cache:yes and vmstorage is its designated pool. It has some files on your cache pool though, so mover will ignore those. New files will go to vmstorage, and mover will try to move files from vmstorage to the array. Nothing can move open files so even if you get those shares setup correctly, they can't be moved until you disable Docker and VM Manager in Settings. Let us know if you need help getting those setup.
August 22, 20223 yr Author 6 hours ago, trurl said: Probably some misunderstanding. A few things to consider about mover: Nothing can move open files Mover won't move duplicates Mover moves cache:yes shares from the designated pool to the array Mover moves cache:prefer shares from the array to the designated pool Mover ignores cache:only and cache:no shares Mover ignores files on pools other than the designated pool Ideally, you want appdata, domains, system shares all on a fast pool and set to stay on a fast pool. If these files are on the array, docker/VM performance is impacted by slower array, and array disks can't spin down since these files are always open. Some things about your current configuration that are less than ideal: Your appdata share has files on the array, and it is set to move all files from your cache pool to the array. Probably some can't be moved because they are open. You want it all on a fast pool. Your domains share has all its files on your cache pool, but it is set cache:no, so mover will ignore it, and any new files will go to the array. You want it all on a fast pool. Your system share has files on the array, and it is cache:no, so mover will ignore it and any new files will go to the array. You want it all on a fast pool. You have another share, anonymized as d--a, it is cache:yes and vmstorage is its designated pool. It has some files on your cache pool though, so mover will ignore those. New files will go to vmstorage, and mover will try to move files from vmstorage to the array. Nothing can move open files so even if you get those shares setup correctly, they can't be moved until you disable Docker and VM Manager in Settings. Let us know if you need help getting those setup. That is a lot of good information and will help me out. But at the time I was seeing the issue, none of my shares were set to yes: prefer was the domain. I will set appdata to my prefer my cache, domain will be on vmstorage and set to preferred, will set system to prefer cache. The anonymized one is probably my data directory that I have all my storage for my old Dockers (not appdata) that contains my media and storage files. One thing I would question is if system should be on a fast pool, why is it default to no? As of right now everything moved correctly to where I had it placed. Not that my initial data migration is complete, I've added the parity drive and running the initial parity. Edit - for the initial data migration I was reading about having everything go to cache then let mover deal with it to speed up. So I did take everything off cache for more room. I think some of my issues were that my initial cache was 500gb and filling way before the hour. After I got the mover plugin and set it with vmstorage and also added to start it at 75% made it way better. Edited August 22, 20223 yr by ZerkerEOD Added explanation of why no caches in use.
August 22, 20223 yr Community Expert 16 hours ago, ZerkerEOD said: if system should be on a fast pool, why is it default to no? appdata, domains, system default to prefer so you must have changed it to no. Also, if you enable dockers / VMs before you install cache these prefer shares will be created on the array. 16 hours ago, ZerkerEOD said: initial data migration Best idea is to NOT cache for initial data load, except for those things you want to stay on cache or other pool. Mover doesn't really help if you are transferring more data than cache can hold. It is impossible to move from fast cache to slower array as fast as you can write to fast cache. Mover is intended for idle time. I personally don't bother with the mover tuning plugin, and it won't really help anything if you are writing new data to cache and hoping it will get moved before you fill it up. One thing that many do for initial data load is to not install parity until initial load is complete, so writes to the array will be faster.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.