Mover been running almost and hour and nothing is moving - No errors found when moving.

[ZerkerEOD]

I am at my wits end. I have been trying to run mover for a couple hours. I have disabled VM's and Docker and it wasn't moving stuff from the cache to the array. I upgraded to 6.12.3 and the same issues persists. I thought it could be because some of the shares have NFS attached so I disabled them. I rebooted a couple times. I also enabled the logging through scheduler (and unless I am missing other logs) there are no errors. I made all my shares yes:cache or yes:vmstorage trying to move them off of them so that I can swap out the NVMe drives with new ones, but they won't move.

 

Just to add with the new version under share it sows everything should move to the array but nothing is moving. I see where mover starts and every hour mover tries to start again and says that it is already running but it isn't. All my drives spin down after so long and I have forced them back on a couple times.

 

Diagnostics attached and I hope someone can help out.

nasty-diagnostics-20230715-1924.zip

Edited July 15, 2023 by ZerkerEOD
Added information

[itimpi]

18 minutes ago, ZerkerEOD said:

I am at my wits end. I have been trying to run mover for a couple hours. I have disabled VM's and Docker and it wasn't moving stuff from the cache to the array. I upgraded to 6.12.3 and the same issues persists. I thought it could be because some of the shares have NFS attached so I disabled them. I rebooted a couple times. I also enabled the logging through scheduler (and unless I am missing other logs) there are no errors. I made all my shares yes:cache or yes:vmstorage trying to move them off of them so that I can swap out the NVMe drives with new ones, but they won't move.

 

Just to add with the new version under share it sows everything should move to the array but nothing is moving. I see where mover starts and every hour mover tries to start again and says that it is already running but it isn't. All my drives spin down after so long and I have forced them back on a couple times.

 

Diagnostics attached and I hope someone can help out.

nasty-diagnostics-20230715-1924.zip 127.52 kB · 0 downloads

 

The diagnostics shows that there IS a mover process running.

 

However it also shows that you are using the Mover Tuning plugin so you might want to try removing (or disabling) this in case it is causing problems in some way/

[ZerkerEOD]

5 minutes ago, itimpi said:

 

The diagnostics shows that there IS a mover process running.

 

However it also shows that you are using the Mover Tuning plugin so you might want to try removing (or disabling) this in case it is causing problems in some way/

I disabled it after and it didn't do anything. I found another topic that told me how to stop mover. I already reenabled it, but I was able to start the mover on a single share. So I am going to manually start the mover from the share menu until I find which one was causing issue. Can the System share be moved or does that require any special methods?

[itimpi]

1 minute ago, ZerkerEOD said:

Can the System share be moved or does that require any special methods?

That requires the docker and VM services to be disabled in Setting as they keep files open that will stop files being moved.

[ZerkerEOD]

Just now, itimpi said:

That requires the docker and VM services to be disabled in Setting as they keep files open that will stop files being moved.

Both of them are disabled, I did that first because I wanted to move appdata. I have VM's on them that are off but they are managed through proxmox with the NFS connection (NFS is disabled now too). So VM is always off since I don't run VM's straight from unraid. Sounds like I will move each share manually until I see one that just freezes.

[ZerkerEOD]

@itimpi I was able to finally get everything moved. I had to delete my snapshot directory because the snapshots were readonly. But I had them backed up offsite anyway. My issue now that I have the new drives in and everything is on the array, when I try to change the settings so that mover will move from array to vmstorage, the option isn't there. If I change Primary storage Array where everything is at, it won't let me select either my cache or vmstorage drives for secondary storage. So mover won't do anything. Do you know anything about this?

[itimpi]

23 minutes ago, ZerkerEOD said:

@itimpi I was able to finally get everything moved. I had to delete my snapshot directory because the snapshots were readonly. But I had them backed up offsite anyway. My issue now that I have the new drives in and everything is on the array, when I try to change the settings so that mover will move from array to vmstorage, the option isn't there. If I change Primary storage Array where everything is at, it won't let me select either my cache or vmstorage drives for secondary storage. So mover won't do anything. Do you know anything about this?

 

You need to set the pool as Primary storage (i.e. the location where new files get put) , and then the array as secondary storage. Then set the mover direction to be array->cache.

[ZerkerEOD]

18 minutes ago, itimpi said:

 

You need to set the pool as Primary storage (i.e. the location where new files get put) , and then the array as secondary storage. Then set the mover direction to be array->cache.

When I do that I can't run Mover on the share anymore. I also when trying to move files manually I get 

 

mv /mnt/user/domains /mnt/vmstorage/domains
mv: cannot remove '/mnt/user/domains/images/102': Directory not empty
mv: cannot remove '/mnt/user/domains/images/103': Directory not empty
mv: cannot remove '/mnt/user/domains/images/104': Directory not empty

 

[itimpi]

That will always fail as /mnt/vmstorage/domains is part of /mnt/user/domains so you are telling Unraid to overwrite files with themselves.  In fact in a worst case it will end up destroying the contents of the 'domains' share by truncating files to 0 length.   You must never mix user shares and physical drives in the same command as it can cause data loss.  If you must move files manually use the Dynamix File Manager as it will protect you against this doing this and getting data loss as a result.

 

You probably need to set the VM and Docker services to be disabled in settings to allow this share to be operated on by mover as is the case with the 'system' and 'appdata' shares.

 

[ZerkerEOD]

1 hour ago, itimpi said:

That will always fail as /mnt/vmstorage/domains is part of /mnt/user/domains so you are telling Unraid to overwrite files with themselves.  In fact in a worst case it will end up destroying the contents of the 'domains' share by truncating files to 0 length.   You must never mix user shares and physical drives in the same command as it can cause data loss.  If you must move files manually use the Dynamix File Manager as it will protect you against this doing this and getting data loss as a result.

 

You probably need to set the VM and Docker services to be disabled in settings to allow this share to be operated on by mover as is the case with the 'system' and 'appdata' shares.

 

They are still disabled. When I change the mover direction I lose the option of move now. How can I disable a plugin without uninstalling it? 

[itimpi]

25 minutes ago, ZerkerEOD said:

How can I disable a plugin without uninstalling it?

What plugin?

[ZerkerEOD]

44 minutes ago, itimpi said:

What plugin?

It was for the mover tuning. But I think I have mostly figured out. Hopefully last question and I really appreciate all the help, what is say I did something stupid like copy from /mnt/user/appdata to /mnt/vmstorage/appdata. How would I fix individual files showing as both on vmstorage and on disk4 now? 

[itimpi]

2 minutes ago, ZerkerEOD said:

It was for the mover tuning. But I think I have mostly figured out. Hopefully last question and I really appreciate all the help, what is say I did something stupid like copy from /mnt/user/appdata to /mnt/vmstorage/appdata. How would I fix individual files showing as both on vmstorage and on disk4 now? 

You need to decide which one is the one you want to keep.    You can then use Dynamix File Manager (plugin) to move files as needed or to delete ones that you no longer need.   You want to end up with any given file on only one location.

[ZerkerEOD]

@itimpi thanks again and really appreciate the help. I think I have most everything working. Looks like I might have lost some appdata file for PostgreSQL, but I am restoring from backup right now.