Is this behaviour normal during a move?

[eribob]

Hi! 

My cache drive was getting full so I wanted to empty it today. I realised that I had set the appdata share to "prefer" cache so I changed its cache setting to "yes" and made sure all other shares have "yes" as well. In addition I stopped all docker containers and all VMs. Then I started the move. 

 

The problem is that it has been doing the move for a long time now (2 hours) and it is reading from one drive in the array that is almost empty and writing to another drive and to the parity disk. It is however not reading from the cache drive. Some data seems to have been moved of the cache, but very slowly. Is this how it is supposed to work? 

 

It does not seem logical to only move things around in the array when I actually want to move from cache to array?

 

See attached screenshots. 

 

I also activated "turbo write mode" in an attempt to speed the process up. 

[FreeMan]

Totally OT to your question, but I see that your boot drive is "DataTraveler_3.0" I presume that's a USB3 device. It's strongly recommended that you not plug it into a USB3 port on the server, but have it on a USB2 port instead. There are known issues with the boot drive living in a USB3 port.

 

I doubt this has anything at all to do with your issue, and you may currently have it plugged into a USB2 port, but I just thought I'd mention it in passing.

[Squid]

Due to how it works, mover is not particularly efficient in moving files (checking if they're in use etc)

 

This will be magnified especially if you have say Plex installed where any reasonably sized library will result in many thousands of files within the appdata being moved.  Not sure though why you want appdata to be moved from the cache drive to the array.  "Prefer" is the general setting for the appdata share

[eribob]

Thanks for quick replies to a noobs questions!

 

10 minutes ago, Squid said:

Not sure though why you want appdata to be moved from the cache drive to the array.

I wanted to move the appdata off the cache because it was getting big and consuming a lot of the cache drive space. Is it better to put it on an unassigned drive? 

 

11 minutes ago, Squid said:

This will be magnified especially if you have say Plex installed where any reasonably sized library will result in many thousands of files within the appdata being moved. 

Still, I wonder why there is no activity on the cache drive at all during the move, only on two of the drives in the array? 

 

17 minutes ago, FreeMan said:

Totally OT to your question, but I see that your boot drive is "DataTraveler_3.0" I presume that's a USB3 device. It's strongly recommended that you not plug it into a USB3 port on the server, but have it on a USB2 port instead. There are known issues with the boot drive living in a USB3 port.

Thanks for the heads up. The boot drive is put in a USB2 port. 

[Squid]

From the terminal,

mover stop

Then under settings, scheduler, mover settings, enable mover logging

 

Then restart mover.  After a suitable delay for when you see what you were seeing before, post your diagnostics

 

8 minutes ago, eribob said:

I wanted to move the appdata off the cache because it was getting big and consuming a lot of the cache drive space. Is it better to put it on an unassigned drive? 

Downside to appdata on the array is that it's slow access compared to the cache drive and multiple drives will have to stay up and spinning pretty much all the time.  I'd either use an unassigned drive or get a bigger cache drive.  Personally I'd choose the larger cache or have the shares not set to use the cache drive.  I only have my appdata and my downloads share using the cache.  Everything else writes directly to the array (use cache: no)

[eribob]

Thank you for the advice. I changed the behavior now so that appdata is on the cache ("prefer"). I also noticed that the "system" share was set to "prefer" for the cache drive. Is this advisable as well? It contains the docker apps right?

[eribob]

Now I think I messed up, just asking if there is a magic fix or if I need to reinstall my VM:s... 

 

When trying to move I got the following in the log: 

 

Dec 29 00:54:08 Monsterservern root: mover: started
Dec 29 00:54:08 Monsterservern move: move: file /mnt/cache/domains/Ubuntu - jupyterhub/vdisk1.img
Dec 29 00:54:08 Monsterservern move: move_object: /mnt/cache/domains/Ubuntu - jupyterhub/vdisk1.img File exists
Dec 29 00:54:08 Monsterservern move: move: file /mnt/cache/domains/Windows 10/vdisk2.img
Dec 29 00:54:08 Monsterservern move: move_object: /mnt/cache/domains/Windows 10/vdisk2.img File exists
Dec 29 00:54:08 Monsterservern root: mover: finished

It looked like there were two copies of the vdisk.img files, one in the array /mnt/user/domains/... and one on the cache drive. I removed the ones on the cache drive and now my VM:s will not start. I guess the .img files were partly in both places and now they are corrupted? Is it fixable? 

[Squid]

The one on the cache drive probably was the good one.

 

When duplicated files exist in the system, and you access them via /mnt/user/domains (eg), it goes in alphabetical order in finding the one to use (ie: cache, disk1, disk2 etc)

 

What is the use cache setting for the domains share?  Normally, it would be use cache:prefer

[eribob]

Thanks. It used to be prefer, but i wanted to change it to yes to move the vdisk img from cache to array. 
 

also, one of the vdisks was 500GB and my cache drive is only 256GB. Thats why I thought it was actually on the array, not the cache disk. 
 

when browsing cache drive files through the unraid web interface it looked like a 500GB vdisk was on the 256GB cache... 

[itimpi]

15 minutes ago, eribob said:

when browsing cache drive files through the unraid web interface it looked like a 500GB vdisk was on the 256GB cache... 

That is possible as vdisk files are by default created as 'sparse' files that means they only occupy enough physical space to hold the parts of the file that have been written to.   However over time as the VM runs the physical space tends to grow towards the assigned size.  If the physical media then runs out of space the VM will start mis-behaving.   One needs to be careful, therefore, about overcommitting the vdisk space.