I'm having an issue where mover is extremely slow with moving a lot of small files. And when trying to Stop Mover the process just continues. Even though the process is killed and the GUI says that mover isn't running.

 

Current build:

Unraid 7.0.0

1x 1TB NVME = cache drive & appdata folder (appdata = 90 GB)

2x 2TB SSD BTRFS pool = nextcloud & plex folder (1.18 TB)

 

I want to re-organize and upgrade. For this reason I started with emptying the BTRFS pool to the array. I changed the mover location in Shares for just the Nextcloud share and started the mover. Overnight the used space on the pool went from 1.21 TB to now 1.18 TB... That's almost 12 hours for not a lot of data. I understand that all these shares contain almost nothing but small files and reading online I can see that mover has trouble with these kind of situations. So, in order to try and find a different and faster solution, I stopped the mover.

Kill PID > this killed the process, but files are still being moved.

Stop mover > this updated the GUI, but files are still being moved.

 

So mover isn't moving according to Unraid, but files are still being moved just looking at the read/write action on the Main tab and it's been over an hour now since issuing these commands. I would like to move forward, but I don't know how.

1) What's the best course of action to really stop the moving of files? Stopping the array? Rebooting the server? I haven't done that because I'm afraid of damaging files that are in transit.

2) While I'm here asking for help; What is the best practice of emptying both drive (pools) and start over? It's a lot of data with mostly just small files.  For context; the above setup is going to be transformed to:
1x 1TB NVME = cache only
2x 500GB SSD ZFS pool = appdata (including Plex)

3x 2TB SSD ZFS pool = personal files (including Nextcloud share)

Mover is always super slow with very small files because it checks if every file is in use before attempting to move it, if it's a one time reorganization, I would recommend using another tool like midnight commander (mc on the console)

Yes, well, there is no warning before running mover so I didn't know that before. If I had known it would take mover 1 month to complete... I just went with what I had learned previously with changing cache drives. Move appdata to array. Change drive and put everything back.

 

Before trying different things I would like to know how to stop the moving process.

 

mover stop

mover: not running

 

While I see in the Main tab read/write on both the array and btrfs pool at the same time and at the same speed, telling me that mover is still doing it's thing. Only thing I can think off is that it's finishing it's 'cache' of files it had already lined up for processing? Anyhow, it's been 3 full hours and it's still going and I have no idea how to monitor what's going on or how to safely stop it, aside from stopping the array or rebooting the server.

EDIT (before posting) and for anyone getting into this same situation;

I had a chat with chatgpt and checked everything afterwards with Gemini to make sure Told it what I did and what was happening right now and if we could try the least invasive options first to fix this. It wanted me to look which processes were busy with files on my btrfs pool.
 

lsof | grep "/mnt/cache_raid"

This returned a lot of the same UID 122 messages, but also a couple of 'find' messages

 

lsof: no pwd entry for UID 122 
find 3707572 root 6r DIR 0,44 206 856278 /mnt/cache_raid/nextcloud/appdata_ocwgts04pj90/preview/8/8/a/c/7/2/8/371816

 

chatgpt suggested next to kill the process with "kill -9 3707572", but after googling "-9" (terminate process immediately) and being afraid of data getting corrupted, I opted for for "kill 3707572". After running that it took around 15 seconds for the Main tab showing me that read/write stopped completely. Looking at syslog straight after that it showed this:

 

Jan 21 15:17:48 Server root: ionice -c 2 -n 0 nice -n 0 /usr/local/sbin/mover.old stop

Could this perhaps have something to do with the mover tuning plugin and Unraid 7? I'm seeing mover and mover.old script.

 

After the upgrade to Unraid 7 mover wouldn't move. I googled and removed the plugin after reading that I had still an old version installed. I thought Unraid would warn users for incompatible plugins before upgrading? Anyway, after removing that plugin and rebooting the server Mover was working again. But I wonder why mover.old is now the cause of this all... Or maybe was I wrong for killing the process first and should I have just ran "mover stop"...

The mover part has been fixed, so now I need an option to move all the small files to another drive. @JorgeB Do you mean with Midnight Commander I should use Krusader? Just move files from one window to another, into the right disk/folder? Chatgpt suggested rsync
 

rsync -av --ignore-existing /mnt/cache_raid/<sharename> /mnt/user/<sharename>

I was using a specific disk in my array, so maybe change out /mnt/user for /mnt/disk ? And perhaps I should just use an unassigned drive for this, so the array won't slow things down doing parity calculations? Given the fact there are so many small files. I will be deleting that preview folder for sure and just redo that when it's all back up.

Sorry for the long post.

 

1 hour ago, FreakyUnraid said:

Do you mean with Midnight Commander I should use Krusader?

Those are two different options, both will work, Midnight Commander is included, and doesn't need any setup, though.

 

Rsync is another good option, but never copy from a disk share to a user share, either disk share to disk share, or user share to user share, in this case use the former.