relink Posted March 21, 2021 Share Posted March 21, 2021 As the title says I am trying to upgrade my cache disk so I am moving everything to the array. I stopped docker and VM services first, and set all shares to use cache yes. The mover is running but it’s been over 12 hours and it’s not even half done. I would have expected a transfer time closer to 6 hours. so far I have used “renice” and “ionice” to set the priorities of the processes involved in moving higher, but it doesn’t seem to have made much difference. I was hoping to do that cache upgrade this morning, but at this rate I’ll be lucky if I can do it by tonight. Quote Link to comment
JorgeB Posted March 21, 2021 Share Posted March 21, 2021 It's normal if there are a lot of small files, like Plex metadata for example. Quote Link to comment
relink Posted March 21, 2021 Author Share Posted March 21, 2021 16 minutes ago, JorgeB said: It's normal if there are a lot of small files, like Plex metadata for example. Crap...that’s exactly what it is. Tons of tiny little files. Quote Link to comment
JorgeB Posted March 21, 2021 Share Posted March 21, 2021 Mover is very inefficient when dealing with those, next time I suggest using the console or a tool like midnight commander. Quote Link to comment
relink Posted March 21, 2021 Author Share Posted March 21, 2021 What command would I need to run to move everything from cache to array? unless there’s some reason I shouldn’t, I will gladly stop the mover and do that instead. Quote Link to comment
relink Posted March 21, 2021 Author Share Posted March 21, 2021 I just realized that midnight commander is built in to unraid. Woul I just stop the mover and then use that to move the folders from “/mnt/cache” to “/mnt/user”? Quote Link to comment
Dovy6 Posted March 21, 2021 Share Posted March 21, 2021 I've done that. User "mover stop" in the terminal. Unfortunately millions of tiny files will take forever to move no matter how you do it. Quote Link to comment
StevenD Posted March 21, 2021 Share Posted March 21, 2021 Whenever I upgrade my cache, I ALWAYS use the CA Backup/Restore plugin. Its is way, way faster for backing up and restoring all the appdata. My appdata folder is about 300GB, most of which is Plex metadata. It does it in about 25 minutes. Quote Link to comment
itimpi Posted March 21, 2021 Share Posted March 21, 2021 1 hour ago, relink said: I just realized that midnight commander is built in to unraid. Woul I just stop the mover and then use that to move the folders from “/mnt/cache” to “/mnt/user”? NO! /mnt/user is for User Shares that include the cache drive and if you try this will end up losing your files as they will be truncated to 0 bites as they try and copy on top of themselves. You need to either use a named drive such as /mnt/diskX for the target or if you want it to be a User Share use /mnt/user0 as the target as that is the user share files that are only on the main array omitting the cache drive Quote Link to comment
relink Posted March 22, 2021 Author Share Posted March 22, 2021 (edited) 9 hours ago, itimpi said: NO! Good thing I ended up not doing that at the last second. instead I decided to add my new nvme drive as a second cache, and use MC to move everything from the old one to the new one and reassign the cache for each share. however, this is still ungodly slow. Midnight commander “claims” it’s transferring at 15MB/s ...it’s not, there’s no way. It’s maybe 15Mb/s if that. I calculated out the transfer time beforehand and this should have been done hours ago, yet it’s still going. Edited March 22, 2021 by relink Quote Link to comment
relink Posted March 22, 2021 Author Share Posted March 22, 2021 Well it was a total success! After quadruple the amount of time I thought it was going to take my new cache drive is up and running. I think next time I’ll take the advice above and just restore from my backup. Quote Link to comment
SimonMisc Posted April 21, 2022 Share Posted April 21, 2022 On 3/21/2021 at 10:02 AM, itimpi said: NO! /mnt/user is for User Shares that include the cache drive and if you try this will end up losing your files as they will be truncated to 0 bites as they try and copy on top of themselves. You need to either use a named drive such as /mnt/diskX for the target or if you want it to be a User Share use /mnt/user0 as the target as that is the user share files that are only on the main array omitting the cache drive So I just found this thread while remembering vaguely that there's a no-no somewhere when it comes to transferring files manually, in this case the 0-byte truncation issue you mentioned. What's still not clear to me is which directories are safe to move files between. As an example, I'm hoping to move everything from my Cache pool to a new Temp Pool drive. Can I safely move for example /mnt/cache/appdata to (newly created) /mnt/temp/appdata? Quote Link to comment
JorgeB Posted April 21, 2022 Share Posted April 21, 2022 10 minutes ago, SimonMisc said: What's still not clear to me is which directories are safe to move files between. You can move from a disk share to another disk share and from a user share to another user share, just not from a disk share to a user share or vice versa. /mnt/cache and /mnt/temp are both disk shares so that's fine, anything with /mnt/user/share_name is a user share. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.