[SOLVED] Mover isn't moving files from Cache to Array


27 posts in this topic Last Reply

Recommended Posts

Just what it says.

 

Cache is pretty full. It seems to run fine after a few hours (can't say it runs on 1 hour schedule like I set it up because it seems to take longer than 1 hour to actually do anything). Maybe I have set it up wrong?

 

Trying to run it manually now. The button presses, says mover is running. After refreshing the page. It shows it's not running.

Screenshot_2020-11-17 Tower Main.png

Link to post

I'm trying to move the files from the cache drive to /tank which is comprised of those other 3 drives you see above.

It's a share called /tank which the mover should be moving filed downloaded to the cache off to the main drives every hour. For some reason though it only does this when it feels like it (at least that's how it seems) despite my set hourly schedule.

 

I've attached some files. As you can see from the last screenshot, the cache drive has filled up. All of my docker images have now crashed and when trying to invoke the mover manually, it says it's running but a refresh shows it's stopped immediately and is doing nothing.

cache disk info.png

Screenshot_2020-11-17 Tower Shares.png

Screenshot_2020-11-17 Tower Main(1).png

tower-diagnostics-20201117-1755.zip

Link to post

appdata share is being moved to cache, since there's no space the mover aborts and it never gets to the tank share, they are moved alphabetically, change appdata share to cache="only" or "no" so nothing is moved for now, then run the mover to move the tank share and change it back.

Link to post

Thanks for the continued assist. Can I leave the appdata share on cache=only, or do I need to move it back later? I thought I had set it to cache only just to make containers perform better anyway.

 

Setting it to no or only doesn't seem to make the mover do anything still. It just stops immediately. /tank is now the only share that is using the cache btw, which is set to "yes". It would be nice if in an update the mover skipped moving items to the cache if it was full and instead focused on moving items off, instead of outright stopping like it is in this case.

 

Screenshot_2020-11-17 Tower Share.png

Edited by plantsandbinary
Link to post

Since your appdata and system (and domains) shares have files on the array, and mover can't move open files, you will have to go to Settings and disable Docker and VM Manager service to get those moved. If you also want to move domains to cache you will have to set it to cache-prefer before moving.

3 hours ago, plantsandbinary said:

(can't say it runs on 1 hour schedule like I set it up because it seems to take longer than 1 hour to actually do anything)

I really question the idea of running mover so often. Mover is intended for idle time. If your system isn't really idle then you might as well be writing that share directly to the array.

Link to post

It seems to be working now, but I had to restart the server and then press "move" for it to actually do anything. No idea why. I had even turned off Docker before restarting so there really shouldn't have been any activity whatsoever. Either way, it seems to be okay now but I do wish it would just move things off the cache first regardless of whether or not there are tasks to put things on the cache too.

Link to post

Looks like we posted at the same time. See my comments above your latest post.

 

You can see how much of each disk each user share is using by clicking on Compute... for the share on the User Shares page.

 

Or post new diagnostics and I will take a look to see if your docker/VM related shares (appdata,domains,system) still have files on the array.

Link to post
2 hours ago, plantsandbinary said:

It seems to be working now, but I had to restart the server and then press "move" for it to actually do anything. No idea why. I had even turned off Docker before restarting so there really shouldn't have been any activity whatsoever. Either way, it seems to be okay now but I do wish it would just move things off the cache first regardless of whether or not there are tasks to put things on the cache too.

Have you set a Minimum Free Space value for the cache under Settings->Global Share settings?     That would stop mover from filling the cache to capacity and should also allow it to get to the files that need to be moved off the cache.

Link to post
3 hours ago, JorgeB said:

Mover won't touch a share set to "no" or "only", all the appdata should be on the same place, so you should finish moving it to cache, after it's done you can leave it as cache "only" or "prefer".

I've set appdata to "only". I want the appdata on the cache drive always, I don't want it on my RAID array. So I guess that's the correct setting?

 

2 hours ago, trurl said:

Since your appdata and system (and domains) shares have files on the array, and mover can't move open files, you will have to go to Settings and disable Docker and VM Manager service to get those moved. If you also want to move domains to cache you will have to set it to cache-prefer before moving.

I really question the idea of running mover so often. Mover is intended for idle time. If your system isn't really idle then you might as well be writing that share directly to the array.

Thanks I did this. Though I didn't have domains on the cache drive. I just checked. I had only some files from the array (those new ones downloaded via rutorrent, waiting to be moved to the array) and appdata. I turned off Docker via the settings tab and ran the mover with app data set to "only" and /tank set to "yes". I don't really understand the difference between "yes" and "prefer" though.

 

I have the mover run every hour (or at least I try to get it to) because I have a gigabit line with 100MB/s download and 50MB/s upload. I cycle through a lot of bluray rips up to 80GB in size. So if I am downloading multiple files at once I need the mover to run asap to move files from the cache to the array so my docker containers don't crash until the mover runs and clears space on my cache drive. Setting it to every day wasn't enough because it will fill up instantly and then not move for the rest of the day. As far as I understood the scheduler, the hourly run means only that it will "check" if files need to be moved, but still decide not to if the cache isn't very full. Whereas if I set it to run every 12 hours, it'll check and decide but if it's not full at that time, it won't run. Meaning any time between the next 12/24 hours nothing will happen if the drive fills up quickly.

 

As I understood, there isn't any kind of downside running the mover that often.

17 minutes ago, itimpi said:

Have you set a Minimum Free Space value for the cache under Settings->Global Share settings?     That would stop mover from filling the cache to capacity and should also allow it to get to the files that need to be moved off the cache.

It looks like it's set to: 2000000. Would be nice to know if that was bytes/kilobytes etc. I'm guessing that's kilobytes. Seeing as my cache drive won't fill past 1.6GB and 2000000 kilobytes is about 2GB in base 10, I guess that's accurate. I'll add an extra 0 just in case and hopefully it will stop my docker containers crashing when the cache gets really full.

Link to post

Well I'm not sure what just happened but I did what I wrote above and the mover finished, with only 30GB left on the cache. I turned Docker back on, started ruTorrent and every single one of my torrents immediately started redownloading and overwrote my completely written files... I had already downloaded all of those files over a 9-12 month period. But it's now downloading over 3.9TB of data all over again... this is a total fucking catastrophe... and now I have to go through each one and figure out what I want to re-download...

 

I cannot even stress what a catastrophe this is. I lost over 1TB of music... half of those torrents don't even exist any more and I was the only seeder. Now that I don't have the full files they aren't downloading. I have no idea where I will get some of those albums again. They took months to find.

 

Maybe the biggest irony. My fucking cache drive is 95% full again in an instant thanks to ruTorrent allocating space for all the files. Why the hell couldn't the mover just move the damn files off the cache before trying to move appdata back on. Honestly, how tf is that not a simple thing to do. I would never have had to change anything if it had just skipped appdata for a moment and moved the stupid movies onto the fucking array first!

 

I am so pissed. I have to turn this PoS off and think about when I feel like I have the patience to sort this mess out. But I bet I'm going to wind up with a dozen warnings on my private trackers complaining that I've done partial downloads and am not seeding back.

Edited by plantsandbinary
Link to post

If you had your docker mapping directly to cache instead of to a user share, then that would explain why torrents couldn't be found after they were moved to the array.

 

Also, not clear if you actually got all of appdata moved to cache, since you didn't say you had checked. You just said you had set it to cache-only. As mentioned already (and you actually quoted this part)

3 hours ago, JorgeB said:

Mover won't touch a share set to "no" or "only"

In order to get something moved to cache it must be set to cache-prefer (and in order for something to get moved to the array it must be cache-yes).

Link to post

Well I had a stab at this again and fixing my configuration. I literally am starting to question entirely how my device is meant to be setup to avoid this problem happening in the future.

 

I rebuilt my torrents and setup rutorrent again after importing everything. It seems to be okay now but all of this has me wondering if it's set up the right way.

 

The mover is still not moving files to the array when I am trying to run it. Here's the setup

 

Here's the setup pics and diagnostics (attachments)

 

SPECS:

 

  • Model: HP MicroServer Gen8
  • M/B: Version - s/n: ??????????
  • BIOS: HP Version J06. Dated: 04/04/2019 (latest)
  • CPU: Intel® Xeon® CPU E31265L @ 2.40GHz
  • HVM: Enabled
  • IOMMU: Enabled
  • Cache: 128 KiB, 1024 KiB, 8192 KiB
  • Memory: 16 GiB DDR3 Single-bit ECC (max. installable capacity 16 GiB)
  • Network: bond0: fault-tolerance (active-backup), mtu 1500
  •  eth0: 1000 Mbps, full duplex, mtu 1500
  •  eth1: 1000 Mbps, full duplex, mtu 1500
  • Kernel: Linux 4.19.107-Unraid x86_64
  • OpenSSL: 1.1.1d
  • Uptime: 2 days, 18:07:08

 

In short-hand. As I understand for best performance I have it set up like this:

 

  • appdata for docker containers is on the cache drive always. So appdata is set to cache -> only
  • tank (which is where all my files go spread across the array) is set to cache -> yes so new files etc. are put there. This is really just for quick downloading via rutorrent
  • every other share is set to cache -> no

 

If that's wrong, or I am not understanding this right or I've done something wrong, or there is a better way to do this. Someone please tell me so I don't blow up my device in the future. A short explanation of how/what to do would be greatly appreciated.

 

With this setup. The mover refuses to do anything when trying to invoke it manually. It will just move when it feels like it from what I can see. There's no other way for me to describe when/how it actually makes the decision to move files from the cache to the array.

 

Thanks for the help and sorry for losing my head before but it was one hell of a crippling loss...

Screenshot_2020-11-20 Tower Share(1).png

Screenshot_2020-11-20 Tower Share.png

Screenshot_2020-11-20 Tower Browse(3).png

Screenshot_2020-11-20 Tower Browse(2).png

Screenshot_2020-11-20 Tower Browse(1).png

Screenshot_2020-11-20 Tower Browse.png

Screenshot_2020-11-20 Tower Main.png

tower-diagnostics-20201120-1754.zip

Link to post

Mover still hasn't moved anything manually or automatically. It's strangling my server because my cache is totally full. Can someone please tell me why it's not working?

 

EDIT: Seems to be something to do with Docker I think. Restarting and immediately running the mover looks to be okay.

Edited by plantsandbinary
Link to post

I hope this has the right information. It should be at the bottom of the log.

 

I rebooted, ran the mover. It ran for a while, moved about 125GB and then stopped halfway. I'm assuming because I didn't turn off Docker and something on one of my docker containers spun up and made it stop. I tried running it again (this time with logging enabled) and it stopped immediately again.

 

I hope this diag tells why.

tower-diagnostics-20201121-1255.zip

Link to post
4 minutes ago, JorgeB said:

There's a couple of files that already exist, also one skipped, maybe it's in use, like seeding?

 

That's probably it. So the mover won't move the files if they are seeding? In that case I'll shut down rutorrent before engaging the mover and see if that works.

 

Does everything else in the screenshots above look okay though? appdata cache set to - only and /tank set to cache - yes?

 

What about the rest? I can handle shutting down ruTorrent if the mover fills up and I need to run it manually after having downloaded several large files, but if something else breaks like last time I'm going to spit the dummy.

Edited by plantsandbinary
Link to post
  • JorgeB changed the title to [SOLVED] Mover isn't moving files from Cache to Array

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.