15 hours ago, Squid said:

Added in two more tunables:

 

Priority of the mover process itself (Normal, Low, Very Low) and

I/O priority of the process (Normal, Low, and Idle)

 

Setting these *may* help with the odd user where the mover process interferes with the normal operation of the GUI / other apps.  Note that if your system pauses all I/O streams if/when a disk is required to spin up that these tunables will not help out all (In which case, do what I did -> adjust Kodi's (eg) caching settings)

Awsome update @Squid. I'm already using the plugin, but don' really have the CPU issue people are mentioning, but seen a lot of people whining about it, so very nice addition to an already awesome plugin Tnx for all the hard work

 

Quote

Added in two more tunables:

 

Priority of the mover process itself (Normal, Low, Very Low) and

I/O priority of the process (Normal, Low, and Idle)

 

Setting these *may* help with the odd user where the mover process interferes with the normal operation of the GUI / other apps.  Note that if your system pauses all I/O streams if/when a disk is required to spin up that these tunables will not help out all (In which case, do what I did -> adjust Kodi's (eg) caching settings)

Thanks @Squid!  Mover used to slow down all running docker apps until it was done.  I can now run it without it negatively impacting my system after lowering the cpu and I/O priority.  This update is great!

Edited June 24, 20197 yr by Iciclebar

Just be aware that setting it to very low and idle may significantly impact the amount of time it takes to move the files depending upon what else is going on in the system when mover runs

Sent from my NSA monitored device

Thank you for adding that in Squid, i just tracked down my problems to Mover yesterday.

Sent over a Beer your way.

2 hours ago, Maticks said:

Sent over a Beer your way.

Thank you.  Always appreciated  

I've just discovered this plugin. It really seems to scratch an itch with the mover, so thanks a bunch. I do have a feature request, though:

 

I have a large cache drive (4tb SSD), and it would be helpful if I could use set it up so, during a scheduled move, turbo write is used only if the cache drive is filled to a certain percent. That way small scheduled moves would happen without turbo write, but big ones that need to write faster to clear the cache would spin up all the drives.

 

Also, and maybe it's asking for too much, but it'd be great if we could:

 

A) stop the mover manually; and

B) automatically stop scheduled moves if they haven't finished by a certain time.

 

My need for those last two may be abrogated by being able to set the mover priority lower / cancel it during parity scans, but I haven' had a chance to test the low priority much.

 

Anyway, thanks again!

Been having a lot of new errors in my system logs for the last 3 days or so. After studying them a little bit, it seems the error only happens when the mover is running. And lines up with the time that I updated this plugin and started using the new priority settings.

 

Here's a piece of my logs: https://pastebin.com/xVpiJDfY

 

I've tried it with and without turbowrite on, and same errors. 

 

Anyone else see any new errors that occur while mover is running?

unraid-diagnostics-20190630-0016.zip

Edited June 30, 20197 yr by drawmonster

1 minute ago, drawmonster said:

Here's a piece of my logs: 

Always best practice to attach the full diagnostics.

Added them

All the errors are from python which is 100% nothing to do with mover or the plugin, unless mover is attempting to move something that your python script doesnt like being moved

Sent via telekinesis

Great plugin! I've got a feature request:

 

- Ability to set a mover stop time (fixed time or relative to starting time).

 

I've got a rather large cache drive (8TB) and I got my mover scheduled for the night ours so it doesn't impact user performance. But when a lot of the cache drive has been filled by daily operations the mover runs into daytime and impacts the day performance. I would love the feature to tell it to stop the mover (I invoke the command manually via the web terminal > 'mover stop') on a certain time or after x hours of running.

5 hours ago, Mevius said:

Great plugin! I've got a feature request:

 

- Ability to set a mover stop time (fixed time or relative to starting time).

 

I've got a rather large cache drive (8TB) and I got my mover scheduled for the night ours so it doesn't impact user performance. But when a lot of the cache drive has been filled by daily operations the mover runs into daytime and impacts the day performance. I would love the feature to tell it to stop the mover (I invoke the command manually via the web terminal > 'mover stop') on a certain time or after x hours of running.

Just use user scripts and have a script that runs at whatever time with the mover stop command.  Doesn't do anything if its not running.

 

Thanks for the plugin, squid.  I do have a "feature" request, if you could call it that

Currently, I modify mover (now called mover.old) to ignore certain directories in certain user shares and print log messages about it.

 

I had noticed after I installed your plugin that mover.old's logs no longer print.  Digging into it, it's due to it using "echo" and it's caller used to previously pipe mover's output to logger (ie. /usr/local/sbin/mover(.old) |& logger).  I can easily add that line into your mover.php (and I do), though I assume an update would get rid of it.

 

Would this be possible, so mover.old's existing logs would also print?

I've added this to my own mover script, but a good feature for some would be to have certain docker containers shut down before mover runs and restarted after. For example, Radarr and Sonarr both like to get currupt databases if you run mover and they weren't set to use the cache drive as their appdata directory specifically. I don't have any appdata on my cache drive, as it's only used for speeding up my downloads and file transfers. 

Thanks for the plugin, squid.  I do have a "feature" request, if you could call it that

Currently, I modify mover (now called mover.old) to ignore certain directories in certain user shares and print log messages about it.
 
I had noticed after I installed your plugin that mover.old's logs no longer print.  Digging into it, it's due to it using "echo" and it's caller used to previously pipe mover's output to logger (ie. /usr/local/sbin/mover(.old) |& logger).  I can easily add that line into your mover.php (and I do), though I assume an update would get rid of it.
 
Would this be possible, so mover.old's existing logs would also print?

Honestly I never enable mocer logging, so never checked it at all. Will pump out an update in the next day or so

Ashamed to admit that I no longer live up to my teenage ethos of "it's better to burn out than fade away"

On 8/20/2019 at 11:40 AM, gooner said:

Would this be possible, so mover.old's existing logs would also print?

Should be fixed on today's update

Recently I've spotted mover running a few times when I'm nowhere near my threshold - diagnostics attached.highlander-diagnostics-20190829-0738.zip

is it possible to move files immediately after they have finished writing? (like a physical HDD cache?)

 

So it can write to cache (fast) then trickle feed it to array. keeping the cache empty and only using it as, well, a a cache for new files?

Closest you can get is hourly itll run (or even say every 10 minutes if you set a force move custom cron)

Sent from my NSA monitored device

14 hours ago, Squid said:

Closest you can get is hourly itll run (or even say every 10 minutes if you set a force move custom cron)

Sent from my NSA monitored device
 

Ah thanks, I'll keep it set to overnight, as the mover always messes up my Kodi playback. (I can't change the Kodi cache, but that's a different Problem)

 

Is it possible to limit the speed,  only write to the array at 4mbs for example.

Edited September 5, 20196 yr by TheBuz

are the priority settings still working under the latest unraid?

 

I'm sure they used to work, but now when the mover runs I get buffering on my server if it's moving (i had them set to low and it seemed to work before, never had issues with emby server under docker)?

 

Is there any way of confirming it is Working?

 

it will be great if this plugin support running custom scripts before and after the mover, so we can have script to stop vm/docker before mover running and restart them after mover is finished

I have observed that with "Force Turbo" enabled, at the end of the move, it returns to regular write mode without all data being written, so the last couple megabytes always write in read + overwrite mode.

 

I wonder if that could be improved by calling 'sync' before reverting turbo mode.

Hi, I want to add my own script to run before and after mover runs (to pause some processes I know don't like mover), how would I go about doing this? I tried adding a command to run my script before the following line within /usr/local/sbin/mover.

 

/usr/local/emhttp/plugins/ca.mover.tuning/mover.php $P_COMMAND $1

 

but this file doesn't appear to be called anymore.

Edited December 28, 20196 yr by naturalcarr

15 minutes ago, naturalcarr said:

Hi, I want to add my own script to run before and after mover runs (to pause some processes I know don't like mover), how would I go about doing this? I tried adding a command to run my script before the following line within /usr/local/sbin/mover.

 

/usr/local/emhttp/plugins/ca.mover.tuning/mover.php $P_COMMAND $1

 

but this file doesn't appear to be called anymore.

By "anymore", do you mean it did work but now it doesn't?

 

The Unraid OS is in RAM and a fresh copy is loaded from flash at each boot. So any change you might have made in /usr doesn't survive reboot.