[Plugin] Mover Tuning


Recommended Posts

40 minutes ago, MvL said:

Ah, so I have to set "Force move of all files on a schedule" to yes and set a time?

That will run mover at whatever time you set regardless of whether you have exceeded the threshold or not.

 

An example, I have mine configured to check hourly. If my threshold is exceeded (60%) then mover will run. I also set a forced move to run every 24 hours just so I don’t have files sitting in the cache for long periods if I’m not transferring many files to my shares.

Link to comment
6 minutes ago, MvL said:

I'm confused.., so if the threshold is reached then the mover will do his work without setting a schedule? If I want every 60 minutes a move job then I have to set the schedular for 60 minutes?

 

Here’s mine-

 

SafariScreenSnapz093.jpg.8ebd76ee2b843649f3fe47196fa9b751.jpg

 

Mover Schedule is set to “hourly” so every hour the plugin will check to see jf the threshold (60%) has been reached. If it has then mover will run. If not, then it will check again in another hour.

 

Force move of all files is set to run every morning at 3:30 am, so at 3:30 mover will run regardless of whether the threshold has been exceeded or not.

 

Turning on “help” will give you a brief explanation of each setting.

Link to comment

Ah yes, now it's clear. I did see the plugin mover and unRAID mover as 2 separate things but the work together. It's just a enhancement. Like the plugin name says. 

 

Maybe it's a idea to make this more clear that these functions work together thus unRAID OS schedule checker and setting the threshold.

 

Going to play a bit with setting and check if it's working now.

 

Thanks wgstarks!

Link to comment

In that case you can set the mover schedule to hourly and the threshold to 75%. The plugin will check every hour to see if it’s 75% full yet and when it is it’ll run mover. Just remember that depending on the size of your cache drive and the files you’re transferring it might take a while to reach 75%.

Link to comment
  • 1 month later...

Is there a way to check manually via a script more frequently if the threshold has been passed e.g. every 30 or 15 minutes, and if so run mover? Every hour is too late for me as I have a fast connection .  If I install the old script will they conflict?

 

Thanks

Link to comment
4 hours ago, DZMM said:

Is there a way to check manually via a script more frequently if the threshold has been passed e.g. every 30 or 15 minutes, and if so run mover? Every hour is too late for me as I have a fast connection .  If I install the old script will they conflict?

 

Thanks

I strongly recommend rethinking your arrangement if you are running mover that often. It would probably be much more efficient and faster overall to download or unpack directly to the array instead of the cache.

 

Normally the cache drive is used for 2 primary functions. Keep the array spun down as much as possible, and speed up user initiated new data writes. If you need to run the mover that often, your destination drive will stay spun up anyway, and the new data is coming from a source that you aren't directly waiting on, like if you were copying a file from your desktop to the server.

Link to comment
6 minutes ago, jonathanm said:

I strongly recommend rethinking your arrangement if you are running mover that often. It would probably be much more efficient and faster overall to download or unpack directly to the array instead of the cache.

 

Normally the cache drive is used for 2 primary functions. Keep the array spun down as much as possible, and speed up user initiated new data writes. If you need to run the mover that often, your destination drive will stay spun up anyway, and the new data is coming from a source that you aren't directly waiting on, like if you were copying a file from your desktop to the server.

I understand what you are saying.  That's exactly how I use my cache 90% of the time.

 

The problem I have is if I suddenly have a lot of writes to my cache, because my download speed is fast it can fill up quickly.  Normal usage is the mover won't run more than once a day, but if it does fill fast I have a problem at the moment.  I just want a bit of insurance by being able to poll more often.

 

It's also hard for me to seperate 'background writes' from 'foreground writes' i.e. yes I'm ok with movies getting downloaded and written to /movies_adults slowly, but when I'm manually ripping or moving a movie to the same directory I want it to happen quickly.

 

 

Link to comment
2 hours ago, DZMM said:

It's also hard for me to seperate 'background writes' from 'foreground writes' i.e. yes I'm ok with movies getting downloaded and written to /movies_adults slowly, but when I'm manually ripping or moving a movie to the same directory I want it to happen quickly.

Not hard at all. Set the download destination in your docker to /mnt/user0/movies_adults.

That way downloads go directly to the array drives, but writes to the user share movies_adults go to the cache drive.

Link to comment
9 minutes ago, jonathanm said:

Not hard at all. Set the download destination in your docker to /mnt/user0/movies_adults.

That way downloads go directly to the array drives, but writes to the user share movies_adults go to the cache drive.

That would still mess up my flow or lead to inefficiencies as I can't fully split my flows e.g. I download tv show via   /mnt/user0/downloads --->  /mnt/user0/tv_shows, but then I want to manually import a file in sonarr from  /mnt/user0/downloads - I want it to happen fast so I want to use the cache drive not /mnt/user0/tv_shows.

 

Also, by pooling my writes (normally max two big writes per day max) means my array drives spin up fewer times so I get less noise, and the turbo write plugin kicks in nicely to make the array write go quickly in one big chunk, rather than drives spinning up and down all day.

 

I'm happy with my flow, I've just been caught out a few times when doing a big download session

Link to comment

Two choices.

You can manually modify shares.cfg and change the cron entry there or you can create a user script to run mover (would probably need to specify the complete path in the script)

AFK though and I can't remember the logic I'm using to determine if it's the forced mover or the scheduled mover running. Will have to check tomorrow

Sent from my SM-T560NU using Tapatalk

Link to comment
1 hour ago, Squid said:

Two choices.

You can manually modify shares.cfg and change the cron entry there or you can create a user script to run mover (would probably need to specify the complete path in the script)

AFK though and I can't remember the logic I'm using to determine if it's the forced mover or the scheduled mover running. Will have to check tomorrow

Sent from my SM-T560NU using Tapatalk
 

thanks

Link to comment
  • 2 months later...

Hi,

 

I do not know if it's due to this plugin, but after pressing "Move now", more than 100Go moved from array to cache. This concerns two shares that have the option "Use cache disk" to "prefer".
These two shares are no longer on any of the three disks in the array, but only on the cache.
I'm surprised because mover should move data from cache to array but not vice versa. I am wrong  ?

 

How to force mover to move the data from the array to the cache ?

 

mover every hour:

Aug 25 23:00:01 Tower root: Cache used space threshhold (60) not exceeded.  Used Space: 22.  Not moving files

 

manual mover:

Aug 25 23:12:27 Tower emhttpd: req (18): shareMoverSchedule=0+*%2F1+*+*+*&shareMoverLogging=yes&cmdStartMover=Move+now&csrf_token=****************
Aug 25 23:12:27 Tower emhttpd: shcmd (21540): /usr/local/sbin/mover |& logger &
Aug 25 23:12:27 Tower root: Manually starting mover
Aug 25 23:12:27 Tower move: move: file /mnt/disk1/Logiciels/Mac/Apple/xxx.dmg
Aug 25 23:12:48 Tower move: move: file /mnt/disk1/Logiciels/Mac/Apple/xxxxx].d

 

Link to comment

Mover can move files in either direction!  What you see is expected behaviour if you have set a user share to Use Cache=Prefer.   You want Use Cache=Yes for mover to move files from cache to array.    Turn on the Help in the GUI to get more details on how the setting affects mover.

Edited by itimpi
Link to comment
7 hours ago, itimpi said:

Mover can move files in either direction!  What you see is expected behaviour if you have set a user share to Use Cache=Prefer.   You want Use Cache=Yes for mover to move files from cache to array.    Turn on the Help in the GUI to get more details on how the setting affects mover.

 

I just tried it and it works: the data comes back from cache to array.

I have only one SSD for the cache. I have to buy another one to secure my data.

 

Thank you very much.

Link to comment
  • 1 month later...
9 hours ago, userdefault said:

I am having trouble getting this to work. It will let my cache fill up and never run mover, what are some troubleshooting steps I should do?

Enable log when not moving due to rules, a screenshot of the settings, and the diagnostics

Link to comment

Hi, may a suggest for another option ;)

 

free to a specified amount and move by age.

 

sample

 

mover runs when cache >85 %, then free up to <50 % by moving the "oldest" files.

 

Like this more "actual" files keep in cache, example like media elements, actual records and stuff, i assume drives would have less spinups ...

 

just as suggestion ;)

 

actually this script is an awesome improvement already, thanks alot herefore ;)

  • Like 1
Link to comment

what do i have to do if i want the mover to run on a schedule and when the cache size is filled to a certain percentage (not bound to a schedule, but triggered at x%)? I understand that with this plugin it is possible to do this every 1 hour, but this is not refined enough for me since it is bound to a schedule and the cache drive can easily be filled within 1 hour.

 

i am pretty astonished that this is is still not included in unraid, seems like such a basic feature to prevent unnecessary startups of the array while also assuring that the cache drive is used most efficiently.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.