Make more use of the cache/move schedules when moving files around


je82

Recommended Posts

As it stand now with unraid when you move a file from a share that is for example:

 

Share 1 (disk 1 , 2 ,3 included)

Move files to

Share 2 (disk 4, 5, 6 included)

 

You issue this command over SMB it will cause unraid to start moving files from Share 1 to the Cache drive for moving to Share 2 during nighttime when the mover runs. You're limited to the speed of disks 1 ,2 ,3 during this process. Lets say 50gb of data will usually take a couple of minutes to process as most harddrives are limited to around 150/180mb/s read speeds.

 

What i would want is to make an queue system for these kind of events and have another cronjob run before the regular mover runs that does these things right before the mover does its regular thing.

 

The result would be:

 

You issue a move request of files:

Share 1 (disk 1 , 2 ,3 included)

Move files to

Share 2 (disk 4, 5, 6 included)

 

The move would be nearly instant, because unraid just writes a log of which files it will move during night time to the cache and after this queue is processed the regular mover will run.

 

This way all movement within the array feels instant, you're limited to the amount of files you can move per day depending on your cache size.

I think it would be a great improvement to unraid making everything feel faster etc.

 

Thoughts? Issues?

 

Link to comment

Without really thinking it through, I see a various technical problems with that.

 

But, if you're really into instantaneous transfers between shares and using SMB, then why not experiment with

 

  • Don't limit shares to specific drives.  I've never particularly understood why people actually do it except for OCD reasons
  • Set up a root share for unRaid ( @SpaceInvaderOne has a video that covers that).
  • All moves now between shares are instant, without the cache drive involved at all

The first point is technically optional for this, but would alleviate the concerns as to why the files aren't actually confined onto the disks specified.

Link to comment
13 minutes ago, Squid said:

Without really thinking it through, I see a various technical problems with that.

 

But, if you're really into instantaneous transfers between shares and using SMB, then why not experiment with

 

  • Don't limit shares to specific drives.  I've never particularly understood why people actually do it except for OCD reasons
  • Set up a root share for unRaid ( @SpaceInvaderOne has a video that covers that).
  • All moves now between shares are instant, without the cache drive involved at all

The first point is technically optional for this, but would alleviate the concerns as to why the files aren't actually confined onto the disks specified.

Interesting, i will look into the root share, that may be a good way for me to deal with this.

The only real reason i limited my unraid shares to specific drives was in hopes of not having to "wake the entire array" everytime i want to access data. Attempting to separate the data i use personally and the data i just need to keep around into 2 different set of drives so i can access my regular data often without "waking the beast" so to say :)

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.