[Plugin] unbalanced


Recommended Posts

Just now, jonathanm said:

Which feature of unbalance do you want to do the opposite of? unbalance does many things.

sorry ...

 

what i am meaning is that i have just manually moved stuff around to make space for a 3TB disk and now most of my disks are in the red and i know it's just a cosmetic thing...

i would like unraid via a plugin to balance all the files over the various disks.

 

thanks

Link to comment

Since your shares are currently scattered over your full disks, why not use unbalance's gather function to consolidate one of those shares to your mostly empty drive?

 

Here's the thing. Full vs. empty. You want folders that aren't going to be written to, only read, on mostly full drives, so your actively written folders can be on your mostly empty drives. It's not a matter of automatically shuffling things around, you need to make some intelligent choices over what files and folders go where. Seldom used stuff, fill those drives up and let them stay spun down almost all the time. Keep your activity to the least number of drives, and keep them as empty as possible so your write speeds are better.

 

Think of a full drive as "done", and partial drives as "in progress". If you keep chasing empty space around, you are going to be churning data every time you add a drive. Every time you move data around, you risk messing up and accidentally erasing something. Better to fill and move on.

Link to comment
13 hours ago, jbrodriguez said:

Hi wreave, you didn't send the whole log, but what I can see it's only transferring 4.68gb, it does seem like it's running a dry-run.

 

Can you send the first part of the log, with the calculate section ?

 

I am not quite sure where to find the full log file outside of the GUI.

Link to comment
12 hours ago, jonathanm said:

Since your shares are currently scattered over your full disks, why not use unbalance's gather function to consolidate one of those shares to your mostly empty drive?

 

Here's the thing. Full vs. empty. You want folders that aren't going to be written to, only read, on mostly full drives, so your actively written folders can be on your mostly empty drives. It's not a matter of automatically shuffling things around, you need to make some intelligent choices over what files and folders go where. Seldom used stuff, fill those drives up and let them stay spun down almost all the time. Keep your activity to the least number of drives, and keep them as empty as possible so your write speeds are better.

 

Think of a full drive as "done", and partial drives as "in progress". If you keep chasing empty space around, you are going to be churning data every time you add a drive. Every time you move data around, you risk messing up and accidentally erasing something. Better to fill and move on.

noted thank you

Link to comment

@jbrodriguez I'm a huge fan of where this app is going! I tried it earlier this week to move files off a single disk that I wanted to exclude from a user share and re-disturbe it across the existing shares still remaining. Worked flawlessly.

 

I'm very interested as it might be a means by which I can migrate to using the split level feature in UnRaid. Without UnBalance, such a migration would involve a lot of work.

 

You've saved me countless hours I'd otherwise be spending using midnight commander to do this manually. Thank you!!! I'd glad buy you beers and more beers. Can you update your donation link in your first post to include BitCoin or Ether?

Edited by Lev
Link to comment
2 hours ago, Lev said:

You've saved me countless hours I'd otherwise be spending using midnight commander to do this manually. Thank you!!! I'd glad buy you beers and more beers. Can you update your donation link in your first post to include BitCoin or Ether?

 

Thank you for the kind comments !

 

Yes, I will add a bitcoin option :)

Link to comment
29 minutes ago, wreave said:

Here you are. I tried again to day and grabbed all the days logs.

 

Thanks. 

 

The log looks clean.

 

I've added some logging to the app showing whether dry-run is on/off when starting an operation.

 

I can't think of a reason why it ran so fast.

 

How much ram do you have ?

 

Can you update the app to 3.3.1 and try again with full verbosity (change it in the settings page) and dry-run ?

Link to comment

Just had this happen to me, Gathered 12 Monkies and had to manually remove the directories on disk7 and 8.

 

Are you changing working directory to the src: drive?  Or using the full path to actually remove the directories?

 

I: 2017/08/18 12:20:14 core.go:775: Running Move operation ...
I: 2017/08/18 12:20:14 core.go:905: Command Started: (src: /mnt/disk7) rsync -avPRX "TV/12 Monkeys/Season 1" "/mnt/disk1/"
I: 2017/08/18 12:27:02 core.go:987: Command Finished
I: 2017/08/18 12:27:02 core.go:993: Current progress: 6.14% done ~ 1h44m2s left (53.75 MB/s)
I: 2017/08/18 12:27:02 core.go:1005: Removing: rm -rf "TV/12 Monkeys/Season 1"
I: 2017/08/18 12:27:02 core.go:905: Command Started: (src: /mnt/disk8) rsync -avPRX "TV/12 Monkeys/Season 2" "/mnt/disk1/"
I: 2017/08/18 12:29:36 core.go:987: Command Finished
I: 2017/08/18 12:29:36 core.go:993: Current progress: 8.86% done ~ 1h36m27s left (56.29 MB/s)
I: 2017/08/18 12:29:36 core.go:1005: Removing: rm -rf "TV/12 Monkeys/Season 2"
I: 2017/08/18 12:29:36 core.go:1092:
unBALANCE - MOVE operation completed


Started: Aug 18, 2017 12:20:14
Ended: Aug 18, 2017 12:29:36

Elapsed: 9m22.689s

Move operation has finished

Transferred 33.21 GB at ~ 56.29 MB/s

 

Edited by Tybio
Link to comment

Ok, here you go:

I: 2017/08/18 18:13:10 core.go:775: Running Move operation ...
I: 2017/08/18 18:13:10 core.go:905: Command Started: (src: /mnt/disk4) rsync -avPRX "TV/Andromeda/Season 1" "/mnt/disk3/"
I: 2017/08/18 18:13:58 core.go:987: Command Finished
I: 2017/08/18 18:13:58 core.go:993: Current progress: 1.06% done ~ 1h15m51s left (66.70 MB/s)
I: 2017/08/18 18:13:58 core.go:1005: Removing: rm -rf "TV/Andromeda/Season 1"

However:

root@Storage:/boot/logs# ls /mnt/disk4/TV/Andromeda/Season\ 1/
Andromeda\ -\ 01x01\ -\ Under\ the\ Night\ (1).avi              Andromeda\ -\ 01x12\ -\ The\ Mathematics\ of\ Tears.avi
Andromeda\ -\ 01x02\ -\ An\ Affirming\ Flame\ (2).avi           Andromeda\ -\ 01x13\ -\ Music\ of\ a\ Distant\ Drum.avi
Andromeda\ -\ 01x03\ -\ To\ Loose\ the\ Fateful\ Lightning.avi  Andromeda\ -\ 01x14\ -\ Harper\ 2.0.avi
Andromeda\ -\ 01x04\ -\ D\ Minus\ Zero.avi                      Andromeda\ -\ 01x15\ -\ Forced\ Perspective.avi
Andromeda\ -\ 01x05\ -\ Double\ Helix.avi                       Andromeda\ -\ 01x16\ -\ The\ Sum\ of\ Its\ Parts.avi
Andromeda\ -\ 01x06\ -\ Angel\ Dark,\ Demon\ Bright.avi         Andromeda\ -\ 01x17\ -\ Fear\ and\ Loathing\ in\ the\ Milky\ Way.avi
Andromeda\ -\ 01x07\ -\ The\ Ties\ That\ Blind.avi              Andromeda\ -\ 01x18\ -\ The\ Devil\ Take\ the\ Hindmost.avi
Andromeda\ -\ 01x08\ -\ The\ Banks\ of\ the\ Lethe.avi          Andromeda\ -\ 01x19\ -\ The\ Honey\ Offering.avi
Andromeda\ -\ 01x09\ -\ A\ Rose\ in\ the\ Ashes.avi             Andromeda\ -\ 01x20\ -\ Star-Crossed.avi
Andromeda\ -\ 01x10\ -\ All\ Great\ Neptune's\ Ocean.avi        Andromeda\ -\ 01x21\ -\ It\ Makes\ a\ Lovely\ Light.avi
Andromeda\ -\ 01x11\ -\ The\ Pearls\ that\ Were\ His\ Eyes.avi  Andromeda\ -\ 01x22\ -\ Its\ Hour\ Come\ 'Round\ at\ Last\ (1).avi

 

Link to comment

I'm also not sure the % complete for Gather is operating correctly, I just passed 65G of a ~235G gather operation where I'm moving all ~235G to a new disk and it is sitting at 3.15% complete, thus the time to completion seems way off.

 

Minor problem, but I thought I'd point it out!

 

Edit: The gather finished, it was at exactly 11.11% when complete.  Hope that helps.

Edited by Tybio
Link to comment
On 8/18/2017 at 6:35 PM, jbrodriguez said:

 

Thanks. 

 

The log looks clean.

 

I've added some logging to the app showing whether dry-run is on/off when starting an operation.

 

I can't think of a reason why it ran so fast.

 

How much ram do you have ?

 

Can you update the app to 3.3.1 and try again with full verbosity (change it in the settings page) and dry-run ?

Here you go.

https://pastebin.com/jCB0kFRt

 

I have 16 gigs of RAM

Link to comment
  • jbrodriguez changed the title to [Plugin] unbalanced

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.