[DOCKER] unBALANCE

[Bigdady92]

Maybe I'm a complete idiot but I can't seem to get this to work.

 

I tell the system to move my files by giving it a folder.  In this case the top level of the file tree /mnt/user/data

 

From there I have a dozen subfolders that I put all my data into. 

 

Now when I tell unbalance to check/dry-run the files it says "Nothing to do."

 

What am I missing?

[Felix]

Maybe I'm a complete idiot but I can't seem to get this to work.

 

I tell the system to move my files by giving it a folder.  In this case the top level of the file tree /mnt/user/data

 

From there I have a dozen subfolders that I put all my data into. 

 

Now when I tell unbalance to check/dry-run the files it says "Nothing to do."

 

What am I missing?

 

Make sure you are giving it folders that are relative to /mnt/user?  E.g. if you have /mnt/user/movies on your filesystem, just give Unbalance "movies"

 

It looks like you are trying to put in "/mnt/user/data" which Unbalance would probably parse as /mnt/user/mnt/user/data...

[rm414]

Can I safely run this while my array is doing other stuff (Other dockers running, such as Plex and downloaders writing to/from disk), or should I shut things down so it's the only thing manipulating data?

[jbrodriguez]

Maybe I'm a complete idiot but I can't seem to get this to work.

 

I tell the system to move my files by giving it a folder.  In this case the top level of the file tree /mnt/user/data

 

From there I have a dozen subfolders that I put all my data into. 

 

Now when I tell unbalance to check/dry-run the files it says "Nothing to do."

 

What am I missing?

 

Make sure you are giving it folders that are relative to /mnt/user?  E.g. if you have /mnt/user/movies on your filesystem, just give Unbalance "movies"

 

It looks like you are trying to put in "/mnt/user/data" which Unbalance would probably parse as /mnt/user/mnt/user/data...

 

That's correct.

 

Thanks Felix !

[jbrodriguez]

Can I safely run this while my array is doing other stuff (Other dockers running, such as Plex and downloaders writing to/from disk), or should I shut things down so it's the only thing manipulating data?

 

unBALANCE should be the only process writing data to disks, otherwise results (depending on some factors) will be unpredictable.

[m3ki]

Sorry for cross posting but I think I may have posted in the wrong forum earlier:

 

This is my issue, hopefully someone can help:

Hi,

 

I am having a strange issue with consolidating a share, and getting the following error:

 

http://i.imgur.com/KDh1hT7.png[/img]

 

Share maybe has 250GB of data or so and the name if it matters: "Time Machine - Marina (Air)"

 

I don't see anything out of the ordinary in the log either

I: 2015/10/28 11:43:56 unraid.go:224: Unraid Box Condition: &{NumDisks:7 NumProtected:7 Synced:2015-10-28 03:47:05 -0500 CDT SyncErrs:0 Resync:0 ResyncPrcnt:0 ResyncPos:0 State:STARTED Size:29993064161280 Free:10742451089408 NewFree:10742451089408}
I: 2015/10/28 11:43:56 unraid.go:225: Unraid Box SourceDiskName: 
I: 2015/10/28 11:43:56 unraid.go:226: Unraid Box BytesToMove: 0
I: 2015/10/28 11:43:56 unraid.go:237: Id(1); Name(md1); Path(/mnt/disk1); Device(sdj), Free(2.2T); NewFree(2.2T); Size(4.5T); Serial(WDC_WD50EFRX-68MYMN1_WD-WX11D259VFSJ); Status(DISK_OK); Bin()
I: 2015/10/28 11:43:56 unraid.go:237: Id(2); Name(md2); Path(/mnt/disk2); Device(sdi), Free(1.1T); NewFree(1.1T); Size(4.5T); Serial(WDC_WD50EFRX-68MYMN1_WD-WX21D25R5VAR); Status(DISK_OK); Bin()
I: 2015/10/28 11:43:56 unraid.go:237: Id(3); Name(md3); Path(/mnt/disk3); Device(sde), Free(1.1T); NewFree(1.1T); Size(4.5T); Serial(WDC_WD50EFRX-68MYMN1_WD-WX21DB41SLHY); Status(DISK_OK); Bin()
I: 2015/10/28 11:43:56 unraid.go:237: Id(4); Name(md4); Path(/mnt/disk4); Device(sdd), Free(1.1T); NewFree(1.1T); Size(4.5T); Serial(WDC_WD50EFRX-68MYMN1_WD-WX21DC42ETNT); Status(DISK_OK); Bin()
I: 2015/10/28 11:43:56 unraid.go:237: Id(5); Name(md5); Path(/mnt/disk5); Device(sdh), Free(1.9T); NewFree(1.9T); Size(4.5T); Serial(WDC_WD50EFRX-68MYMN1_WD-WX21DC4D5E61); Status(DISK_OK); Bin()
I: 2015/10/28 11:43:56 unraid.go:237: Id(6); Name(md6); Path(/mnt/disk6); Device(sdf), Free(2.3T); NewFree(2.3T); Size(4.5T); Serial(WDC_WD50EFRX-68MYMN1_WD-WX21D1526VXH); Status(DISK_OK); Bin()
I: 2015/10/28 11:43:59 core.go:156: calculateBestFit:Begin:srcDisk(/mnt/disk5); dstDisks(5)
I: 2015/10/28 11:43:59 core.go:159: calculateBestFit:elegibleDestDisk(/mnt/disk1)
I: 2015/10/28 11:43:59 core.go:159: calculateBestFit:elegibleDestDisk(/mnt/disk2)
I: 2015/10/28 11:43:59 core.go:159: calculateBestFit:elegibleDestDisk(/mnt/disk3)
I: 2015/10/28 11:43:59 core.go:159: calculateBestFit:elegibleDestDisk(/mnt/disk4)
I: 2015/10/28 11:43:59 core.go:159: calculateBestFit:elegibleDestDisk(/mnt/disk6)
I: 2015/10/28 11:43:59 core.go:238: getFolders:Scanning source-disk(/mnt/disk5):folder(Time Machine - Marina (Air))
I: 2015/10/28 11:43:59 core.go:252: getFolders:Readdir(2)
I: 2015/10/28 11:43:59 core.go:267: getFolders:Executing find "/mnt/disk5/Time Machine - Marina (Air)/." ! -name . -prune -exec du -bs {} +
I: 2015/10/28 11:43:59 core.go:295: getFolders:find(/mnt/disk5/Time Machine - Marina (Air)/.): 10485796	/mnt/disk5/Time Machine - Marina (Air)/./.AppleDB
I: 2015/10/28 11:43:59 core.go:295: getFolders:find(/mnt/disk5/Time Machine - Marina (Air)/.): 169	/mnt/disk5/Time Machine - Marina (Air)/./Marina’s MacBook Air.sparsebundle
I: 2015/10/28 11:43:59 boot.go:48: unBALANCE v0.7.4-160.07ff3ab starting up ...
I: 2015/10/28 11:43:59 settings.go:124: Config file saved as: 
ReservedSpace=450000000
Folders=[Time Machine - Marina (Air)]
DryRun=true
Notifications=false
[email protected]
[email protected]
Host=smtp.gmail.com
Port=465
Encrypt=false
User=myaccount
I: 2015/10/28 11:43:59 settings.go:54: Config file loaded from (/config) as:
ReservedSpace=450000000
Folders=[Time Machine - Marina (Air)]
DryRun=true
Notifications=false
[email protected]
[email protected]
Host=smtp.gmail.com
Port=465
Encrypt=false
User=myaccount
I: 2015/10/28 11:43:59 socket.go:71: starting service Socket ...
I: 2015/10/28 11:43:59 server.go:30: Starting service Server ...
I: 2015/10/28 11:43:59 server.go:42: Serving files from /usr/local/share/unbalance
I: 2015/10/28 11:43:59 server.go:68: started listening on :6237
I: 2015/10/28 11:43:59 core.go:68: starting service Core ...
I: 2015/10/28 11:43:59 boot.go:63: Press Ctrl+C to stop ...
I: 2015/10/28 11:44:00 boot.go:48: unBALANCE v0.7.4-160.07ff3ab starting up ...
I: 2015/10/28 11:44:00 settings.go:124: Config file saved as: 
ReservedSpace=450000000
Folders=[Time Machine - Marina (Air)]
DryRun=true
Notifications=false
[email protected]
[email protected]
Host=smtp.gmail.com
Port=465
Encrypt=false
User=myaccount
I: 2015/10/28 11:44:00 settings.go:54: Config file loaded from (/config) as:
ReservedSpace=450000000
Folders=[Time Machine - Marina (Air)]
DryRun=true
Notifications=false
[email protected]
[email protected]
Host=smtp.gmail.com
Port=465
Encrypt=false
User=myaccount
I: 2015/10/28 11:44:00 socket.go:71: starting service Socket ...
I: 2015/10/28 11:44:00 server.go:30: Starting service Server ...
I: 2015/10/28 11:44:00 server.go:42: Serving files from /usr/local/share/unbalance
I: 2015/10/28 11:44:00 server.go:68: started listening on :6237
I: 2015/10/28 11:44:00 core.go:68: starting service Core ...
I: 2015/10/28 11:44:00 boot.go:63: Press Ctrl+C to stop ...

[kasper1705]

Hi everyone

I've heard there was some problems with unBALANCE in a previous version of unRaid, where it only will come with a message saying "The array is not operational. Please start the array first."

 

I also read there should come an update to fix this. But i am still encountering it. I have the lastest version of unraid installed (ver. 6.1.3), tried to restart both the server and the docker service. Tried to reinstall unBALANCE, but nothing works. Are there any compatibility problem with unBalance under Unraid ver. 6.1.3 or am I doing something wrong?.

 

I have also attached the log for unBalance, if that helps

Log_for_unBALANCE.pdf

[jbrodriguez]

...

...
I: 2015/10/28 11:43:59 core.go:295: getFolders:find(/mnt/disk5/Time Machine - Marina (Air)/.): 169	/mnt/disk5/Time Machine - Marina (Air)/./Marina’s MacBook Air.sparsebundle
I: 2015/10/28 11:43:59 boot.go:48: unBALANCE v0.7.4-160.07ff3ab starting up ...
...

 

Hi m3ki,

 

In the two lines above, it seems the app was restarted or did you do it manually ?

 

The other thing was that you had disk1 deselected as target, but the log file showed it as elegible, not sure if the screenshot was out of sync with the log.

 

Can you run the process from scratch and send me the log (pm me if you want) ?

[jbrodriguez]

Hi everyone

I've heard there was some problems with unBALANCE in a previous version of unRaid, where it only will come with a message saying "The array is not operational. Please start the array first."

 

I also read there should come an update to fix this. But i am still encountering it. I have the lastest version of unraid installed (ver. 6.1.3), tried to restart both the server and the docker service. Tried to reinstall unBALANCE, but nothing works. Are there any compatibility problem with unBalance under Unraid ver. 6.1.3 or am I doing something wrong?.

 

I have also attached the log for unBalance, if that helps

 

Hi kasper1705,

 

Please check that you're mapping "/usr/local/sbin":"/usr/local/sbin" in the volume mappings of the docker.

 

Let me know if that fixes it for you

[kasper1705]

That did it. I must have changed it in the volume mapping for /root by mistake. Thanks for the quick help

[m3ki]

...

...
I: 2015/10/28 11:43:59 core.go:295: getFolders:find(/mnt/disk5/Time Machine - Marina (Air)/.): 169	/mnt/disk5/Time Machine - Marina (Air)/./Marina’s MacBook Air.sparsebundle
I: 2015/10/28 11:43:59 boot.go:48: unBALANCE v0.7.4-160.07ff3ab starting up ...
...

 

pm sent

 

Hi m3ki,

 

In the two lines above, it seems the app was restarted or did you do it manually ?

 

The other thing was that you had disk1 deselected as target, but the log file showed it as elegible, not sure if the screenshot was out of sync with the log.

 

Can you run the process from scratch and send me the log (pm me if you want) ?

[Dephcon]

I added a new Seagate 4TB NAS disk to my array (my first Seagate disk) and unbalance shows it as having 0 bytes free and used and I am unable to shift data to it.

[jbrodriguez]

I added a new Seagate 4TB NAS disk to my array (my first Seagate disk) and unbalance shows it as having 0 bytes free and used and I am unable to shift data to it.

 

Hi Dephcon, please make sure that ...

 

Known Issue

--

If you add/format a drive, it will not be recognized by unBALANCE unless you reboot the server or restart the docker service (stop all containers, then $ /etc/rc.d/rc.docker restart).

Thanks to derbtv and dnoyeb for their input on this.

 

On the other hand, if you're copying a whole drive to the new disk, please consider advice such as this http://lime-technology.com/forum/index.php?topic=39707.msg411972#msg411972

[Calvin]

Just wanted to say thanks for this Docker.  Works great.

 

I do have a couple notes coming at this as a newbie.  While I've used unraid for many years I don't often use telnet or a command line with it.  Keep in mind I've upgraded from v5 to 6.  Discovered what dockers even are and played with unbalance all within the  last 2 days.  I have a couple small things you may wish to consider. 

 

It took me far to long to figure out why unbalance kept telling me there was nothing to do.  I know the example given is a movie dir or something, but i was looking to move an entire disk.  Nearly all other boxes during this weekend upgrade have been clickable and when I clicked on yours and nothing happened I made the mistake of assuming that blank meant all.  Once I put a / mark in there unbalance finally started doing something.  This of course is my fault.  I should have started with a small example and it would have likely made sense to me faster.  But when I saw it could do a dry run without actually touching my data I naturally tried the big job instead of the little one.

 

None of that is the fault of unbalance but making the setting box clickable or perhaps different wording on how things work might help.  Of course now that I know how and what its trying to do it all makes perfect sense when I go back and reread it.  But it might be confusing for new person looking at it for the first time.  Its also possible its just me. 

 

Another thing to consider is putting in some sort of notice that "yes" the program knows I clicked the "calculate" button and its currently working on it.  As I had an entire drive selected for moving it took a few min to come back with its calculations.  In the meantime I had stopped it, restarted it and played with the settings trying to figure out what I was doing wrong.  It was not until I looked at the cpu usage on the server that I realized that yes... unbalance was actually trying to do something and that I should give it a min to come back to me.  Once I gave it more then 20 seconds after clicking the calculate button,  it of course worked fine.

 

Last note.  I've done a few test and in one occasion it failed to move a few files, or so I thought.  Turns out it they are dupes.  I don't know if unbalance caused that or if they were like that already.  I did note however that the log file showed the duplicates as the reason.  But in the email I got, it said all was good and it was done.  You might wish to include the details in the email notice when something does not go fully as planned.

 

Overall.  Very nice.

 

 

 

As for unraid v6 itself.  Wow.  What an upgrade.  There is lots of new stuff to figure out.  The only bad thing.  My hardware is to old to support the VM stuff.  Guess I need to stick with the docker stuff and I have to wait till I get a new server for the VM side..  I'm just amazed my 8 year old hardware still works and its not having any issues with the new ver.

 

 

[squirrellydw]

Just wanted to say thanks for this Docker.  Works great.

 

It took me far to long to figure out why unbalance kept telling me there was nothing to do.  I know the example given is a movie dir or something, but i was looking to move an entire disk.  Nearly all other boxes during this weekend upgrade have been clickable and when I clicked on yours and nothing happened I made the mistake of assuming that blank meant all.  Once I put a / mark in there unbalance finally started doing something.  This of course is my fault.  I should have started with a small example and it would have likely made sense to me faster.  But when I saw it could do a dry run without actually touching my data I naturally tried the big job instead of the little one.

 

None of that is the fault of unbalance but making the setting box clickable or perhaps different wording on how things work might help.  Of course now that I know how and what its trying to do it all makes perfect sense when I go back and reread it.  But it might be confusing for new person looking at it for the first time.  Its also possible its just me. 

 

Another thing to consider is putting in some sort of notice that "yes" the program knows I clicked the "calculate" button and its currently working on it.  As I had an entire drive selected for moving it took a few min to come back with its calculations.  In the meantime I had stopped it, restarted it and played with the settings trying to figure out what I was doing wrong.  It was not until I looked at the cpu usage on the server that I realized that yes... unbalance was actually trying to do something and that I should give it a min to come back to me.  Once I gave it more then 20 seconds after clicking the calculate button,  it of course worked fine.

 

 

Overall.  Very nice.

 

 

I agree, and once you figure it out it seems to work great

[GoChris]

I'm not sure if this is supposed to work or not, if not it would be nice, but after starting the mover closing the web page and opening it again later. Or starting it, putting my pc to sleep, coming back in the morning and see where the progress is. It seemed to throw errors if I closed and reopened it.

[jbrodriguez]

Just wanted to say thanks for this Docker.  Works great.

 

It took me far to long to figure out why unbalance kept telling me there was nothing to do.  I know the example given is a movie dir or something, but i was looking to move an entire disk.  Nearly all other boxes during this weekend upgrade have been clickable and when I clicked on yours and nothing happened I made the mistake of assuming that blank meant all.  Once I put a / mark in there unbalance finally started doing something.  This of course is my fault.  I should have started with a small example and it would have likely made sense to me faster.  But when I saw it could do a dry run without actually touching my data I naturally tried the big job instead of the little one.

 

None of that is the fault of unbalance but making the setting box clickable or perhaps different wording on how things work might help.  Of course now that I know how and what its trying to do it all makes perfect sense when I go back and reread it.  But it might be confusing for new person looking at it for the first time.  Its also possible its just me. 

 

Another thing to consider is putting in some sort of notice that "yes" the program knows I clicked the "calculate" button and its currently working on it.  As I had an entire drive selected for moving it took a few min to come back with its calculations.  In the meantime I had stopped it, restarted it and played with the settings trying to figure out what I was doing wrong.  It was not until I looked at the cpu usage on the server that I realized that yes... unbalance was actually trying to do something and that I should give it a min to come back to me.  Once I gave it more then 20 seconds after clicking the calculate button,  it of course worked fine.

 

 

Overall.  Very nice.

 

 

I agree, and once you figure it out it seems to work great

 

Calvin and squirrellydw, thanks for your comments !

 

It's very good to receive usability comments from an outside point of view.

 

Since I was the one who coded the app, I know how to operate it, while for other users, especially first time users as you mentioned, some things may not be as straightforward as I would assume they'd be

 

I'm currently swamped with work, but I'm considering your suggestions for a future revision, which I hope to release by Jan.

 

So far, this is what I have in mind for the next version (including previous requests):

 

- Dropdown (or similar) to input share names in the settings page

- Plug into unRAID's new notification system (rather than rolling my own)

- Show files/folders that won't be moved (due to space restrictions) right after calculations have completed (instead of at the end of the move operation)

- Rewording of the settings page

- Progress report during calculation phase

 

Thanks again for the feedback !

[jbrodriguez]

I'm not sure if this is supposed to work or not, if not it would be nice, but after starting the mover closing the web page and opening it again later.  ...

 

Hi GoChris, this should work. The progress 'console' will only show what's happening after the page was reopened.

Did you have a different experience ?

 

... Or starting it, putting my pc to sleep, coming back in the morning and see where the progress is. It seemed to throw errors if I closed and reopened it.

Is it possible that the move operation had finished when you woke up your pc ?

What errors were you getting ?

[dirrtyjoe]

My TV share throws an error on Calculate:

 

"Cannot read property 'condition' of undefined"

 

and

 

"XHR Failed for calculateBestFit"

 

 

Am I missing something?

[dirrtyjoe]

So some of the errors I was running into lead me to clean up any Apple files and that seems to fix the problem I was having when adding my TV share. Not sure if that will be the case with everyone else but I thought I would share.

 

 

.DS_Store

.AppleDB

.AppleDouble

etc....

[squirrellydw]

Someone should make a plugin or docker to remove these

[dirrtyjoe]

Someone should make a plugin or docker to remove these

 

There used to be an unMenu plugin and I found this topic that discusses it and a shell script to help out:

 

https://lime-technology.com/forum/index.php?topic=41128.0

 

[dirrtyjoe]

FYI: another item that was throwing errors for me was permissions on the files/folders I was looking to move.

[jbrodriguez]

Thanks for your feedback @dirrtyjoe.

 

Indeed, there seem to be some problems with Apple files.

 

With m3ki, we also saw that Time Machines wouldn't be mounted inside the docker, so they wouldn't be moved.

 

There's not much that I can currently do about this, though 

 

 

[wgstarks]

I haven't used this docker for a while. Evidently something changed in one of the unRaid updates maybe? When I access the UI I get an error that my array must be started. The array is started. How can I fix this?