Jump to content
jbrodriguez

[Plug-In] unBALANCE

1244 posts in this topic Last Reply

Recommended Posts

So i just tried deleting that folder contents on my windows machine, and

windows popup reporting as 1.0EB

Im baffled!

image.thumb.png.997ea3524ad8a5b29738ff99a24c19e4.png

Share this post


Link to post
1 hour ago, bdydrp said:

So i just tried deleting that folder contents on my windows machine, and

windows popup reporting as 1.0EB

Im baffled!

 

Check the filesystem on that disk.

 

Share this post


Link to post
6 hours ago, jbrodriguez said:

As for best practices, you should use unbalance exclusively (no other writes to at least the target disk(s)).

 

Turbo Write (unraid settings) should get you a speed bump.

This plugin is awesome, so easy to use! Thanks!!

Share this post


Link to post

The main reason im using this, is to be able to change file system on 2 of my drives.

Once i have change both drives, and i want to re-scatter files around, im going to assume that unbalance still follows the split levels set by each share?

Share this post


Link to post

Hi bdydrp,

 

It works at disk level, not share level, so it doesn't use/see the split levels you set.

Share this post


Link to post
11 minutes ago, jbrodriguez said:

Hi bdydrp,

 

It works at disk level, not share level, so it doesn't use/see the split levels you set.

Thanks..

Share this post


Link to post

Have just run another instance. Moving contents of Disk 2 to Disk 1

Instead of Move, it seems to have copied the files instead.

image.thumb.png.5ff9f711578c5095976d5deeaa6c2bd2.png

 

Could it be because, for some reason disk 2  is set to read only in Fix Common Problems

 

image.thumb.png.77d3574e80e3a298fae403b72e0e3034.png

 

Is there a quick way to verify all of disk 2 contents are on disk 1?

apart from exporting disk shares, and using explorer to cross reference?

 

Cheers

Share this post


Link to post

I can see from this screenshot, that all Disk 2 is the same size as Disk 1, for each share (except pc backups, but that fine)

Is this good sign that it indeed copied everything to disk 1?

image.thumb.png.f3ff977027289d2567b7fd31bbe2e897.png

Share this post


Link to post
30 minutes ago, bdydrp said:

Could it be because, for some reason disk 2  is set to read only in Fix Common Problems

Fix Common Problems is just reporting that it is set to read only. The Unraid OS set it to read only due to filesystem corruption so it won't get worse. And when it is read only obviously nothing can be removed from it.

Share this post


Link to post

So the move cmd in unbalance will move the files (or this case copy them) and not delete from source?

 

Share this post


Link to post
32 minutes ago, bdydrp said:

So the move cmd in unbalance will move the files (or this case copy them) and not delete from source?

It will delete the source files if the rsync (copy) command was successful, which seems to be the case as per you screenshot.

 

If you send me the log (/boot/logs/unbalance.log), I can take a look if it was unable to delete the source for some reason (pm if you prefer).

Share this post


Link to post
40 minutes ago, bdydrp said:

So the move cmd in unbalance will move the files (or this case copy them) and not delete from source?

Nothing can delete from a read only source, not any process. Your problem has nothing to do with unBALANCE.

 

You already said your source was reported as read only. Everything is working as it should. It would delete from the source but it can't because the source is read only. You must have filesystem corruption on the source disk.

Share this post


Link to post

Thanks - I was double checking that even tho i had used the move cmd, files were still copied to the target disk even tho disk 2 had errors.

Which Yes it did. Disk 2 had an error, so in this case, file were copied from Disk 2 to disk 1. leaving file on source disk.

Knowing i had duplicate files, i could safely format disk 2.

Share this post


Link to post
5 minutes ago, trurl said:

You already said your source was reported as read only.

Oh ok, thanks trurl, I hadn't read that through.

Share this post


Link to post
4 hours ago, bdydrp said:

I was double checking that even tho i had used the move cmd, files were still copied to the target disk even tho disk 2 had errors.

Remember that move is just like "copy then delete".  If the source disk can't be altered, then the delete portion can't run.

Share this post


Link to post

I can get the planning phase to work fine on my second server, but the "move" and "copy" buttons are still not available to click.  Here is the output:

 

--

 

I: 2019/02/11 16:49:05 app.go:51: unbalance v5.4.0-1094-9eff134-v2018.09.18a starting ...

I: 2019/02/11 16:49:05 app.go:59: No config file specified. Using app defaults ...

I: 2019/02/11 16:49:05 server.go:77: Starting service Server ...

I: 2019/02/11 16:49:05 server.go:94: Serving files from /usr/local/emhttp/plugins/unbalance

I: 2019/02/11 16:49:05 array.go:46: starting service Array ...

I: 2019/02/11 16:49:05 server.go:155: Server started listening https on :6238

I: 2019/02/11 16:49:05 planner.go:52: starting service Planner ...

I: 2019/02/11 16:49:05 core.go:101: starting service Core ...

I: 2019/02/11 16:49:05 server.go:145: Server started listening http on :6237

W: 2019/02/11 16:49:05 core.go:116: Unable to read history: open /boot/config/plugins/unbalance/unbalance.hist: no such file or directory

I: 2019/02/11 16:49:05 app.go:73: Press Ctrl+C to stop ...

I: 2019/02/11 16:52:40 core.go:175: Sending config

I: 2019/02/11 16:52:40 core.go:180: Sending state

I: 2019/02/11 16:52:40 core.go:190: Sending storage

I: 2019/02/11 16:53:58 planner.go:70: Running scatter planner ...

I: 2019/02/11 16:53:58 planner.go:84: scatterPlan:source:(/mnt/disk7)

I: 2019/02/11 16:53:58 planner.go:86: scatterPlan:dest:(/mnt/disk1)

I: 2019/02/11 16:53:58 planner.go:86: scatterPlan:dest:(/mnt/disk2)

I: 2019/02/11 16:53:58 planner.go:86: scatterPlan:dest:(/mnt/disk3)

I: 2019/02/11 16:53:58 planner.go:86: scatterPlan:dest:(/mnt/disk4)

I: 2019/02/11 16:53:58 planner.go:86: scatterPlan:dest:(/mnt/disk6)

I: 2019/02/11 16:53:58 planner.go:86: scatterPlan:dest:(/mnt/disk5)

I: 2019/02/11 16:53:58 planner.go:525: planner:array(7 disks):blockSize(4096)

I: 2019/02/11 16:53:58 planner.go:527: disk(/mnt/disk1):fs(btrfs):size(3000592928768):free(142756143104):blocksTotal(732566633):blocksFree(34852574)

I: 2019/02/11 16:53:58 planner.go:527: disk(/mnt/disk2):fs(btrfs):size(3000592928768):free(163589238784):blocksTotal(732566633):blocksFree(39938779)

I: 2019/02/11 16:53:58 planner.go:527: disk(/mnt/disk3):fs(btrfs):size(3000592928768):free(186394562560):blocksTotal(732566633):blocksFree(45506485)

I: 2019/02/11 16:53:58 planner.go:527: disk(/mnt/disk4):fs(btrfs):size(3000592928768):free(187017498624):blocksTotal(732566633):blocksFree(45658569)

I: 2019/02/11 16:53:58 planner.go:527: disk(/mnt/disk5):fs(btrfs):size(4000786976768):free(2186855804928):blocksTotal(976754633):blocksFree(533900343)

I: 2019/02/11 16:53:58 planner.go:527: disk(/mnt/disk6):fs(btrfs):size(3000592928768):free(1202956013568):blocksTotal(732566633):blocksFree(293690433)

I: 2019/02/11 16:53:58 planner.go:527: disk(/mnt/disk7):fs(btrfs):size(2000398901248):free(186992017408):blocksTotal(488378638):blocksFree(45652348)

I: 2019/02/11 16:53:58 planner.go:356: scanning:disk(/mnt/disk7):folder(backups)

W: 2019/02/11 17:02:06 planner.go:367: issues:not-available:(exit status 1)

W: 2019/02/11 17:02:08 planner.go:383: items:not-available:(exit status 1)

I: 2019/02/11 17:02:08 planner.go:466: scatterPlan:No items can be transferred.

I: 2019/02/11 17:02:08 planner.go:493: scatterPlan:ItemsLeft(0)

I: 2019/02/11 17:02:08 planner.go:494: scatterPlan:Listing (7) disks ...

I: 2019/02/11 17:02:08 planner.go:508: =========================================================

I: 2019/02/11 17:02:08 planner.go:509: disk(/mnt/disk1):no-items:currentFree(142.76 GB)

I: 2019/02/11 17:02:08 planner.go:510: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:511: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:512:

I: 2019/02/11 17:02:08 planner.go:508: =========================================================

I: 2019/02/11 17:02:08 planner.go:509: disk(/mnt/disk2):no-items:currentFree(163.59 GB)

I: 2019/02/11 17:02:08 planner.go:510: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:511: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:512:

I: 2019/02/11 17:02:08 planner.go:508: =========================================================

I: 2019/02/11 17:02:08 planner.go:509: disk(/mnt/disk3):no-items:currentFree(186.39 GB)

I: 2019/02/11 17:02:08 planner.go:510: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:511: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:512:

I: 2019/02/11 17:02:08 planner.go:508: =========================================================

I: 2019/02/11 17:02:08 planner.go:509: disk(/mnt/disk4):no-items:currentFree(187.02 GB)

I: 2019/02/11 17:02:08 planner.go:510: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:511: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:512:

I: 2019/02/11 17:02:08 planner.go:508: =========================================================

I: 2019/02/11 17:02:08 planner.go:509: disk(/mnt/disk5):no-items:currentFree(2.19 TB)

I: 2019/02/11 17:02:08 planner.go:510: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:511: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:512:

I: 2019/02/11 17:02:08 planner.go:508: =========================================================

I: 2019/02/11 17:02:08 planner.go:509: disk(/mnt/disk6):no-items:currentFree(1.20 TB)

I: 2019/02/11 17:02:08 planner.go:510: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:511: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:512:

I: 2019/02/11 17:02:08 planner.go:508: =========================================================

I: 2019/02/11 17:02:08 planner.go:509: disk(/mnt/disk7):no-items:currentFree(186.99 GB)

I: 2019/02/11 17:02:08 planner.go:510: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:511: ---------------------------------------------------------

I: 2019/02/11 17:02:08 planner.go:512:

I: 2019/02/11 17:02:08 planner.go:516: =========================================================

I: 2019/02/11 17:02:08 planner.go:517: Bytes To Transfer: 0B

I: 2019/02/11 17:02:08 planner.go:518: ---------------

I: 2019/02/11 17:02:08 planner.go:466: scatterPlan:No items can be transferred.

 

 

 

 

--

 

This is the line that is getting me confused:

 

I: 2019/02/11 17:02:08 planner.go:466: scatterPlan:No items can be transferred.

 

 

Not sure how to troubleshoot this, the health of the array is fine. 

 

Thanks.

Share this post


Link to post

It shows two warnings, for commands that exited with an error

9 hours ago, tucansam said:

W: 2019/02/11 17:02:06 planner.go:367: issues:not-available:(exit status 1)

W: 2019/02/11 17:02:08 planner.go:383: items:not-available:(exit status 1) 

The first one is for command

find "/mnt/disk7/backups/." -exec stat --format "%%A|%%U:%%G|%%F|%%n" {} \;

The second is for

find "/mnt/disk7/backups/." ! -name . -prune -exec du -bs {} +

Run them from the command line and check what errors you get

Share this post


Link to post

Thanks!

 

The first command returns a zillion pages of this:


%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
%A|%U:%G|%F|%n
 

And the second one returns this:

 

root@ffs1:~# find "/mnt/disk7/backups/." ! -name . -prune -exec du -bs {} +    

1594424381632   /mnt/disk7/backups/./scripted

 

Share this post


Link to post

v2019.02.19b (5.5.0) is out !

 

This is overall a very small release.

 

There was a bug that prevented Reserved Space to be saved.

 

Also, rsync error 13 (Errors with program diagnostics) is now also
flagged so it won't stop the whole operation, you will still need
to check the logs as for the reasons of the error.

 

Changes:

 

- Allow ReservedAmount/ReservedUnit to be saved to disk
- Add error 13 as flagged command
- Core libraries updated

Share this post


Link to post
9 minutes ago, tucansam said:

%A|%U:%G|%F|%n

That's odd. You included all the quotes right ?

 

Do you have some shell other than bash or zsh on your server ?

 

The du command output looks fine.

 

At a loss for ideas, maybe try the most recent version (5.5.0) ?

Share this post


Link to post

Yessir, I copied-and-pasted directly from your post.  I do not have any fancy shells, just plain vanilla unraid with few modifications.  

 

Right now I am using rsync by hand to move data, but its nowhere near as elegant as your plugin.

Share this post


Link to post
13 hours ago, tucansam said:

Yessir, I copied-and-pasted directly from your post.

Try typing it out instead. copy paste has been known to insert some invisible weird characters. Alternatively, copy paste into an intermediary text editor, one that doesn't honour formatting and html, text only.

Share this post


Link to post
4 minutes ago, jonathanm said:

Try typing it out instead. copy paste has been known to insert some invisible weird characters. Alternatively, copy paste into an intermediary text editor, one that doesn't honour formatting and html, text only.

Same result, same output, when typed by hand.  Good suggestion though.

Share this post


Link to post

Not sure when it started, but Unbalance will no longer run for me, just says 'stopped'. I noticed on 6.7.0-rc3 so I upgraded to 6.7.0-rc4 and still the same. Am I missing something?

 

 

Screen Shot 2019-02-18 at 6.47.32 am.png

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.