[Plug-In] unBALANCE


1679 posts in this topic Last Reply

Recommended Posts

Can anyone help with this problem. I am unable to move the data because of these errors. 

Thank you

 

I: 2021/02/12 18:45:12 planner.go:509: disk(/mnt/disk1):no-items:currentFree(1.19 TB)

I: 2021/02/12 18:45:12 planner.go:510: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:511: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:512:

I: 2021/02/12 18:45:12 planner.go:508: =========================================================

I: 2021/02/12 18:45:12 planner.go:509: disk(/mnt/disk2):no-items:currentFree(504.92 GB)

I: 2021/02/12 18:45:12 planner.go:510: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:511: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:512:

I: 2021/02/12 18:45:12 planner.go:508: =========================================================

I: 2021/02/12 18:45:12 planner.go:509: disk(/mnt/disk3):no-items:currentFree(350.33 GB)

I: 2021/02/12 18:45:12 planner.go:510: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:511: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:512:

I: 2021/02/12 18:45:12 planner.go:508: =========================================================

I: 2021/02/12 18:45:12 planner.go:509: disk(/mnt/disk4):no-items:currentFree(488.64 GB)

I: 2021/02/12 18:45:12 planner.go:510: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:511: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:512:

I: 2021/02/12 18:45:12 planner.go:508: =========================================================

I: 2021/02/12 18:45:12 planner.go:509: disk(/mnt/disk5):no-items:currentFree(782.85 GB)

I: 2021/02/12 18:45:12 planner.go:510: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:511: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:512:

I: 2021/02/12 18:45:12 planner.go:508: =========================================================

I: 2021/02/12 18:45:12 planner.go:509: disk(/mnt/disk6):no-items:currentFree(516.84 GB)

I: 2021/02/12 18:45:12 planner.go:510: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:511: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:512:

I: 2021/02/12 18:45:12 planner.go:508: =========================================================

I: 2021/02/12 18:45:12 planner.go:509: disk(/mnt/disk7):no-items:currentFree(1.17 TB)

I: 2021/02/12 18:45:12 planner.go:510: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:511: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:512:

I: 2021/02/12 18:45:12 planner.go:508: =========================================================

I: 2021/02/12 18:45:12 planner.go:509: disk(/mnt/disk8):no-items:currentFree(1.99 TB)

I: 2021/02/12 18:45:12 planner.go:510: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:511: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:512:

I: 2021/02/12 18:45:12 planner.go:508: =========================================================

I: 2021/02/12 18:45:12 planner.go:509: disk(/mnt/disk9):no-items:currentFree(1.66 TB)

I: 2021/02/12 18:45:12 planner.go:510: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:511: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:512:

I: 2021/02/12 18:45:12 planner.go:508: =========================================================

I: 2021/02/12 18:45:12 planner.go:509: disk(/mnt/disk10):no-items:currentFree(1.81 TB)

I: 2021/02/12 18:45:12 planner.go:510: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:511: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:512:

I: 2021/02/12 18:45:12 planner.go:508: =========================================================

I: 2021/02/12 18:45:12 planner.go:509: disk(/mnt/disk11):no-items:currentFree(992.71 GB)

I: 2021/02/12 18:45:12 planner.go:510: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:511: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:512:

I: 2021/02/12 18:45:12 planner.go:497: =========================================================

I: 2021/02/12 18:45:12 planner.go:498: disk(/mnt/disk12):items(2)-(2.81 TB):currentFree(3.11 TB)-plannedFree(298.37 GB)

I: 2021/02/12 18:45:12 planner.go:499: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:502: [1.51 TB] /mnt/disk1/ServerPlus/./Video Projects

I: 2021/02/12 18:45:12 planner.go:502: [1.30 TB] /mnt/disk1/ServerPlus/./New File System

I: 2021/02/12 18:45:12 planner.go:505: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:506:

I: 2021/02/12 18:45:12 planner.go:508: =========================================================

I: 2021/02/12 18:45:12 planner.go:509: disk(/mnt/cache):no-items:currentFree(450.94 GB)

I: 2021/02/12 18:45:12 planner.go:510: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:511: ---------------------------------------------------------

I: 2021/02/12 18:45:12 planner.go:512:

I: 2021/02/12 18:45:12 planner.go:516: =========================================================

I: 2021/02/12 18:45:12 planner.go:517: Bytes To Transfer: 2.81 TB

I: 2021/02/12 18:45:12 planner.go:518: ---------------------------------------------------------

I: 2021/02/12 18:45:50 core.go:687: Running Move operation ...

I: 2021/02/12 18:45:50 core.go:710: Command Started: (src: /mnt/disk1) rsync -avPR -X "ServerPlus/Video Projects" "/mnt/disk12/"

I: 2021/02/12 18:45:50 core.go:185: Sending operation

I: 2021/02/12 18:45:58 core.go:835: monitor:transfer:(/mnt/disk1/ServerPlus/Video Projects)

W: 2021/02/12 18:46:09 shell.go:51: flag:(rsync: get_xattr_data: lgetxattr("/mnt/disk1/ServerPlus/Video Projects/Music/Patriotic March","user.com.apple.quarantine",67) failed: Input/output error (5)

)

W: 2021/02/12 18:46:10 core.go:763: command:end:error(exit status 13)

I: 2021/02/12 18:46:10 core.go:767: command:retcode(0):exitcode(13)

I: 2021/02/12 18:46:10 core.go:1028: Command Finished

I: 2021/02/12 18:46:10 core.go:1041: Current progress: 53.62% done ~ 17s left (71731.32 MB/s)

W: 2021/02/12 18:46:10 core.go:960: skipping:deletion:(rsync command was flagged):(/mnt/disk12/ServerPlus/Video Projects)

I: 2021/02/12 18:46:10 core.go:710: Command Started: (src: /mnt/disk1) rsync -avPR -X "ServerPlus/New File System" "/mnt/disk12/"

W: 2021/02/12 18:46:10 shell.go:51: flag:(rsync: get_xattr_data: lgetxattr("/mnt/disk1/ServerPlus/New File System/Cellular One","user.DosStream.com.apple.metadata:_kMDItemUserTags:$DATA",1) failed: Input/output error (5)

)

W: 2021/02/12 18:46:10 core.go:763: command:end:error(exit status 13)

I: 2021/02/12 18:46:10 core.go:767: command:retcode(0):exitcode(13)

I: 2021/02/12 18:46:10 core.go:1028: Command Finished

I: 2021/02/12 18:46:10 core.go:1041: Current progress: 100.00% done ~ 0s left (130523.86 MB/s)

W: 2021/02/12 18:46:10 core.go:960: skipping:deletion:(rsync command was flagged):(/mnt/disk12/ServerPlus/New File System)

I: 2021/02/12 18:46:10 core.go:1107:

unBALANCE - MOVE operation completed

Started: Feb 12, 2021 18:45:50

Ended: Feb 12, 2021 18:46:10

Elapsed: 20.531s

Move operation has finished

Transferred 2.81 TB at ~ 130523.70 MB/s

 

Link to post
  • Replies 1.7k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Introduction unBALANCE is a plugin to transfer files/folders between disks in your array.   Support Fund If you wish to do so, learn how to support the developer.   Video

v2019.10.26 (5.6.3) is available !   - Support 6.8.x - Add missing isNumber function - Upgrade some libraries - Switch to yarn

v4.0.0 is now available !   2017-12-05 - 4.0.0   This is major release bringing in new features to improve the end user experience !   Transfer page Here you can monito

Posted Images

  • 3 weeks later...

Stupid question...

I used the scatter option to move some video files.  I selected move but it appears that it copied because the files appear on both drives now.  What did I do wrong?  To fix this, is it as simple as deleting the files on the original disk?  If so, Is there anything that needs to be done with the parity drives?

 

When I browse to the share, I only see 1 copy of each file.

Edited by GPT347
Link to post
1 hour ago, GPT347 said:

I selected move but it appears that it copied because the files appear on both drives now.  What did I do wrong?

Move copies first, then deletes the source. If the process is interrupted, either manually or some error occurs, the delete won't happen.

1 hour ago, GPT347 said:

To fix this, is it as simple as deleting the files on the original disk?

Assuming the copy is accurate and complete, yes. The fact you have copies implies that may not be the case, see my first point.

You may want to verify the copy by using checksums or a program that compares files, there are many different options.

1 hour ago, GPT347 said:

Is there anything that needs to be done with the parity drives?

Parity doesn't contain files, so no, parity is either in sync or it's not. Parity is updated in realtime for every disk operation, be it copy, move, format, encrypt, whatever. As long as you don't have a hardware issue, or have an unclean shutdown, parity should stay in sync. Periodic (monthly is recommended) parity checks are a good idea to be sure parity has stayed in sync.

1 hour ago, GPT347 said:

When I browse to the share, I only see 1 copy of each file.

That's because the user shares only show the first unique instance of any identically named file in each folder on all the disks.

Link to post
15 hours ago, jonathanm said:

Move copies first, then deletes the source. If the process is interrupted, either manually or some error occurs, the delete won't happen.  Assuming the copy is accurate and complete, yes. The fact you have copies implies that may not be the case, see my first point.  You may want to verify the copy by using checksums or a program that compares files, there are many different options.

 

Thanks for the explanation.  It occurs to me though that I don't even know how to delete the original copy on the first disk.  Any chance you could explain the process?  I am able to view the files on the disk in Unraid under the main tab.  It doesn't appear to allow deletion from there.  

 

I have an MD5 checker but again, I don't know how to point that at the data on the disk, just the share overall.

Link to post
30 minutes ago, GPT347 said:

Thanks for the explanation.  It occurs to me though that I don't even know how to delete the original copy on the first disk.  Any chance you could explain the process?  I am able to view the files on the disk in Unraid under the main tab.  It doesn't appear to allow deletion from there.  

 

I have an MD5 checker but again, I don't know how to point that at the data on the disk, just the share overall.

Temporarily turn on disk shares, and you can connect to the individual disks like you do the user shares. Just be VERY careful to work exclusively in the disk shares while you have them active, DON'T copy or move between disk shares and user shares. Since disks and user shares show two views of the same files, it's easy to mess up and lose data, even if you think you are doing things correctly.

 

You can move disk to disk, or user share to user share, just don't mix between them.

Link to post
On 10/9/2020 at 2:33 PM, FlorinB said:

With multiple pools I am seeing this as multiple "cache" drives. Since the Mover is able to move the data from the pools to Array, it will be helfpul (at least for me) to use the unBalance as well for Scatter & Gather the data between Array and pools.

 

+1 for this

as unbalance was able to utilize the "old" cache drive, it would absolutely make sence (also for me ;) ) to have the same functionality with the multi-pools

Link to post

hello all, trying to move some data around and i'm getting the following error:

I: 2021/03/15 23:16:14 core.go:710: Command Started: (src: /mnt/disk10) rsync -avPR -X "TV/Show" "/mnt/disk6/"
I: 2021/03/15 23:16:15 core.go:767: command:retcode(0):exitcode(0)
I: 2021/03/15 23:16:15 core.go:1028: Command Finished
I: 2021/03/15 23:16:15 core.go:1041: Current progress: 100.00% done ~ 0s left (10792.87 MB/s)
I: 2021/03/15 23:16:15 core.go:972: removing:(rm -rf "/mnt/disk10/TV/Neds Declassified School Survival Guide")
W: 2021/03/15 23:16:15 shell.go:51: transferProgress::(rm: cannot remove '/mnt/disk10/TV/Show Guide/banner.jpg': Read-only file system)
W: 2021/03/15 23:16:15 shell.go:51: transferProgress::(rm: cannot remove '/mnt/disk10/TV/Show Guide/fanart.jpg': Read-only file system)
W: 2021/03/15 23:16:15 shell.go:99: transferProgress:: waitError: exit status 1
W: 2021/03/15 23:16:15 core.go:985: Unable to remove source folder (/mnt/disk10/TV/Show): exit status 1

 

it appears the moved work, the files exist on the new drive but they are also still on the source drive.

Link to post
11 hours ago, drawde said:

(rm: cannot remove '/mnt/disk10/TV/Show Guide/banner.jpg': Read-only file system)

Run a file system check on disk 10. Click on the disk in the main GUI, and follow the instructions.

Link to post

Hi, is this plugin still working with the current unraid?

 

I have dry run checked and greyed out, so am unable to turn it off and use it correctly.

Also getting that the site is insecure as certificate is invalid.

 

Edited by subwars
Link to post

Hello,

I have "dry run" checked and greyed out. My UnRaid version is 6.9.1 and UnBalance is version 5.6.3.

I reinstall the plugin UnBalance and this behavior still same.

 

Do you have any idea why checkbox "dry run" checked and greyed out?

 

Many Thanks

Link to post

Is this the right place for an unbalance feature request? Can future versions be built as a plug-in that is available natively via the unraid gui rather that a web plugin that only works on another port?

 

as the cert doesn’t work I have to use safari as chrome won’t let me load the page with a cert warning.

 

it would be so handy just to have this available via the unraid gui.

 

I know this would prob be a lot of work so just an idea.

Link to post
On 3/27/2021 at 11:27 AM, ViperSuicide said:

Is this the right place for an unbalance feature request? Can future versions be built as a plug-in that is available natively via the unraid gui rather that a web plugin that only works on another port?

 

as the cert doesn’t work I have to use safari as chrome won’t let me load the page with a cert warning.

 

it would be so handy just to have this available via the unraid gui.

 

I know this would prob be a lot of work so just an idea.

I think I've thought about it in the past, I think I thought of having a thin php wrapper for the actual UI, but I didn't make a proof of concept or anything, so I'm not really sure that's possible

Link to post
  • 2 weeks later...

Good morning from Germany :)

I have seen some strange behavior when i want to move some folders with unbalance.

 

PLANNING: Planning started
PLANNING: Scanning Movies/4K/Some Movie on /mnt/disk4
PLANNING: Checking issues ...
PLANNING: Getting items ...
PLANNING: Found /mnt/disk4/ Movies/4K/Some Movie (1.00 B)
PLANNING: Trying to allocate items to disk2 ...
PLANNING: Ended: Apr 12, 2021 07:01:53
PLANNING: Elapsed: 4ms
PLANNING: Planning Finished

 

So what drives me crazy is, that this is a 50GB 4K Bluray and it shows up as 1.00 B. The file itself is not corrupted, i can play it and everything works fine. This problem occurs only with some folders. Some of them show up as 1.00 B and some of them with the right amount of data.

When i start the "move" it takes about a second and says, that it is complete. But the file stays an the "old" disk. On the new disk i can find an empty folder.

 

Any ideas?

Thx!

 

Edit:

SOLVED it. I dont know why but that particular movie already was on disk 2 (the destination). On disk 4 was an empty folder. So the plugin was correct. I didnt know that if u use unbalance and move a folder there will be an empty folder on the source disk. Thx u!

Edited by pizzakante
Link to post
3 hours ago, pizzakante said:

Good morning from Germany :)

I have seen some strange behavior when i want to move some folders with unbalance.

 


PLANNING: Planning started
PLANNING: Scanning Movies/4K/Some Movie on /mnt/disk4
PLANNING: Checking issues ...
PLANNING: Getting items ...
PLANNING: Found /mnt/disk4/ Movies/4K/Some Movie (1.00 B)
PLANNING: Trying to allocate items to disk2 ...
PLANNING: Ended: Apr 12, 2021 07:01:53
PLANNING: Elapsed: 4ms
PLANNING: Planning Finished

 

So what drives me crazy is, that this is a 50GB 4K Bluray and it shows up as 1.00 B. The file itself is not corrupted, i can play it and everything works fine. This problem occurs only with some folders. Some of them show up as 1.00 B and some of them with the right amount of data.

When i start the "move" it takes about a second and says, that it is complete. But the file stays an the "old" disk. On the new disk i can find an empty folder.

 

Any ideas?

Thx!

Does this only happen if the file has a space in it? If you rename to SomeMovie does that work ok?

Link to post
  • 2 weeks later...
13 hours ago, UnraidDuck said:

Thanks for the update. Will there be a version that supports more than one cache pool drive? Mine still seem to only show one.

I see, probably there's a new 'api' to get the other cache pools.

I'll take a look, but it might take a bit of time.

Link to post

Im trying to consolodate my data from many small 3-4 TB drives onto a new 14TB drive. After selecting the drive in Unbalance and the folders and choosing the disk, I pressed plan.   Then nothing happens.  The unbalance logo still animates but nothing works it like its frozen

 

Im not sure why its not working, even after 30mins nothing happens

 

Im not sure what im doing wrong

Link to post
1 hour ago, RiDDiX said:

Maybe this is wrong here. But I have to say thank you for this great Plugin, because this helped me alot to get my drives sorted for me. 🙂

Thanks!

 

Perfect place to post it. ;)

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.