[Plugin] unbalanced


Recommended Posts

I am in the process of using unbalance and rsync (directly) to move data around a 14-drive array so I can slowly convert each disk to encrypted.

 

I'm about two weeks into the ordeal and I'm just over halfway done.  My 8TB disks are all that's left, and they are taking 50-60 hours per disk to copy data off of them and onto other member disks.

 

After this server is done, I have a second one to convert, in the same way.

 

I am averaging 35MB/s copy speeds.

 

I have set up the write method as per other threads, to attain the best possible speeds.  What else can I do to improve speeds when copying from one disk to another?  I believe I read something about disabling parity.....

Link to comment

I'm currently doing a bit of research into this gearing up to convert my disks. In this vid he disables the parity and uses copy to speed up the transfer speeds.

 

https://www.youtube.com/watch?v=uPPnY3jbRKg&index=3&list=PL7g5ef-W3a3o1WXPZrKPqdm7ewTxk39FI&t=0s

 

@9:50 he's getting 100MB speeds

 

 

 

Link to comment

Can unBalance's "Scatter" function move data between disks while *STILL* utilising the source disk?

 

I have a backup set that's taking up 7.1TB of a 10TB disk and a bunch of other disks that aren't very well utilised.  I want to scatter the backup set across disks but don't want to completely remove all the data from disk1.

 

Is this possible?  Thanks.

Link to comment
9 hours ago, jbrodriguez said:

I guess you could specify just some folders of the backup set to be scattered.

 

Depends on the size of your data mostly.

Yeah I looked into that.  You're right, though - that does achieve the desired result but is just more time-consuming than I'd like since the backup data is literally 100s of 1000s of tiny files that are anywhere from 1MB to 50MB.

 

Thanks for the suggestion, though. :)

Link to comment

I don't know if I have something misconfigured.  I've been running unbalance and it copies data to drive I indicate, but the rsync is not removing data from the source.   I'm having to go in an manually delete this data with mc.  I've run the Docker Safe Permissions run.  Any ideas on how to fix this?

Link to comment

Are you sure you're running a move operation (either scatter/gather) ?

 

Can you double check the permissions on the data that doesn't get deleted, for any issues ?

 

Finally, the logs should show any issues while removing the source folders. You could check them or send me the relevant parts to check.

Link to comment

Versions Unraid 6.6.7,  unBalance v2019.02.12b

 

I get the same results with scatter or gather.  

 

All rsyncs use the same flags as this: rsync -avPR -X "Joe/Software" "/mnt/disk3/"

 

It explicitly says skipping deletion, but I don't know if that is for source or target.

W: 2019/03/20 11:58:33 core.go:763: command:end:error(exit status 13)
I: 2019/03/20 11:58:33 core.go:767: command:retcode(0):exitcode(13)
I: 2019/03/20 11:58:33 core.go:1028: Command Finished
I: 2019/03/20 11:58:33 core.go:1041: Current progress: 100.00% done ~ 0s left (141003.94 MB/s)
W: 2019/03/20 11:58:33 core.go:960: skipping:deletion:(rsync command was flagged):(/mnt/disk3/Joe/NeedsOrganizing)
I: 2019/03/20 11:58:33 core.go:710: Command Started: (src: /mnt/disk4) rsync -avPR -X "Joe/Software" "/mnt/disk3/"
W: 2019/03/20 11:58:33 shell.go:51: flag:(rsync: failed to set times on "/mnt/disk3/Joe": Operation not permitted (1)

Seeing permissions of drwxrwxrwx nobody users for all directories and drw-rw-rw- nobody users for files.  

Edited by sacherjj
Link to comment

rsync was flagged due to error 13, "Errors with program diagnostics", according to its manual (some rsync erros messages are pretty opaque).

 

Because of this error, the plugin skipped deletion of the source (skipping deletion always refers to the source).

 

If you check the History tab, that particular operation should be shown with a yellow icon, as well as the specific rsync command that failed.

 

Additionally, the flagged rsync command should have a 'rmsrc' (remove source) button, so that you can manually delete the source if you want to.

 

The next rsync command in your log was flagged due to 'failed to set times', so it looks like you have some files with 'crazy' date/times.

 

Recapping:

- An rsync warned about an error 13 (probably due to irregular date/times on some file(s))

- The copy continued, but the source files weren't removed, because of the error/warning

- It's now up to you to decided if the copy on the target is valid to you. So you either

  a - Delete the source (manually or via the rmsrc button), keep the target

  b - Delete the target, fix the issues on the source, copy/move again

 

Hope this helps.

Link to comment

Helps quite a bit.  The part I was missing is seeing the pull down that gives the rmsrc commands to help easily clean up.   Still not sure why permissions are causing this, but working through that as I get to understand unraid better.

 

Thanks for the education.

Link to comment
  • 2 weeks later...
52 minutes ago, fschris said:

I went to the Apps section and did a search for 'unbalance' and nothing is found ?

unbala.PNG

Hmmm.  Shows up for me searching on that term.  I am not sure if it matters, but it looks like you might not have the latest version of CA.  That's an older version in your screenshot.

 

image.thumb.png.4d7ad9256c125c4ea30df2ca9fcba93a.png

Edited by Hoopster
Link to comment
11 hours ago, fschris said:

I went to the Apps section and did a search for 'unbalance' and nothing is found ?

unbala.PNG

10 hours ago, Hoopster said:

I am not sure if it matters, but it looks like you might not have the latest version of CA.  That's an older version in your screenshot.

 

That version is over a year old. You should make sure ALL your plugins are up-to-date.

Link to comment
  • 2 weeks later...
  • jbrodriguez changed the title to [Plugin] unbalanced

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.