[Plugin] unbalanced

jbrodriguez

By jbrodriguez
in Plugin Support

Recommended Posts

  • Replies 2.1k
  • Created
  • Last Reply

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

jbrodriguez
jbrodriguez

Introduction unbalanced is a plugin to transfer files/folders between disks in your array.   Support Fund If you wish to do so, learn how to support the developer.   Desc

jbrodriguez
jbrodriguez

✭ unBALANCE is now unbalanced !! ✭     NOTE: To install it, look for "unbalanced" in Community Apps (the old unbalance plugin will be deprecated by Squid)   i had the op

jbrodriguez
jbrodriguez

there's something in the works with unbalance(d), i thought i could be done by now, turns out that's not the case 😀   happy holidays to all !!   teaser pic:  

Posted Images

jbrodriguez Rising Star

jbrodriguez

Posted
  • Author
Posted

v1.8.3 is available !

 

2016-11-25 - 1.8.3

- Keep separate config files (one for plugin system, one for app)

 

Well, just restored functionality as it was in 1.8.0 (separate config files), but made some changes to how the server component handles the settings.

 

If you installed 1.8.1, 1.8.2 or are having any problems, delete /boot/config/plugins/unbalance/unbalance.conf (you will need to setup your options again :o)

Link to comment
jbrodriguez Rising Star

jbrodriguez

Posted
  • Author
Posted

Performed a dry run calc and it gave me all these permissions issues. As instructed, ran Fix Common Problems and there were no problems with permissions. I don't use Dockers.

 

Please advise.

 

Hi Fuggin, those permissions issues are more like warnings.

 

If Fix Common Problems doesn't find any issues, you should be ok.

 

Also, if you run calc again after Fix Common Problems, do you still get the warnings ?

 

In any case, I'll check if I'm on sync with FCP issue detection.

Link to comment
Fuggin Contributor

Fuggin

Posted
Posted

Performed a dry run calc and it gave me all these permissions issues. As instructed, ran Fix Common Problems and there were no problems with permissions. I don't use Dockers.

 

Please advise.

 

Hi Fuggin, those permissions issues are more like warnings.

 

If Fix Common Problems doesn't find any issues, you should be ok.

 

Also, if you run calc again after Fix Common Problems, do you still get the warnings ?

 

In any case, I'll check if I'm on sync with FCP issue detection.

 

Yes...I get the warnings all the time even after doing a rescan of FCP. I just didn't want to risk moving anything until I was sure. Thx!

Link to comment
mgladwin Apprentice

mgladwin

Posted
Posted

Hi all,

 

I have a similar problem with the permission errors in unBALANCE.

 

I run a 'calculate' and it returns 12,000 "file" permission errors (no "directory" issues) such as:-

 

12000 files(s) with a permission other than '-rw-rw-rw-' or '-r--r--r--'

 

I run the FCP 'Docker safe' permissions tool and it seems to do nothing to as far as unBALANCE is concerned. Running another 'calculate' returns the same amount of errors.

 

I manually edited permissions using the command line and got it down to 100 or so 'file' permission errors but it created two new 'directory' issues. I then ran the FCP tool again thinking this might work now however it fixed the directory issues but returned the 12000 file permission errors again.

 

I also found a couple of scripts from gfjardim on these forums and they were:-

 

find /mnt/user/ -type f ! \( -user "nobody" -o -group "users" -o -perm "660" \) -exec chmod 660 '{}' \; -exec chown nobody:users '{}' \;
find /mnt/user/ -type d ! \( -user "nobody" -o -group "users" -o -perm "770" \) -exec chmod 770 '{}' \; -exec chown nobody:users '{}' \;

 

I cant follow exactly what the commands are doing (lack of knowledge) but i assume its 'find everything that isn't nobody:users and perms 770 for dirs (or 660 for files) and changes it to this.

 

I ran these two commands on /mnt/disk1/ as going by the unBALANCE log this is disk with the 'offending' files on it.

 

I also tried changing a portion of the scripts above to -exec chmod 666 as this is what i believe unBALANCE is looking for but this also didn't work.

 

I don't want to run the move without correct permissions for obvious reasons and would love to get this working properly.

I am sure this is user or configuration error my end but i just cant work it out.

 

Any help appreciated.

 

Cheers.

 

I tried both of these and they also didn't remove the permission issues.

Link to comment
jbrodriguez Rising Star

jbrodriguez

Posted
  • Author
Posted

...

I manually edited permissions using the command line and got it down to 100 or so 'file' permission errors but it created two new 'directory' issues. I then ran the FCP tool again thinking this might work now however it fixed the directory issues but returned the 12000 file permission errors again.

...

 

Hi mgladwin,

 

Hopefully you can help me look into this.

 

In your log file (/boot/logs/unbalance.log), you should have a lot of lines starting with these messages:

 

perms:User != nobody

perms:Group != users

perms:Folder perms != rwxrwxrwx

perms:File perms != rw-rw-rw- or -r--r--r--

 

(mostly the last one, by what you're saying)

 

Can you sample a couple of them and check the actual owner/permission on your disk ?

 

Feel free to pm, if you need some assistance.

 

Link to comment
mgladwin Apprentice

mgladwin

Posted
Posted

...

I manually edited permissions using the command line and got it down to 100 or so 'file' permission errors but it created two new 'directory' issues. I then ran the FCP tool again thinking this might work now however it fixed the directory issues but returned the 12000 file permission errors again.

...

 

Hi mgladwin,

 

Hopefully you can help me look into this.

 

In your log file (/boot/logs/unbalance.log), you should have a lot of lines starting with these messages:

 

perms:User != nobody

perms:Group != users

perms:Folder perms != rwxrwxrwx

perms:File perms != rw-rw-rw- or -r--r--r--

 

(mostly the last one, by what you're saying)

 

Can you sample a couple of them and check the actual owner/permission on your disk ?

 

Feel free to pm, if you need some assistance.

 

Hi jbrodriguez,

 

Thanks for your reply.

 

Here are a couple of lines from my unBALANCE log as requested:-

 

I: 2016/12/09 11:16:50 core.go:701: perms:File perms != rw-rw-rw- or -r--r--r--: [rw-rw-rw-]: /mnt/disk1/STORAGE/./AU D40/data/MENU2.TTF
I: 2016/12/09 11:16:50 core.go:701: perms:File perms != rw-rw-rw- or -r--r--r--: [rw-rw-rw-]: /mnt/disk1/STORAGE/./AU D40/data/MENU3.DAH

 

Here is the permissions as I see them on disk:-

 

-rw-rw-rw- 1 nobody users 117028 Mar 31  2003 MENU2.TTF
-rw-rw-rw- 1 nobody users   9578 Jul  5  2005 MENU3.DAH

 

I hope this is what you meant.

 

Are those lines in the log checking the file perms are either rw-rw-rw- or -r--r--r-- ?

Or are the lines in the log the permission warnings?

Link to comment
jbrodriguez Rising Star

jbrodriguez

Posted
  • Author
Posted

...

Here are a couple of lines from my unBALANCE log as requested:-

 

I: 2016/12/09 11:16:50 core.go:701: perms:File perms != rw-rw-rw- or -r--r--r--: [rw-rw-rw-]: /mnt/disk1/STORAGE/./AU D40/data/MENU2.TTF
I: 2016/12/09 11:16:50 core.go:701: perms:File perms != rw-rw-rw- or -r--r--r--: [rw-rw-rw-]: /mnt/disk1/STORAGE/./AU D40/data/MENU3.DAH

 

Here is the permissions as I see them on disk:-

 

-rw-rw-rw- 1 nobody users 117028 Mar 31  2003 MENU2.TTF
-rw-rw-rw- 1 nobody users   9578 Jul  5  2005 MENU3.DAH

 

I hope this is what you meant.

 

Are those lines in the log checking the file perms are either rw-rw-rw- or -r--r--r-- ?

Or are the lines in the log the permission warnings?

 

Thanks for the quick reply mgladwin !

 

Yes, the app checks that the files have either a "rw-rw-rw-" or a "r--r--r--" permission.

 

But I see an issue in the code: the check is inverted  :o

 

I'm uploading a release to fix this.

 

Link to comment
mgladwin Apprentice

mgladwin

Posted
Posted

...

Here are a couple of lines from my unBALANCE log as requested:-

 

I: 2016/12/09 11:16:50 core.go:701: perms:File perms != rw-rw-rw- or -r--r--r--: [rw-rw-rw-]: /mnt/disk1/STORAGE/./AU D40/data/MENU2.TTF
I: 2016/12/09 11:16:50 core.go:701: perms:File perms != rw-rw-rw- or -r--r--r--: [rw-rw-rw-]: /mnt/disk1/STORAGE/./AU D40/data/MENU3.DAH

 

Here is the permissions as I see them on disk:-

 

-rw-rw-rw- 1 nobody users 117028 Mar 31  2003 MENU2.TTF
-rw-rw-rw- 1 nobody users   9578 Jul  5  2005 MENU3.DAH

 

I hope this is what you meant.

 

Are those lines in the log checking the file perms are either rw-rw-rw- or -r--r--r-- ?

Or are the lines in the log the permission warnings?

 

Thanks for the quick reply mgladwin !

 

Yes, the app checks that the files have either a "rw-rw-rw-" or a "r--r--r--" permission.

 

But I see an issue in the code: the check is inverted  :o

 

I'm uploading a release to fix this.

 

Great work! All my warnings have gone. Thanks jbrodriguez!

Link to comment
Bizmo Noob

Bizmo

Posted
Posted

Bizmo, ok .. I thought that the first command was the one crashing.

 

The message is just a warning, since there are multiple instances where permission issues cause problems with rsync. The Docker Safe New Permissions resets permissions in the best possible way.

 

I had another idea ... when it's sitting idle as you mention .. can you do a 

ps aux | grep unbalance

  and a 

ps aux | grep find

... that would tell us if it's really running or not

 

Sorry it took me a while to get back, still having the same problem here.

 

The output to those is as follows

 

BEFORE hitting calculate

 

root@Tower:~# ps aux | grep unbalance
nobody    7700  0.0  0.0  13312  9800 ?        Sl   09:06   0:00 /usr/local/emhttp/plugins/unbalance/unbalance -port 6237
root     29430  0.0  0.0   9652  1840 pts/0    S+   16:31   0:00 grep unbalance
root@Tower:~# ps aux | grep find
root     29583  0.0  0.0   9652  1880 pts/0    S+   16:31   0:00 grep find

 

AFTER hitting calculate

 

root@Tower:~# ps aux | grep unbalance
root     30233  0.0  0.0   9652  1800 pts/0    S+   16:32   0:00 grep unbalance
root@Tower:~# ps aux | grep find     
root     30243  0.0  0.0   9652  1792 pts/0    S+   16:32   0:00 grep find

Link to comment
jbrodriguez Rising Star

jbrodriguez

Posted
  • Author
Posted

...

BEFORE hitting calculate

 

root@Tower:~# ps aux | grep unbalance
nobody    7700  0.0  0.0  13312  9800 ?        Sl   09:06   0:00 /usr/local/emhttp/plugins/unbalance/unbalance -port 6237
root     29430  0.0  0.0   9652  1840 pts/0    S+   16:31   0:00 grep unbalance
root@Tower:~# ps aux | grep find
root     29583  0.0  0.0   9652  1880 pts/0    S+   16:31   0:00 grep find

 

AFTER hitting calculate

 

root@Tower:~# ps aux | grep unbalance
root     30233  0.0  0.0   9652  1800 pts/0    S+   16:32   0:00 grep unbalance
root@Tower:~# ps aux | grep find     
root     30243  0.0  0.0   9652  1792 pts/0    S+   16:32   0:00 grep find

 

So, it still seems to be crashing  :o

 

Please do the following

 

- Disable the plugin from the unRAID settings.

- From the console, type

sudo -H -u nobody bash -c "/usr/local/emhttp/plugins/unbalance/unbalance -port 6237"

- Open your browser to the unbalance url (http://<server>:6237)

- Hit calculate

 

It should crash, but what I'm interested in is what's shown in the console.

 

Can you post the whole output ?

 

Feel free to send me a pm if you prefer.

Link to comment
nacat78 Contributor

nacat78

Posted
Posted

Attempted to get the update, but am getting this:

 

plugin: installing: https://raw.githubusercontent.com/jbrodriguez/unraid/master/plugins/unbalance.plg
plugin: downloading https://raw.githubusercontent.com/jbrodriguez/unraid/master/plugins/unbalance.plg
plugin: downloading: https://raw.githubusercontent.com/jbrodriguez/unraid/master/plugins/unbalance.plg ... done
Warning: simplexml_load_file(): /tmp/plugins/unbalance.plg:1: parser error : Document is empty in /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin on line 214
Warning: simplexml_load_file(): in /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin on line 214
Warning: simplexml_load_file(): ^ in /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin on line 214
Warning: simplexml_load_file(): /tmp/plugins/unbalance.plg:1: parser error : Document is empty in /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin on line 214
Warning: simplexml_load_file(): in /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin on line 214
Warning: simplexml_load_file(): ^ in /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin on line 214
Warning: simplexml_load_file(): /tmp/plugins/unbalance.plg:1: parser error : Document is empty in /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin on line 214
Warning: simplexml_load_file(): in /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin on line 214
Warning: simplexml_load_file(): ^ in /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin on line 214
plugin: xml parse error

 

Just wanted to let you know..

Link to comment
Conson Droppa Explorer

Conson Droppa

Posted
Posted

Well I finally had room to perform some data moves. But I keep getting this error kick out. I have ran New Permissions a few times also. Can someone shed some light on what I can try or what the problem could be. It is preventing me from completely emptying a drive.

 

I: 2016/12/17 08:00:11 core.go:808: unRaid/Videos/Picture 040.mpg
I: 2016/12/17 08:00:11 core.go:808: unRaid/Videos/Picture 065.mpg
I: 2016/12/17 08:00:12 core.go:808: unRaid/Videos/Picture 068.mpg
I: 2016/12/17 08:00:14 core.go:808: unRaid/Videos/Picture 071.mpg
I: 2016/12/17 08:00:15 core.go:808: unRaid/Videos/STUDIO_STUDIOARCH_7194612318.mp4
I: 2016/12/17 08:00:15 core.go:808: unRaid/Videos/STUDIO_TOWEROFTERRORBRIDE_7193594537.mp4
I: 2016/12/17 08:00:15 core.go:808: unRaid/Videos/STUDIO_TOWEROFTERRORBRIDE_7193690113.mp4
I: 2016/12/17 08:00:16 core.go:808: unRaid/Videos/clip-2009-09-05 12;03;38.mov
I: 2016/12/17 08:00:17 core.go:808: unRaid/Videos/clip-2009-09-05 12;13;10.mov
I: 2016/12/17 08:00:27 core.go:808: 
I: 2016/12/17 08:00:27 core.go:808: sent 83,777,541,532 bytes  received 18,228 bytes  39,903,576.93 bytes/sec
I: 2016/12/17 08:00:27 core.go:808: total size is 83,757,031,044  speedup is 1.00
W: 2016/12/17 08:00:27 shell.go:105: moveProgress: waitError: exit status 23
W: 2016/12/17 08:00:27 core.go:819: Move command (rsync -avRX --partial "unRaid/Videos" "/mnt/disk5/") was interrupted: exit status 23 : Partial transfer due to error
I: 2016/12/17 08:00:27 unraid.go:391: Unraid Box Condition: &{NumDisks:6 NumProtected:0 Synced:2016-12-08 17:10:14 -0600 CST SyncErrs:0 Resync:0 ResyncPos:0 State:STARTED Size:7059818254336 Free:3208549273600 NewFree:3208549273600}
I: 2016/12/17 08:00:27 unraid.go:392: Unraid Box SourceDiskName: 
I: 2016/12/17 08:00:27 unraid.go:393: Unraid Box BytesToMove: 0
I: 2016/12/17 08:00:27 unraid.go:404: Id(1); Name(disk1); Path(/mnt/disk1); Device(sdg); Type(Data); FsType(reiserfs); Free(233.6G); NewFree(233.6G); Size(1.8T); Serial(ST2000DM001-1CH164_Z1E3KGNY); Status(DISK_OK); Bin(<nil>)
I: 2016/12/17 08:00:27 unraid.go:404: Id(2); Name(disk2); Path(/mnt/disk2); Device(sdc); Type(Data); FsType(reiserfs); Free(1.7T); NewFree(1.7T); Size(1.8T); Serial(ST2000DM001-1CH164_Z1E3KH8B); Status(DISK_OK); Bin(<nil>)
I: 2016/12/17 08:00:27 unraid.go:404: Id(3); Name(disk3); Path(/mnt/disk3); Device(sdf); Type(Data); FsType(xfs); Free(232.1G); NewFree(232.1G); Size(1.8T); Serial(ST2000DM001-1CH164_Z1E3KHTW); Status(DISK_OK); Bin(<nil>)
I: 2016/12/17 08:00:27 unraid.go:404: Id(4); Name(disk4); Path(/mnt/disk4); Device(sdb); Type(Data); FsType(xfs); Free(465.5G); NewFree(465.5G); Size(465.5G); Serial(Hitachi_HTS725050A7E630_TF0500Y9HVUEXA); Status(DISK_OK); Bin(<nil>)
I: 2016/12/17 08:00:27 unraid.go:404: Id(5); Name(disk5); Path(/mnt/disk5); Device(sde); Type(Data); FsType(xfs); Free(306.6G); NewFree(306.6G); Size(465.5G); Serial(Hitachi_HTS727550A9E364_J3360081HATKJD); Status(DISK_OK); Bin(<nil>)
I: 2016/12/17 08:00:27 unraid.go:404: Id(30); Name(cache); Path(/mnt/cache); Device(sdh); Type(Cache); FsType(xfs); Free(24.4G); NewFree(24.4G); Size(55.9G); Serial(OCZ-VERTEX3_OCZ-217TQ913Z1HZ8FL1); Status(DISK_OK); Bin(<nil>)
I: 2016/12/17 08:00:29 core.go:921: unBALANCE - MOVE operation INTERRUPTED
I: 2016/12/17 08:00:29 core.go:922: 

Started: Dec 17, 2016 07:25:27
Ended: Dec 17, 2016 08:00:27

Elapsed: 34m59.15s

Move command (rsync -avRX --partial "unRaid/Videos" "/mnt/disk5/") was interrupted: exit status 23 : Partial transfer due to error

These are the commands that were executed:

Link to comment
  • jbrodriguez changed the title to [Plugin] unbalanced

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
Go to topic listing