[Plugin] unbalanced


Recommended Posts

Well I finally had room to perform some data moves. But I keep getting this error kick out. I have ran New Permissions a few times also. Can someone shed some light on what I can try or what the problem could be. It is preventing me from completely emptying a drive.

 

I: 2016/12/17 08:00:11 core.go:808: unRaid/Videos/Picture 040.mpg
I: 2016/12/17 08:00:11 core.go:808: unRaid/Videos/Picture 065.mpg
I: 2016/12/17 08:00:12 core.go:808: unRaid/Videos/Picture 068.mpg
I: 2016/12/17 08:00:14 core.go:808: unRaid/Videos/Picture 071.mpg
I: 2016/12/17 08:00:15 core.go:808: unRaid/Videos/STUDIO_STUDIOARCH_7194612318.mp4
I: 2016/12/17 08:00:15 core.go:808: unRaid/Videos/STUDIO_TOWEROFTERRORBRIDE_7193594537.mp4
I: 2016/12/17 08:00:15 core.go:808: unRaid/Videos/STUDIO_TOWEROFTERRORBRIDE_7193690113.mp4
I: 2016/12/17 08:00:16 core.go:808: unRaid/Videos/clip-2009-09-05 12;03;38.mov
I: 2016/12/17 08:00:17 core.go:808: unRaid/Videos/clip-2009-09-05 12;13;10.mov
I: 2016/12/17 08:00:27 core.go:808: 
I: 2016/12/17 08:00:27 core.go:808: sent 83,777,541,532 bytes  received 18,228 bytes  39,903,576.93 bytes/sec
I: 2016/12/17 08:00:27 core.go:808: total size is 83,757,031,044  speedup is 1.00
W: 2016/12/17 08:00:27 shell.go:105: moveProgress: waitError: exit status 23
W: 2016/12/17 08:00:27 core.go:819: Move command (rsync -avRX --partial "unRaid/Videos" "/mnt/disk5/") was interrupted: exit status 23 : Partial transfer due to error
I: 2016/12/17 08:00:27 unraid.go:391: Unraid Box Condition: &{NumDisks:6 NumProtected:0 Synced:2016-12-08 17:10:14 -0600 CST SyncErrs:0 Resync:0 ResyncPos:0 State:STARTED Size:7059818254336 Free:3208549273600 NewFree:3208549273600}
I: 2016/12/17 08:00:27 unraid.go:392: Unraid Box SourceDiskName: 
I: 2016/12/17 08:00:27 unraid.go:393: Unraid Box BytesToMove: 0
I: 2016/12/17 08:00:27 unraid.go:404: Id(1); Name(disk1); Path(/mnt/disk1); Device(sdg); Type(Data); FsType(reiserfs); Free(233.6G); NewFree(233.6G); Size(1.8T); Serial(ST2000DM001-1CH164_Z1E3KGNY); Status(DISK_OK); Bin(<nil>)
I: 2016/12/17 08:00:27 unraid.go:404: Id(2); Name(disk2); Path(/mnt/disk2); Device(sdc); Type(Data); FsType(reiserfs); Free(1.7T); NewFree(1.7T); Size(1.8T); Serial(ST2000DM001-1CH164_Z1E3KH8B); Status(DISK_OK); Bin(<nil>)
I: 2016/12/17 08:00:27 unraid.go:404: Id(3); Name(disk3); Path(/mnt/disk3); Device(sdf); Type(Data); FsType(xfs); Free(232.1G); NewFree(232.1G); Size(1.8T); Serial(ST2000DM001-1CH164_Z1E3KHTW); Status(DISK_OK); Bin(<nil>)
I: 2016/12/17 08:00:27 unraid.go:404: Id(4); Name(disk4); Path(/mnt/disk4); Device(sdb); Type(Data); FsType(xfs); Free(465.5G); NewFree(465.5G); Size(465.5G); Serial(Hitachi_HTS725050A7E630_TF0500Y9HVUEXA); Status(DISK_OK); Bin(<nil>)
I: 2016/12/17 08:00:27 unraid.go:404: Id(5); Name(disk5); Path(/mnt/disk5); Device(sde); Type(Data); FsType(xfs); Free(306.6G); NewFree(306.6G); Size(465.5G); Serial(Hitachi_HTS727550A9E364_J3360081HATKJD); Status(DISK_OK); Bin(<nil>)
I: 2016/12/17 08:00:27 unraid.go:404: Id(30); Name(cache); Path(/mnt/cache); Device(sdh); Type(Cache); FsType(xfs); Free(24.4G); NewFree(24.4G); Size(55.9G); Serial(OCZ-VERTEX3_OCZ-217TQ913Z1HZ8FL1); Status(DISK_OK); Bin(<nil>)
I: 2016/12/17 08:00:29 core.go:921: unBALANCE - MOVE operation INTERRUPTED
I: 2016/12/17 08:00:29 core.go:922: 

Started: Dec 17, 2016 07:25:27
Ended: Dec 17, 2016 08:00:27

Elapsed: 34m59.15s

Move command (rsync -avRX --partial "unRaid/Videos" "/mnt/disk5/") was interrupted: exit status 23 : Partial transfer due to error

These are the commands that were executed:

 

Hi rmeaux,

 

Do you get any warnings when running calculate (especially on the unRaid/Videos folder) ?

 

Is unBALANCE the only app/plugin writing to disks ? Maybe you have couchpotato, mover, nzbget, etc running in the background ?

 

Other than that, maybe a double check on the health of the source disk ?

Link to comment

 

 

Hi rmeaux,

 

Do you get any warnings when running calculate (especially on the unRaid/Videos folder) ?

 

Is unBALANCE the only app/plugin writing to disks ? Maybe you have couchpotato, mover, nzbget, etc running in the background ?

 

Other than that, maybe a double check on the health of the source disk ?

 

Well the only errors it had after calculate was a few folders didn't have the user as drwxr..... but it was not the Videos folder. Ran new permissions again. No plugins or dockers running. All shares Were unmounted on clients and my main iTunes app is closed.

 

I ended up manually moving the files and folders from disk2 to disk3. I did find after I moved the files and went back to start deleting the source files i had a few files that had a "~" in the file name and it was kicking back an unable to delete error due to filenames length too long. I removed the tilde and was able to delete it.

 

I wish I would have ran unbalance again to see if it would have performed after the filenames change before deleting them.

 

Is a tilde something that would have suspended move?

 

Also, is there a way to balance the array after I get everything to XFS? I know this was asked before somewhere but I can't find it. I only get search results back on unbalance topics. Having disk color coded red and orange bothers me.

 

Thanks.

 

 

Sent from my iPhone using Tapatalk

Link to comment

 

...

I ended up manually moving the files and folders from disk2 to disk3. I did find after I moved the files and went back to start deleting the source files i had a few files that had a "~" in the file name and it was kicking back an unable to delete error due to filenames length too long. I removed the tilde and was able to delete it.

 

Is a tilde something that would have suspended move?

 

Also, is there a way to balance the array after I get everything to XFS? I know this was asked before somewhere but I can't find it. I only get search results back on unbalance topics. Having disk color coded red and orange bothers me.

...

 

I wouldn't think that the tilde has something to do, but the filename too long error maybe did.

 

As for rebalance, not sure ... probably look into consld8 (https://lime-technology.com/forum/index.php?topic=36201.0) ?

Link to comment

Hey There just installed your plugin, was having issues with permissions (not owned by nobody/users) which i fixed now i get a crash when i try to run a calculate heres the error it gets when it crashes

 

I: 2016/12/21 09:54:19 core.go:648: perms:Scanning disk(/mnt/disk1):folder(/Series/Series/Series)
I: 2016/12/21 09:54:19 core.go:658: perms:Executing find "/mnt/disk1/Series/Series/Series/." -exec stat --format "%A|%U:%G|%F|%n" {} \;
W: 2016/12/21 09:54:19 shell.go:105: perms:find/stat:: waitError: exit status 1
I: 2016/12/21 09:54:19 core.go:599: getFolders:Scanning source-disk(/mnt/disk1):folder(/Series/Series/Series)
F: 2016/12/21 09:54:19 core.go:608: getFolders:Unable to readdir: open /mnt/disk1/Series/Series/Series: permission denied

 

and heres a ls -a

drw-rw-rw-  4 nobody users  46 Dec 21 00:22 B/

Link to comment

Hey There just installed your plugin, was having issues with permissions (not owned by nobody/users) which i fixed now i get a crash when i try to run a calculate heres the error it gets when it crashes

 

I: 2016/12/21 09:54:19 core.go:648: perms:Scanning disk(/mnt/disk1):folder(/Series/Series/Series)
I: 2016/12/21 09:54:19 core.go:658: perms:Executing find "/mnt/disk1/Series/Series/Series/." -exec stat --format "%A|%U:%G|%F|%n" {} \;
W: 2016/12/21 09:54:19 shell.go:105: perms:find/stat:: waitError: exit status 1
I: 2016/12/21 09:54:19 core.go:599: getFolders:Scanning source-disk(/mnt/disk1):folder(/Series/Series/Series)
F: 2016/12/21 09:54:19 core.go:608: getFolders:Unable to readdir: open /mnt/disk1/Series/Series/Series: permission denied

 

and heres a ls -a

drw-rw-rw-  4 nobody users  46 Dec 21 00:22 B/

 

Hi kagorus,

 

I looked at the code and tbh, I shouldn't make it 'crash' on that line.

 

Having said that, it's getting a permission denied when trying to get the files for /mnt/disk1/Series/Series/Series.

 

Can you post the complete output for

ls -al /mnt/disk1/Series/Series/Series

?

 

Feel free to pm if you prefer.

Link to comment

Hey There just installed your plugin, was having issues with permissions (not owned by nobody/users) which i fixed now i get a crash when i try to run a calculate heres the error it gets when it crashes

 

I: 2016/12/21 09:54:19 core.go:648: perms:Scanning disk(/mnt/disk1):folder(/Series/Series/Series)
I: 2016/12/21 09:54:19 core.go:658: perms:Executing find "/mnt/disk1/Series/Series/Series/." -exec stat --format "%A|%U:%G|%F|%n" {} \;
W: 2016/12/21 09:54:19 shell.go:105: perms:find/stat:: waitError: exit status 1
I: 2016/12/21 09:54:19 core.go:599: getFolders:Scanning source-disk(/mnt/disk1):folder(/Series/Series/Series)
F: 2016/12/21 09:54:19 core.go:608: getFolders:Unable to readdir: open /mnt/disk1/Series/Series/Series: permission denied

 

and heres a ls -a

drw-rw-rw-  4 nobody users  46 Dec 21 00:22 B/

 

Hi kagorus,

 

I looked at the code and tbh, I shouldn't make it 'crash' on that line.

 

Having said that, it's getting a permission denied when trying to get the files for /mnt/disk1/Series/Series/Series.

 

Can you post the complete output for

ls -al /mnt/disk1/Series/Series/Series

?

 

Feel free to pm if you prefer.

ls -al /mnt/disk1/Series/Series/Series
total 0
drw-rw-rw- 13 nobody users 105 Dec 21 00:18 ./
drw-rw-rw-  9 nobody users 132 Dec 21 00:18 ../
drw-rw-rw-  4 nobody users  46 Dec 21 00:22 B/
drw-rw-rw-  4 nobody users  43 Dec 21 00:22 C/
drw-rw-rw-  3 nobody users  47 Dec 21 00:22 E/
drw-rw-rw-  3 nobody users  27 Dec 21 00:22 F/
drw-rw-rw-  3 nobody users  23 Dec 21 00:22 H/
drw-rw-rw-  3 nobody users  35 Dec 21 00:22 I/
drw-rw-rw-  3 nobody users  29 Dec 21 00:22 J/
drw-rw-rw-  3 nobody users  28 Dec 21 00:22 L/
drw-rw-rw-  4 nobody users  44 Dec 21 00:22 R/
drw-rw-rw-  6 nobody users 115 Dec 21 00:22 S/
drw-rw-rw-  4 nobody users  38 Dec 21 00:22 T/

Link to comment

kagorus, ok

 

Please run

 

find "/mnt/disk1/Series/Series/Series/." -exec stat --format "%A|%U:%G|%F|%n" {} \;

 

Don't send me the output, it could be quite big.

 

But can you check if you get any permission denied errors ?

 

I'm guessing some files below those B, C, etc folders have some permission issues.

Link to comment
  • 2 weeks later...

I just installed this plugin using Community Applications.

 

I don't seem to be able to change the disk I'm trying to empty.  It says "you choose the source disk in the main page" but i can only get to the settings and log tabs.  clicking the Home tab does nothing.  I've not been able to find "the main page" to specify which disk.

 

Link to comment

I just installed this plugin using Community Applications.

 

I don't seem to be able to change the disk I'm trying to empty.  It says "you choose the source disk in the main page" but i can only get to the settings and log tabs.  clicking the Home tab does nothing.  I've not been able to find "the main page" to specify which disk.

I could be wrong, I'm talking out of my rear at the moment, but can you select something to move in the "Which folders do you want to move?" section? I don't think the Home tab populates until you do. Bad UI if so.
Link to comment

yep, that was it.

 

Bad design.  I don't even think the folder I picked is on the disk I want to work with.  I would think picking the disk to work with should be step 1 in the process.

 

With that said, it looks like it will work way better than doing this by hand, so I still much appreciate this being made available to me; thank you.

Link to comment

...

I could be wrong, I'm talking out of my rear at the moment, but can you select something to move in the "Which folders do you want to move?" section? I don't think the Home tab populates until you do. Bad UI if so.

 

Well, the plugin was born out of my own requirements, which it has suited just fine for me.

 

It has evolved during this time and perhaps the common usage scenarios are different, but I wouldn't go as far as to give it a "Bad UI" rating.

 

If you don't like it, I do welcome suggestions and have implemented almost all (I think) of what I've received in the forums.

Link to comment

yep, that was it.

 

Bad design.  I don't even think the folder I picked is on the disk I want to work with.  I would think picking the disk to work with should be step 1 in the process.

 

With that said, it looks like it will work way better than doing this by hand, so I still much appreciate this being made available to me; thank you.

 

Hi JustinChase, if you feel it has a "Bad design", you're welcome to make suggestions on how it should work, I've implemented most of the ideas thrown around in the forum previously.

 

Thanks for your comment.

Link to comment

I just installed the plugin but can't figure out how to run it.  I went to the Settings tab, under User Utilities, I clicked on UnBalance it.  I then get a page with the options

Enable UnBalance Server

Port

Run as User

 

Am I missing something or doing something wrong?

 

In the settings page, there should be a green link with the text Open Webui ? It isn't there for you ?

Link to comment

...

I could be wrong, I'm talking out of my rear at the moment, but can you select something to move in the "Which folders do you want to move?" section? I don't think the Home tab populates until you do. Bad UI if so.

 

Well, the plugin was born out of my own requirements, which it has suited just fine for me.

 

It has evolved during this time and perhaps the common usage scenarios are different, but I wouldn't go as far as to give it a "Bad UI" rating.

 

If you don't like it, I do welcome suggestions and have implemented almost all (I think) of what I've received in the forums.

On a fresh install, the initial instinct is to click on the "Home" tab. There is nothing on the settings page telling you "hey dummy, add a folder to move" so the home tab link will do something. Perhaps you could move the "which folders do you want to move" section to the home tab, so at least you see some action when you initially click on home.

 

Also, I think the common usage is now to completely clean off a specific disk, for whatever reason, with all the folders included. Maybe populating the home tab with a share level deep drive list with check boxes for drives or individual shares as source, and valid destinations with checkboxes as well. For example, I could check just the movies on disk1 and have the destination be disk2,3,4 or I could check the entire disk1 and send it across all the rest of the drives.

 

It seems a little off to want to empty a specific disk, but step 1 is to pick which folders I want to move. Since unraid's share system by default doesn't force specific disks for a share, it's conceivable that a user may have no clue which shares are on a drive when he opens unBALANCE, he just wants the drive empty, and expects to be shown a list of drives that can be emptied and a list of possible destinations.

 

Thank you for your work on this, it's truly the go to plugin for novice friendly disk level data management, it's a function that should IMHO be Limetech's responsibility, but hey, that's what the community is for, to fill the holes in basic functionality.

Link to comment

I just installed the plugin but can't figure out how to run it.  I went to the Settings tab, under User Utilities, I clicked on UnBalance it.  I then get a page with the options

Enable UnBalance Server

Port

Run as User

 

Am I missing something or doing something wrong?

 

In the settings page, there should be a green link with the text Open Webui ? It isn't there for you ?

 

Nope.  Not if you mean the Settings tab in the web gui.  I do have Unbalance under the section User Utilities on the Settings tab, but nothing else...

Link to comment

This is not good.  After installing the plugin and trying to stop the array, I can't unmount the disks.  I keeps trying and trying...crap I think I'm going to have to do a power off...

 

Not sure what you mean by this.

 

After I installed the plugin I tried to stop and start the Array.  When I hit Stop, the disks kept trying to unmount but couldn't.  I kept seeing a message "trying to unmount....".  It kept up for at least 10min so I had to eventually go to the command line and do a power down.

Link to comment

This is not good.  After installing the plugin and trying to stop the array, I can't unmount the disks.  I keeps trying and trying...crap I think I'm going to have to do a power off...

 

Not sure what you mean by this.

 

After I installed the plugin I tried to stop and start the Array.  When I hit Stop, the disks kept trying to unmount but couldn't.  I kept seeing a message "trying to unmount....".  It kept up for at least 10min so I had to eventually go to the command line and do a power down.

No clear evidence this plugin was responsible. Open Files plugin can help you figure out what is keeping files open and preventing the drives from unmounting.
Link to comment

...

I could be wrong, I'm talking out of my rear at the moment, but can you select something to move in the "Which folders do you want to move?" section? I don't think the Home tab populates until you do. Bad UI if so.

 

Well, the plugin was born out of my own requirements, which it has suited just fine for me.

 

It has evolved during this time and perhaps the common usage scenarios are different, but I wouldn't go as far as to give it a "Bad UI" rating.

 

If you don't like it, I do welcome suggestions and have implemented almost all (I think) of what I've received in the forums.

On a fresh install, the initial instinct is to click on the "Home" tab. There is nothing on the settings page telling you "hey dummy, add a folder to move" so the home tab link will do something. Perhaps you could move the "which folders do you want to move" section to the home tab, so at least you see some action when you initially click on home.

 

Also, I think the common usage is now to completely clean off a specific disk, for whatever reason, with all the folders included. Maybe populating the home tab with a share level deep drive list with check boxes for drives or individual shares as source, and valid destinations with checkboxes as well. For example, I could check just the movies on disk1 and have the destination be disk2,3,4 or I could check the entire disk1 and send it across all the rest of the drives.

 

It seems a little off to want to empty a specific disk, but step 1 is to pick which folders I want to move. Since unraid's share system by default doesn't force specific disks for a share, it's conceivable that a user may have no clue which shares are on a drive when he opens unBALANCE, he just wants the drive empty, and expects to be shown a list of drives that can be emptied and a list of possible destinations.

 

Thank you for your work on this, it's truly the go to plugin for novice friendly disk level data management, it's a function that should IMHO be Limetech's responsibility, but hey, that's what the community is for, to fill the holes in basic functionality.

 

Thanks for the comments jonathanm.

 

That's something I've thought about ... moving the "file explorer" currently on the settings tab to the home tab, under the currently selected source disk, populated with whatever folders are on the disk.

 

This is what I'll be working on next, as time allows.

Link to comment
  • jbrodriguez changed the title to [Plugin] unbalanced

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.