Notice the right side.  In the past, I had level split set to auto.  Now I have it set to the movie folder level, and the TV season level.  That way it fires only a single drive for a movie, or for a TV season.  Unbalance GATHER function works very well to gather the files to the same drive, most of the time only taking 1 or 2 seconds since the majority of the problem folders are .txt files or .jpg files, etc.  Is there a mechasism, or even a similar utility that can gather a large number of these folders at a time rather than doing them individually? As is, I have to click gather, click the movie folder, click the top listed/largest item, click next, click go.  I don't want to multiple that by 2000 movies. 

Can you click on the folder containing those folders (one level up) or that doesn't work for you ?

Hi, and thanks a lot for this great tool. 

 

I am using it to move data off a disk that I want to remove from the array. I made a mistake when moving data off the disk I want emptied (I didn't specify that I want the remaining disks to have roughly 5 % remaining space, so they all filled up with the default ~500mb remaining space). Now I am manually moving some content from each of the full disks into disks with more space (I have 2 disks with 3 TB of emtpy space).

 

Is there any smarter way to do this, other than to run unBalance for each disk I want to have ~5 % remaining space on? I.e. queue up a job for each disk to leave over night, or a default Unraid setting/function that will take care of this automatically? Or am I stuck starting each unBalance job manually for each of the full disks I want to clear some space on? 

Good day all,
This plugin was brought to my attention just today as I am consolidating my existing /user/shares/ to one user share
I want to make sure I am going about this correctly so I attempted to move one media folder from /user/share -> /user/share

I received the following:
There are some permission issues with the folders/files you want to transfer

19 file(s)/folder(s) with an owner other than 'nobody'

0 file(s)/folder(s) with a group other than 'users'

0 folder(s) with a permission other than 'drwxrwxrwx'

0 files(s) with a permission other than '-rw-rw-rw-' or '-r--r--r--'

You can find more details about which files have issues in the log file (/boot/logs/unbalance.log)

At this point, you can transfer the folders/files if you want, but be advised that it can cause errors in the operation

You are STRONGLY suggested to install the Fix Common Problems plugin, then run the Docker Safe New Permissions command

 

So I am going to run the "Docker Safe New Permissions command"

Once completed, will I have the ability to move
/user/<share name>

to
/user/<new share name>

Also, does unBALANCED have the ability to see 'unknown devices' ? 

Thanks, 

Edited April 13, 20206 yr by bombz

18 hours ago, Mith said:

Is there any smarter way to do this, other than to run unBalance for each disk I want to have ~5 % remaining space on?

I guess you could run manual rsyncs or do it via mc (midnight commander).

16 hours ago, bombz said:

So I am going to run the "Docker Safe New Permissions command"

Once completed, will I have the ability to move
/user/<share name>

to
/user/<new share name>

Yes, docker safe new permissions tries to make sure you have permissions to move content around.

 

Do note that unbalance works at a disk level, not user share level, so you will be moving data from /mnt/diskX to /mnt/diskY (for example)

16 hours ago, jbrodriguez said:

Yes, docker safe new permissions tries to make sure you have permissions to move content around.

 

Do note that unbalance works at a disk level, not user share level, so you will be moving data from /mnt/diskX to /mnt/diskY (for example)

Thank you for your response.
Yes I ended up doing disk level copies. I was looking at this the wrong way, and after moving data where it needed to go, I THEN reorganized my shares.
unBALANCED is an AWESOME plugin! 

Thank you again to this wonderful community. You all kick butt!
My gratitude to you all!

This plugin is operating extremely slowly. Almost to the point of being unusable. Please see attached screenshot.

4 hours ago, HomerTheGeek said:

This plugin is operating extremely slowly. Almost to the point of being unusable.

Looks about right for typical hardware. This plugin is just a graphical front end for the rsync command, and when you are moving data from one parity protected array disk to another one, the parity disk is involved in the write to both the source and destination drives, causing a lot of head movement and seek delays.

 

Set it and let it run.

 

If you feel that your specific hardware should be operating faster, post the diagnostic zip file collected during the operation.

Thanks for you work on this plugin.

 

I'm unable to uncheck dry run on either scatter or gather. I've tried chrome, edge and firefox. The last 2 with no plugins.

 

Any logs I can provide to help find a solution to this?

 

Thank you.

Hi, the dry-run checkbox will be enabled after you finish the PLAN step.

 

I can't remember why I made this decision back then, seeing how it's a frequently asked question now

Hi guys, I set the unbalance to run last night to move files across in my quest to start encrypting my drives and moving them into a new supermicro 3u home.

 

As ever something didnt quite go to plan the first try, wondering how i can tell what happened?

 

I set it up like this:https://i.imgur.com/XxILYj3.png

Planned and dry run fine, left it and it stopped at 27% in the logs (I can't go back that far in the webUI, but im sure theyre saved somewhere?) but didn't output an error or anything, just seemed to stop.

Is there some way i can check what the problem was?  I had my docker disabled so that didnt screw it up, i dont have a cache drive setup so no mover from that either.

Because it didnt finish its operation is there anything untoward that can happen?  I presume because the array is mounted and parity is live the movements are all safe evern though not completed?

Cheers

 

Hi, you can check the log (/boot/logs/unbalance.log)

 

If you haven't moved any files around yet, you could retry the operation (replay button on the history tab), but check the logs first to get an idea of what's the issue.

 

Also note that if your source disk is not "healthy", rsync will slow down to a crawl and may even completely stop (I've been direct witness to this

4 hours ago, jbrodriguez said:

Hi, you can check the log (/boot/logs/unbalance.log)

 

If you haven't moved any files around yet, you could retry the operation (replay button on the history tab), but check the logs first to get an idea of what's the issue.

 

Also note that if your source disk is not "healthy", rsync will slow down to a crawl and may even completely stop (I've been direct witness to this

Quote

I: 2020/04/18 04:49:21 core.go:1028: Command Finished
I: 2020/04/18 04:49:21 core.go:1041: Current progress: 26.64% done ~ 5h28m58s left (50.19 MB/s)
I: 2020/04/18 04:49:21 core.go:972: removing:(rm -rf "/mnt/disk2/TV/Dexter")
I: 2020/04/18 04:49:23 core.go:710: Command Started: (src: /mnt/disk2) rsync -avPR -X "TV/Line of Duty" "/mnt/disk7/" 
I: 2020/04/18 04:49:28 core.go:835: monitor:transfer:(/mnt/disk2/TV/Line of Duty/Season 3/Line of Duty - S03E01 - Monsters Bluray-1080p.mkv)
I: 2020/04/18 04:58:28 core.go:767: command:retcode(0):exitcode(0)
I: 2020/04/18 04:58:28 core.go:1028: Command Finished
I: 2020/04/18 04:58:28 core.go:1041: Current progress: 28.59% done ~ 5h21m10s left (50.04 MB/s)
I: 2020/04/18 04:58:28 core.go:972: removing:(rm -rf "/mnt/disk2/TV/Line of Duty")
I: 2020/04/18 04:58:29 core.go:710: Command Started: (src: /mnt/disk2) rsync -avPR -X "TV/Californication" "/mnt/disk7/" 
I: 2020/04/18 04:58:35 core.go:835: monitor:transfer:(/mnt/disk2/TV/Californication/Season 1/Californication - S01E02 - Hell-A Woman HDTV-720p.mkv)
W: 2020/04/18 10:01:31 server.go:414: closing socket: unable to ReadMessage: (websocket: close 1001 (going away))
W: 2020/04/18 10:01:39 server.go:414: closing socket: unable to ReadMessage: (websocket: close 1001 (going away))
I: 2020/04/18 10:01:42 app.go:77: Received signal: (terminated) ... shutting down the app now ...
I: 2020/04/18 10:01:42 core.go:170: stopped service Core ...
I: 2020/04/18 10:01:42 planner.go:64: stopped service Planner ...
I: 2020/04/18 10:01:42 array.go:64: stopped service Array ...
I: 2020/04/18 10:01:42 server.go:170: stopped service Server ...

 

So 04:58 was when it just stopped responding, and 10am is when i woke up to check its progress.

Logs from systemlog for that time:
 

Apr 18 02:04:00 alexserver root: Fix Common Problems Version 2020.03.18
Apr 18 02:04:09 alexserver root: Fix Common Problems: Warning: ca.docker.autostart.plg Not Compatible with unRaid version 6.9.0-beta1
Apr 18 02:04:15 alexserver root: Fix Common Problems: Warning: Syslog mirrored to flash
Apr 18 02:35:42 alexserver kernel: mdcmd (69): spindown 1
Apr 18 02:35:43 alexserver kernel: mdcmd (70): spindown 2
Apr 18 02:35:43 alexserver kernel: mdcmd (71): spindown 3
Apr 18 02:35:44 alexserver kernel: mdcmd (72): spindown 4
Apr 18 02:35:47 alexserver kernel: mdcmd (73): spindown 6
Apr 18 02:38:01 alexserver kernel: mdcmd (74): spindown 5
Apr 18 02:39:31 alexserver emhttpd: cmd: /usr/local/emhttp/plugins/dynamix/scripts/newperms /mnt/disk1|/mnt/disk2|/mnt/disk3|/mnt/disk4|/mnt/disk5|/mnt/disk6|/mnt/disk7|/mnt/user/Alex|/mnt/user/appdata|/mnt/user/backups|/mnt/user/domains|/mnt/user/Downloads|/mnt/user/isos|/mnt/user/Louise|/mnt/user/Motioneye|/mnt/user/Movies|/mnt/user/Music|/mnt/user/NextCloud|/mnt/user/Public Share|/mnt/user/Software|/mnt/user/system|/mnt/user/TV|/mnt/user/urbackup_tmp_files
Apr 18 03:17:31 alexserver kernel: mdcmd (75): spindown 3
Apr 18 03:25:57 alexserver kernel: mdcmd (76): spindown 6
Apr 18 03:40:16 alexserver crond[1793]: exit status 3 from user root /usr/local/sbin/mover &> /dev/null
Apr 18 03:55:33 alexserver kernel: mdcmd (77): spindown 5
Apr 18 04:02:35 alexserver kernel: mdcmd (78): spindown 4
Apr 18 04:10:53 alexserver kernel: mdcmd (79): spindown 1
Apr 18 05:34:46 alexserver kernel: mdcmd (80): spindown 5
Apr 18 05:34:48 alexserver kernel: mdcmd (81): spindown 1
Apr 18 05:37:28 alexserver kernel: mdcmd (82): spindown 2
Apr 18 06:08:52 alexserver kernel: mdcmd (83): spindown 0
Apr 18 06:08:52 alexserver kernel: mdcmd (84): spindown 7
Apr 18 10:00:40 alexserver webGUI: Successful login user root from 192.168.1.118
Apr 18 10:01:42 alexserver ool www[25663]: /usr/local/emhttp/plugins/unbalance/scripts/stop
Apr 18 10:01:46 alexserver ool www[25620]: /usr/local/emhttp/plugins/unbalance/scripts/start
Apr 18 10:01:46 alexserver sudo:     root : TTY=unknown ; PWD=/usr/local/emhttp ; USER=root ; COMMAND=/usr/bin/bash -c /usr/local/emhttp/plugins/unbalance/unbalance -port 6237
Apr 18 10:43:51 alexserver webGUI: Successful login user root from 192.168.1.118

I can give it another go, but i fear 2-3 hours in i might get the same result.

Looks like data is ok I think?

I don't think my source disks are a problem, Disk 4 needs to go probably, but this was only involving Disk 2 and Disk 7.


 

unbalance.log

For context:

This is the overall plan, which is what led me to unbalance, and something didn't quite go right straight out the blocks!

//Edit/ In the spirit of I don't quite know what it was i've just run it again, and i'll report back

 

Edited April 18, 20206 yr by alexdodd

Hi, just looking at this as my shares are a bit of a mess!

 

in my situation for example I have tv shows spread over disks 1 - 20, I have since specified the share to use only disks 1 -4 can unbalance move everything from disks 5 - 20 into the share required disks 1 - 4?

 

Many Thanks 🙂

Not exactly.

 

The GATHER operation works by moving folders/content (which in your case would be the tv shows) from any number of disks into a target disks.

 

So you can't select disks as source, it will find which disks contain the given folder/tv show and transfer it to a selected target disk.

ok, thank you for replying 🙂

I think everything has been copied, but unraid web UI is confusing me, there looks to be remains.

Can you tell me from the attached log the drive is empty?

unbalance.log

I'm trying to make sure Disk 1, Disk 3 & Disk 5 are empty: 

But it looks like there is still something on all three, although I can't see anything in the shares folders, heres disk 3 for eg

Seen here:

 

Edited April 21, 20206 yr by alexdodd

If you chose to COPY, it won't remove the content on the source disk(s).

 

If you browse, let's say disk3, from Unraid's main ui, is there some data ?

Pretty sure I set to move things not copy, everything else moved, maybe my mistake.

These folders are there but empty:

ie all like so:

Edited April 21, 20206 yr by alexdodd

I'm probably just going to just bite the bullet and go for this I think its missrepresented somehow, even though thats a "backups" folder there isnt actually anything really used in there just yet that I would miss I don't think.  Its appdata backup and some other things like that. isos i could lose, seems ok to gamble.

I am having some problems with unBALLANCE. I am trying to gather some series from multiple disks to a single disk. I first choose "Gather", then select the top level folder i want to move under "Select folder", and then choose next. On the "Choose target folder"  nothing happens. No target disks appears (even though 3 disks showed up the first time i tried it, but I still couldn't continue), and the logo-thing on the top-rights is "pulsing". Both the previous and next buttons are greyed out, and I cannot do anything.

Also, I get a green banner under the top menu which says "unBALANCE needs exclusive access to disks, so disable mover and/or any dockers that write to disks, before running it. Also note that transfer speed may be affected by disk health. Check this plugin.". I have removed the DiskSpeed docker, and stopped all other dockers. I have even tried to disable all dockers from the top level settings in unraid. 

Any thoughts?

Edited April 24, 20206 yr by TrondHjertager

I have made a big mistake... i moved files in dry mode and than format this disk...

Can i rebuild my Disk with parity ?

If yes, what is the best way?

On 4/25/2020 at 12:36 PM, Abigel said:

format this disk...

Can i rebuild my Disk with parity ?

No. Parity is realtime, when you format the disk, that is immediately applied to parity.

 

If the data on the disk is irreplaceable from backup or reripping, then data recovery software may be able to help, but it's not simple to do.

On 4/24/2020 at 7:20 AM, TrondHjertager said:

I am having some problems with unBALLANCE.

Hi, this looks like a bug.

 

Can you check the Chrome developer's console for any issues ?