mikesp18 Posted April 13, 2020 Share Posted April 13, 2020 Notice the right side. In the past, I had level split set to auto. Now I have it set to the movie folder level, and the TV season level. That way it fires only a single drive for a movie, or for a TV season. Unbalance GATHER function works very well to gather the files to the same drive, most of the time only taking 1 or 2 seconds since the majority of the problem folders are .txt files or .jpg files, etc. Is there a mechasism, or even a similar utility that can gather a large number of these folders at a time rather than doing them individually? As is, I have to click gather, click the movie folder, click the top listed/largest item, click next, click go. I don't want to multiple that by 2000 movies. Quote Link to comment
jbrodriguez Posted April 13, 2020 Author Share Posted April 13, 2020 Can you click on the folder containing those folders (one level up) or that doesn't work for you ? Quote Link to comment
Mith Posted April 13, 2020 Share Posted April 13, 2020 Hi, and thanks a lot for this great tool. I am using it to move data off a disk that I want to remove from the array. I made a mistake when moving data off the disk I want emptied (I didn't specify that I want the remaining disks to have roughly 5 % remaining space, so they all filled up with the default ~500mb remaining space). Now I am manually moving some content from each of the full disks into disks with more space (I have 2 disks with 3 TB of emtpy space). Is there any smarter way to do this, other than to run unBalance for each disk I want to have ~5 % remaining space on? I.e. queue up a job for each disk to leave over night, or a default Unraid setting/function that will take care of this automatically? Or am I stuck starting each unBalance job manually for each of the full disks I want to clear some space on? Quote Link to comment
bombz Posted April 13, 2020 Share Posted April 13, 2020 (edited) Good day all, This plugin was brought to my attention just today as I am consolidating my existing /user/shares/ to one user share I want to make sure I am going about this correctly so I attempted to move one media folder from /user/share -> /user/share I received the following: There are some permission issues with the folders/files you want to transfer 19 file(s)/folder(s) with an owner other than 'nobody' 0 file(s)/folder(s) with a group other than 'users' 0 folder(s) with a permission other than 'drwxrwxrwx' 0 files(s) with a permission other than '-rw-rw-rw-' or '-r--r--r--' You can find more details about which files have issues in the log file (/boot/logs/unbalance.log) At this point, you can transfer the folders/files if you want, but be advised that it can cause errors in the operation You are STRONGLY suggested to install the Fix Common Problems plugin, then run the Docker Safe New Permissions command So I am going to run the "Docker Safe New Permissions command" Once completed, will I have the ability to move /user/<share name> to /user/<new share name> Also, does unBALANCED have the ability to see 'unknown devices' ? Thanks, Edited April 13, 2020 by bombz Quote Link to comment
jbrodriguez Posted April 14, 2020 Author Share Posted April 14, 2020 18 hours ago, Mith said: Is there any smarter way to do this, other than to run unBalance for each disk I want to have ~5 % remaining space on? I guess you could run manual rsyncs or do it via mc (midnight commander). Quote Link to comment
jbrodriguez Posted April 14, 2020 Author Share Posted April 14, 2020 16 hours ago, bombz said: So I am going to run the "Docker Safe New Permissions command" Once completed, will I have the ability to move /user/<share name> to /user/<new share name> Yes, docker safe new permissions tries to make sure you have permissions to move content around. Do note that unbalance works at a disk level, not user share level, so you will be moving data from /mnt/diskX to /mnt/diskY (for example) Quote Link to comment
bombz Posted April 15, 2020 Share Posted April 15, 2020 16 hours ago, jbrodriguez said: Yes, docker safe new permissions tries to make sure you have permissions to move content around. Do note that unbalance works at a disk level, not user share level, so you will be moving data from /mnt/diskX to /mnt/diskY (for example) Thank you for your response. Yes I ended up doing disk level copies. I was looking at this the wrong way, and after moving data where it needed to go, I THEN reorganized my shares. unBALANCED is an AWESOME plugin! Thank you again to this wonderful community. You all kick butt! My gratitude to you all! Quote Link to comment
HomerTheGeek Posted April 15, 2020 Share Posted April 15, 2020 This plugin is operating extremely slowly. Almost to the point of being unusable. Please see attached screenshot. Quote Link to comment
JonathanM Posted April 15, 2020 Share Posted April 15, 2020 4 hours ago, HomerTheGeek said: This plugin is operating extremely slowly. Almost to the point of being unusable. Looks about right for typical hardware. This plugin is just a graphical front end for the rsync command, and when you are moving data from one parity protected array disk to another one, the parity disk is involved in the write to both the source and destination drives, causing a lot of head movement and seek delays. Set it and let it run. If you feel that your specific hardware should be operating faster, post the diagnostic zip file collected during the operation. Quote Link to comment
noVem Posted April 17, 2020 Share Posted April 17, 2020 Thanks for you work on this plugin. I'm unable to uncheck dry run on either scatter or gather. I've tried chrome, edge and firefox. The last 2 with no plugins. Any logs I can provide to help find a solution to this? Thank you. Quote Link to comment
jbrodriguez Posted April 17, 2020 Author Share Posted April 17, 2020 Hi, the dry-run checkbox will be enabled after you finish the PLAN step. I can't remember why I made this decision back then, seeing how it's a frequently asked question now Quote Link to comment
alexdodd Posted April 18, 2020 Share Posted April 18, 2020 Hi guys, I set the unbalance to run last night to move files across in my quest to start encrypting my drives and moving them into a new supermicro 3u home. As ever something didnt quite go to plan the first try, wondering how i can tell what happened? I set it up like this:https://i.imgur.com/XxILYj3.png Planned and dry run fine, left it and it stopped at 27% in the logs (I can't go back that far in the webUI, but im sure theyre saved somewhere?) but didn't output an error or anything, just seemed to stop. Is there some way i can check what the problem was? I had my docker disabled so that didnt screw it up, i dont have a cache drive setup so no mover from that either. Because it didnt finish its operation is there anything untoward that can happen? I presume because the array is mounted and parity is live the movements are all safe evern though not completed? Cheers Quote Link to comment
jbrodriguez Posted April 18, 2020 Author Share Posted April 18, 2020 Hi, you can check the log (/boot/logs/unbalance.log) If you haven't moved any files around yet, you could retry the operation (replay button on the history tab), but check the logs first to get an idea of what's the issue. Also note that if your source disk is not "healthy", rsync will slow down to a crawl and may even completely stop (I've been direct witness to this Quote Link to comment
alexdodd Posted April 18, 2020 Share Posted April 18, 2020 (edited) 4 hours ago, jbrodriguez said: Hi, you can check the log (/boot/logs/unbalance.log) If you haven't moved any files around yet, you could retry the operation (replay button on the history tab), but check the logs first to get an idea of what's the issue. Also note that if your source disk is not "healthy", rsync will slow down to a crawl and may even completely stop (I've been direct witness to this Quote I: 2020/04/18 04:49:21 core.go:1028: Command Finished I: 2020/04/18 04:49:21 core.go:1041: Current progress: 26.64% done ~ 5h28m58s left (50.19 MB/s) I: 2020/04/18 04:49:21 core.go:972: removing:(rm -rf "/mnt/disk2/TV/Dexter") I: 2020/04/18 04:49:23 core.go:710: Command Started: (src: /mnt/disk2) rsync -avPR -X "TV/Line of Duty" "/mnt/disk7/" I: 2020/04/18 04:49:28 core.go:835: monitor:transfer:(/mnt/disk2/TV/Line of Duty/Season 3/Line of Duty - S03E01 - Monsters Bluray-1080p.mkv) I: 2020/04/18 04:58:28 core.go:767: command:retcode(0):exitcode(0) I: 2020/04/18 04:58:28 core.go:1028: Command Finished I: 2020/04/18 04:58:28 core.go:1041: Current progress: 28.59% done ~ 5h21m10s left (50.04 MB/s) I: 2020/04/18 04:58:28 core.go:972: removing:(rm -rf "/mnt/disk2/TV/Line of Duty") I: 2020/04/18 04:58:29 core.go:710: Command Started: (src: /mnt/disk2) rsync -avPR -X "TV/Californication" "/mnt/disk7/" I: 2020/04/18 04:58:35 core.go:835: monitor:transfer:(/mnt/disk2/TV/Californication/Season 1/Californication - S01E02 - Hell-A Woman HDTV-720p.mkv) W: 2020/04/18 10:01:31 server.go:414: closing socket: unable to ReadMessage: (websocket: close 1001 (going away)) W: 2020/04/18 10:01:39 server.go:414: closing socket: unable to ReadMessage: (websocket: close 1001 (going away)) I: 2020/04/18 10:01:42 app.go:77: Received signal: (terminated) ... shutting down the app now ... I: 2020/04/18 10:01:42 core.go:170: stopped service Core ... I: 2020/04/18 10:01:42 planner.go:64: stopped service Planner ... I: 2020/04/18 10:01:42 array.go:64: stopped service Array ... I: 2020/04/18 10:01:42 server.go:170: stopped service Server ... So 04:58 was when it just stopped responding, and 10am is when i woke up to check its progress. Logs from systemlog for that time: Apr 18 02:04:00 alexserver root: Fix Common Problems Version 2020.03.18 Apr 18 02:04:09 alexserver root: Fix Common Problems: Warning: ca.docker.autostart.plg Not Compatible with unRaid version 6.9.0-beta1 Apr 18 02:04:15 alexserver root: Fix Common Problems: Warning: Syslog mirrored to flash Apr 18 02:35:42 alexserver kernel: mdcmd (69): spindown 1 Apr 18 02:35:43 alexserver kernel: mdcmd (70): spindown 2 Apr 18 02:35:43 alexserver kernel: mdcmd (71): spindown 3 Apr 18 02:35:44 alexserver kernel: mdcmd (72): spindown 4 Apr 18 02:35:47 alexserver kernel: mdcmd (73): spindown 6 Apr 18 02:38:01 alexserver kernel: mdcmd (74): spindown 5 Apr 18 02:39:31 alexserver emhttpd: cmd: /usr/local/emhttp/plugins/dynamix/scripts/newperms /mnt/disk1|/mnt/disk2|/mnt/disk3|/mnt/disk4|/mnt/disk5|/mnt/disk6|/mnt/disk7|/mnt/user/Alex|/mnt/user/appdata|/mnt/user/backups|/mnt/user/domains|/mnt/user/Downloads|/mnt/user/isos|/mnt/user/Louise|/mnt/user/Motioneye|/mnt/user/Movies|/mnt/user/Music|/mnt/user/NextCloud|/mnt/user/Public Share|/mnt/user/Software|/mnt/user/system|/mnt/user/TV|/mnt/user/urbackup_tmp_files Apr 18 03:17:31 alexserver kernel: mdcmd (75): spindown 3 Apr 18 03:25:57 alexserver kernel: mdcmd (76): spindown 6 Apr 18 03:40:16 alexserver crond[1793]: exit status 3 from user root /usr/local/sbin/mover &> /dev/null Apr 18 03:55:33 alexserver kernel: mdcmd (77): spindown 5 Apr 18 04:02:35 alexserver kernel: mdcmd (78): spindown 4 Apr 18 04:10:53 alexserver kernel: mdcmd (79): spindown 1 Apr 18 05:34:46 alexserver kernel: mdcmd (80): spindown 5 Apr 18 05:34:48 alexserver kernel: mdcmd (81): spindown 1 Apr 18 05:37:28 alexserver kernel: mdcmd (82): spindown 2 Apr 18 06:08:52 alexserver kernel: mdcmd (83): spindown 0 Apr 18 06:08:52 alexserver kernel: mdcmd (84): spindown 7 Apr 18 10:00:40 alexserver webGUI: Successful login user root from 192.168.1.118 Apr 18 10:01:42 alexserver ool www[25663]: /usr/local/emhttp/plugins/unbalance/scripts/stop Apr 18 10:01:46 alexserver ool www[25620]: /usr/local/emhttp/plugins/unbalance/scripts/start Apr 18 10:01:46 alexserver sudo: root : TTY=unknown ; PWD=/usr/local/emhttp ; USER=root ; COMMAND=/usr/bin/bash -c /usr/local/emhttp/plugins/unbalance/unbalance -port 6237 Apr 18 10:43:51 alexserver webGUI: Successful login user root from 192.168.1.118 I can give it another go, but i fear 2-3 hours in i might get the same result. Looks like data is ok I think? I don't think my source disks are a problem, Disk 4 needs to go probably, but this was only involving Disk 2 and Disk 7. unbalance.log For context: This is the overall plan, which is what led me to unbalance, and something didn't quite go right straight out the blocks! //Edit/ In the spirit of I don't quite know what it was i've just run it again, and i'll report back Edited April 18, 2020 by alexdodd Quote Link to comment
mbc0 Posted April 20, 2020 Share Posted April 20, 2020 Hi, just looking at this as my shares are a bit of a mess! in my situation for example I have tv shows spread over disks 1 - 20, I have since specified the share to use only disks 1 -4 can unbalance move everything from disks 5 - 20 into the share required disks 1 - 4? Many Thanks 🙂 Quote Link to comment
jbrodriguez Posted April 20, 2020 Author Share Posted April 20, 2020 Not exactly. The GATHER operation works by moving folders/content (which in your case would be the tv shows) from any number of disks into a target disks. So you can't select disks as source, it will find which disks contain the given folder/tv show and transfer it to a selected target disk. 1 Quote Link to comment
mbc0 Posted April 20, 2020 Share Posted April 20, 2020 ok, thank you for replying 🙂 Quote Link to comment
alexdodd Posted April 21, 2020 Share Posted April 21, 2020 (edited) I think everything has been copied, but unraid web UI is confusing me, there looks to be remains. Can you tell me from the attached log the drive is empty? unbalance.log I'm trying to make sure Disk 1, Disk 3 & Disk 5 are empty: But it looks like there is still something on all three, although I can't see anything in the shares folders, heres disk 3 for eg Seen here: Edited April 21, 2020 by alexdodd Quote Link to comment
jbrodriguez Posted April 21, 2020 Author Share Posted April 21, 2020 If you chose to COPY, it won't remove the content on the source disk(s). If you browse, let's say disk3, from Unraid's main ui, is there some data ? Quote Link to comment
alexdodd Posted April 21, 2020 Share Posted April 21, 2020 (edited) Pretty sure I set to move things not copy, everything else moved, maybe my mistake. These folders are there but empty: ie all like so: Edited April 21, 2020 by alexdodd Quote Link to comment
alexdodd Posted April 22, 2020 Share Posted April 22, 2020 I'm probably just going to just bite the bullet and go for this I think its missrepresented somehow, even though thats a "backups" folder there isnt actually anything really used in there just yet that I would miss I don't think. Its appdata backup and some other things like that. isos i could lose, seems ok to gamble. Quote Link to comment
TrondHjertager Posted April 24, 2020 Share Posted April 24, 2020 (edited) I am having some problems with unBALLANCE. I am trying to gather some series from multiple disks to a single disk. I first choose "Gather", then select the top level folder i want to move under "Select folder", and then choose next. On the "Choose target folder" nothing happens. No target disks appears (even though 3 disks showed up the first time i tried it, but I still couldn't continue), and the logo-thing on the top-rights is "pulsing". Both the previous and next buttons are greyed out, and I cannot do anything. Also, I get a green banner under the top menu which says "unBALANCE needs exclusive access to disks, so disable mover and/or any dockers that write to disks, before running it. Also note that transfer speed may be affected by disk health. Check this plugin.". I have removed the DiskSpeed docker, and stopped all other dockers. I have even tried to disable all dockers from the top level settings in unraid. Any thoughts? Edited April 24, 2020 by TrondHjertager Quote Link to comment
Abigel Posted April 25, 2020 Share Posted April 25, 2020 I have made a big mistake... i moved files in dry mode and than format this disk... Can i rebuild my Disk with parity ? If yes, what is the best way? Quote Link to comment
JonathanM Posted April 26, 2020 Share Posted April 26, 2020 On 4/25/2020 at 12:36 PM, Abigel said: format this disk... Can i rebuild my Disk with parity ? No. Parity is realtime, when you format the disk, that is immediately applied to parity. If the data on the disk is irreplaceable from backup or reripping, then data recovery software may be able to help, but it's not simple to do. 1 Quote Link to comment
jbrodriguez Posted April 26, 2020 Author Share Posted April 26, 2020 On 4/24/2020 at 7:20 AM, TrondHjertager said: I am having some problems with unBALLANCE. Hi, this looks like a bug. Can you check the Chrome developer's console for any issues ? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.