June 28, 20179 yr Author 48 minutes ago, insaneavi said: I installed the plugin via the github link I will check if this causes trouble. 49 minutes ago, insaneavi said: i am unable to get the status of it to change from Stopped Do you mean that all the buttons are disabled ? If that's the case, follow the quick start instructions. If that's not the case, please do the following $ killall unbalance $ cd /usr/local/emhttp/plugins/unbalance $ ./unbalance and post the output here to check it out.
June 28, 20179 yr 34 minutes ago, jbrodriguez said: I will check if this causes trouble. Do you mean that all the buttons are disabled ? If that's the case, follow the quick start instructions. If that's not the case, please do the following $ killall unbalance $ cd /usr/local/emhttp/plugins/unbalance $ ./unbalance and post the output here to check it out. I have also attached what the web inference displays when i try to start the application. as requested: root@TOPSITE:/usr/local/emhttp/plugins/unbalance# killall unbalance unbalance: no process found root@TOPSITE:/usr/local/emhttp/plugins/unbalance# cd /usr/local/emhttp/plugins/unbalance root@TOPSITE:/usr/local/emhttp/plugins/unbalance# ./unbalance root@TOPSITE:/usr/local/emhttp/plugins/unbalance# ls CHANGES VERSION event/ img/ scripts/ unbalance.page README.md app/ images/ index.html unbalance* unbalance.png Edited June 28, 20179 yr by insaneavi
June 28, 20179 yr Author 11 minutes ago, insaneavi said: root@TOPSITE:/usr/local/emhttp/plugins/unbalance# ./unbalance This is the problem here. The executable isn't starting. This used to be an issue in 1.8.x, but it was "solved" I believe that installing via the github link doesn't create the conf file it expects, although it should work without one. Try the following : $ nano /boot/config/plugins/unbalance/unbalance.conf # <paste the following> SERVICE="disable" PORT="6237" RUNAS="nobody" # click control + x, then y, then <enter> to save the file Then go the settings page and enable the server.
June 29, 20179 yr 9 minutes ago, jbrodriguez said: This is the problem here. The executable isn't starting. This used to be an issue in 1.8.x, but it was "solved" I believe that installing via the github link doesn't create the conf file it expects, although it should work without one. Try the following : $ nano /boot/config/plugins/unbalance/unbalance.conf # <paste the following> SERVICE="disable" PORT="6237" RUNAS="nobody" # click control + x, then y, then <enter> to save the file Then go the settings page and enable the server. tried it, same issue: root@TOPSITE:/boot/config/plugins/unbalance# ls unbalance-3.0.0.tgz* unbalance.cfg* unbalance.conf* root@TOPSITE:/boot/config/plugins/unbalance# I was able to create the .conf file and save with the above code, but still unable to start it. Do i need to reboot the server after creating the conf file? here is the system log Jun 28 20:05:33 TOPSITE php: /usr/local/emhttp/plugins/unbalance/scripts/start Jun 28 20:05:33 TOPSITE sudo: root : TTY=unknown ; PWD=/usr/local/emhttp ; USER=nobody ; COMMAND=/bin/bash -c /usr/local/emhttp/plugins/unbalance/unbalance -port 6237 Edited June 29, 20179 yr by insaneavi
June 29, 20179 yr Author 4 minutes ago, insaneavi said: Do i need to reboot the server after creating the conf file? Not really. So, if you try $ cd /usr/local/emhttp/plugins/unbalance $ ./unbalance it comes up empty again ?
June 29, 20179 yr 7 minutes ago, jbrodriguez said: Not really. So, if you try $ cd /usr/local/emhttp/plugins/unbalance $ ./unbalance it comes up empty again ? root@TOPSITE:~# cd /usr/local/emhttp/plugins/unbalance root@TOPSITE:/usr/local/emhttp/plugins/unbalance# ./unbalance configuration variable provided but not defined: SERVICE Usage of ./unbalance: -apiFolders string folders to look for api endpoints (default "/var/local/emhttp") -config string config location (default "/boot/config/plugins/unbalance/unbalance.conf") -dryRun perform a dry-run rather than actual work (default true) -folders string deprecated - do not use -log string pathname where log file will be written to (default "/boot/logs/unbalance.log") -notifyCalc int notify via email after calculation operation has completed (unraid notifications must be set up first): 0 - No notifications; 1 - Simple notifications; 2 - Detailed notifications -notifyMove int notify via email after move operation has completed (unraid notifications must be set up first): 0 - No notifications; 1 - Simple notifications; 2 - Detailed notifications -port string port to run the server (default "6237") -rsyncFlags string custom rsync flags root@TOPSITE:/usr/local/emhttp/plugins/unbalance#it now gives me the above resultslog entry when trying to enable: Jun 28 20:15:41 TOPSITE php: /usr/local/emhttp/plugins/unbalance/scripts/start Jun 28 20:15:41 TOPSITE sudo: root : TTY=unknown ; PWD=/usr/local/emhttp ; USER=nobody ; COMMAND=/bin/bash -c /usr/local/emhttp/plugins/unbalance/unbalance -port 6237 Edited June 29, 20179 yr by insaneavi
June 29, 20179 yr Author Ok, remove the contents of the file (/boot/config/plugins/unbalance/unbalance.conf), but leave it there (an empty file). Then start it from the command line again.
June 29, 20179 yr 5 minutes ago, jbrodriguez said: Ok, remove the contents of the file (/boot/config/plugins/unbalance/unbalance.conf), but leave it there (an empty file). Then start it from the command line again. Thank you!, it now runs properly, I was also able to stop the service, then turn it back on via the web interface!
June 29, 20179 yr Author Just now, insaneavi said: Thank you!, it now runs properly, I was also able to stop the service, then turn it back on via the web interface! Awesome ! I'll make some fixes, so that it works properly. Thanks for the report !
June 29, 20179 yr I did some massive file movement - in fact it was consolidation of shares to dedicated drives, as the spread led to significant spin-up times. I'd like to feed back some of my experience and suggest some points that can possibly be improved. 1. Most of my issues rooted in missing permissions. Unfortunately rsync's failure messages are very cryptic thus useless most of the time. It will never tell you that you have permission issues! The plugin does, but not "strong enough". Therefore it is a matter of usability to add this utility to the plugin. Maybe provide a "change permissions" button if the dry run finds a related issue. It's annoying if you do the "dry run" and are told to "install docker xy first". Much more simple and without the need of a docker is the "newperms" script that comes with unRAID. Long story short, it is recommended to run the "newperms" on the source AND on the target! The plugin should have this option included. 2. Due to 1) I had many aborted runs which left me with lots of duplicates. Obviously the command aborted at some point during the copy so it didn't even come to the "remove sources". After figuring out that the permissions were the issue, I ran the job again and that removed the sources but I had to remember the exact task! My task was consolidating so it was easy to find the dupes as they were the sources, but I imagine this can turn into a problem. e.g. I have some shares spanning multiple drives and I found a couple of identical files on different drives of the share. Must have been moving the source to different drives in different sessions. Need to hunt the dupes now. I have no solution how to handle this, but if an rsync-command is failing at some point, dupes will be created. Maybe there is an rsync option to "remove sources" right after the "copy" of each file? Also a more comprehensive log of commands issues could be useful. Maybe somewhat like: date - rsync xyz source target etc. success - 0 errors - run again knob At the moment the log is too much. Maybe let's have some log levels (normal/debug/full)? That leads straight to 3. Logs are fine but it turned out that the unRAID OS update aborted due to space shortage. You know what I'm gonna say...there must be some sort of log handling. Rotation, size limit, whatever. I had hundreds of MB unbalance log data... Anyway, many thanks for the hard work! ...sry, it got quite long Edited June 29, 20179 yr by Fireball3
June 29, 20179 yr Hello, I am trying to update the plugin and I am receiving the error "plugin: wget: "https://github.com/jbrodriguez/unbalance/releases/download/3.0.0/unbalance-3.0.0.tgz" download failure (Invalid URL / Server error response)" I used the App Tab (Community Application) and also copy and pasting the url "https://raw.githubusercontent.com/jbrodriguez/unraid/master/plugins/unbalance.plg" in the plugins tab. It's been happening for a couple of days now, is this only on my end? Thanks
June 29, 20179 yr 1 minute ago, mikegiann said: Hello, I am trying to update the plugin and I am receiving the error "plugin: wget: "https://github.com/jbrodriguez/unbalance/releases/download/3.0.0/unbalance-3.0.0.tgz" download failure (Invalid URL / Server error response)" I used the App Tab (Community Application) and also copy and pasting the url "https://raw.githubusercontent.com/jbrodriguez/unraid/master/plugins/unbalance.plg" in the plugins tab. It's been happening for a couple of days now, is this only on my end? Thanks I just tried (in chrome) pasting the url for the .tgz you quoted in your first paragraph, and it downloaded the file just fine so it looks like that file is there. Can you ping github from your server?
June 29, 20179 yr 8 minutes ago, trurl said: I just tried (in chrome) pasting the url for the .tgz you quoted in your first paragraph, and it downloaded the file just fine so it looks like that file is there. Can you ping github from your server? Yes, I can ping and get a response, here is the complete log when I try to install: plugin: installing: https://raw.githubusercontent.com/jbrodriguez/unraid/master/plugins/unbalance.plg plugin: downloading https://raw.githubusercontent.com/jbrodriguez/unraid/master/plugins/unbalance.plg plugin: downloading: https://raw.githubusercontent.com/jbrodriguez/unraid/master/plugins/unbalance.plg ... done plugin: downloading: "https://github.com/jbrodriguez/unbalance/releases/download/3.0.0/unbalance-3.0.0.tgz" ... failed (Invalid URL / Server error response) plugin: wget: "https://github.com/jbrodriguez/unbalance/releases/download/3.0.0/unbalance-3.0.0.tgz" download failure (Invalid URL / Server error response) If I click on the link https://raw.githubusercontent.com/jbrodriguez/unraid/master/plugins/unbalance.plg I do see the contents of that page, just not through Unraid. I tried with Firefox and Chrome all latest versions. Edited June 29, 20179 yr by mikegiann Browser Used
June 29, 20179 yr 1 hour ago, mikegiann said: If I click on the link https://raw.githubusercontent.com/jbrodriguez/unraid/master/plugins/unbalance.plg I do see the contents of that page If you instead click on the link https://github.com/jbrodriguez/unbalance/releases/download/3.0.0/unbalance-3.0.0.tgz does it download the file? That's what I did earlier when I said 1 hour ago, trurl said: I just tried (in chrome) pasting the url for the .tgz you quoted in your first paragraph, and it downloaded the file just fine so it looks like that file is there. And just now I tried using CA to install this plugin and it worked for me. Have you tried any other plugins? Do they work?
June 29, 20179 yr Author Thanks for your comments Fireball3 ! 6 hours ago, Fireball3 said: 1. Most of my issues rooted in missing permissions. .... Long story short, it is recommended to run the "newperms" on the source AND on the target! The plugin should have this option included You're right about rsync errors being cryptic About the "dockers safe newperms" (by Squid), it runs on ALL the disks, so that's why it's suggested to run that first In any case, I understand that sometimes you just want to run it on the source (disk or folders), and probably on the target as you mention. I'll take this into consideration. 6 hours ago, Fireball3 said: 2. Due to 1) I had many aborted runs which left me with lots of duplicates. ...... I had to remember the exact task! My task was consolidating so it was easy to find the dupes as they were the sources, .... Maybe there is an rsync option to "remove sources" right after the "copy" of each file? Also a more comprehensive log of commands issues could be useful. Maybe somewhat like: date - rsync xyz source target etc. success - 0 errors - run again knob At the moment the log is too much. Maybe let's have some log levels (normal/debug/full)? You're right, I feel the log is too much too. It logs everything rsync outputs, and if you have many files, it's a lot of output I will look into improving that. Oh, I wanted to mention, it reports each command it ran (even after a failure) in the log file, so you don't need to remember the exact task. It will also send an email if you have notifications set up. 6 hours ago, Fireball3 said: 3. Logs are fine but it turned out that the unRAID OS update aborted due to space shortage. It currently rotates logs, a max of 10 log files of 10Mb each. It shouldn't grow to the hundreds of Mb. In your case, do you have a single 100Mb+ file ? Or is it multiple log files in excess of that ? 6 hours ago, Fireball3 said: Anyway, many thanks for the hard work! Thanks again for the comments ! They help improve the plugin.
June 29, 20179 yr Author 58 minutes ago, trurl said: And just now I tried using CA to install this plugin and it worked for me. Have you tried any other plugins? Do they work? mikegiann, the same for me, I installed with CA with no issues. Just as a test, if you $ wget https://github.com/jbrodriguez/unbalance/releases/download/3.0.0/unbalance-3.0.0.tgz from the a command line on the server, does it work ?
June 29, 20179 yr 1 hour ago, jbrodriguez said: mikegiann, the same for me, I installed with CA with no issues. Just as a test, if you $ wget https://github.com/jbrodriguez/unbalance/releases/download/3.0.0/unbalance-3.0.0.tgz from the a command line on the server, does it work ? I was able to download the files from the browser but not through Unraid, then when trying it from the command line, as you suggested, I saw where the error occurred, it was refusing the certificate from my self signed Untangle UTM appliance. Once I added the github to the exception of not to be scanned by SSL Inspector, the plugin installed quickly and correctly. All the other plugins that I have update automatically without problem, that is why I did not think about the Untangle appliance. Thanks for pointing me in the right direction and the help. Edited June 29, 20179 yr by mikegiann
June 30, 20179 yr Author v3.1.0 is available ! 2017-06-30 - 3.1.0 Log verbosity can now be modified through a setting: - Normal Don't print rsync output - Full Print full rsync output When a transfer has finished, it will now log how much data was transferred and the approximate speed (in MB/s). Additionally, the following changes were made: - Fix an issue when refreshing the UI with notifications enabled - Improve handling of config file (when not present) - Other bug fixes and improvements
July 1, 20179 yr Author v3.1.1 is available 2017-07-01 - 3.1.1 Make the transfer operation use the latest dry-run setting
July 1, 20179 yr 1st time using Unbalance (v3.1.0) and I ran into an issue. I followed the steps as listed: selected the "From" drive, selected 1 "To" drive selected folders to move clicked the CALCULATE button - reported 330GB of files to move clicked "move" with "Dry Run" checked - log output look OK Repeated the above steps, only with "Dry Run" unchecked Process finished in 20 seconds The "To" drive has no files, the the "From" drive is now empty. The entries from the Log shows: .... I: 2017/06/30 22:14:07 core.go:802: Command Started: rsync -avPRX --dry-run "Thumbnails/2" "/mnt/disk6/" I: 2017/06/30 22:14:07 core.go:884: Command Finished I: 2017/06/30 22:14:07 core.go:890: Current progress: 99.71% done ~ 0s left (47655.92 MB/s) I: 2017/06/30 22:14:07 core.go:900: Removing: rm -rf "/mnt/disk4/Thumbnails/2" I: 2017/06/30 22:14:08 core.go:802: Command Started: rsync -avPRX --dry-run "Thumbnails/a" "/mnt/disk6/" I: 2017/06/30 22:14:08 core.go:884: Command Finished I: 2017/06/30 22:14:08 core.go:890: Current progress: 99.74% done ~ 0s left (45807.58 MB/s) I: 2017/06/30 22:14:08 core.go:900: Removing: rm -rf "/mnt/disk4/Thumbnails/a" ... As you can see the rsync command still has "--dry-run" even though the "Dry Run" box had been unchecked. Is there a way of seeing the complete log to see what was all removed? Thanks Edited July 1, 20179 yr by KeithG Fix typos
July 3, 20179 yr Author On 7/1/2017 at 10:31 AM, KeithG said: Is there a way of seeing the complete log to see what was all removed? Hi, logs are available at /boot/logs/unbalance.log. I made a change to how dry-run is handled even after a calculation. Send me the logs (on pm if you want) to further check.
July 6, 20178 yr Just downloaded. Gather is listed as coming soon, but described on first page of this thread. Is this part working or not?
July 6, 20178 yr Author 3 hours ago, 1812 said: Is this part working or not? It's still work in progress. I'll edit the first post.
July 6, 20178 yr 3 minutes ago, jbrodriguez said: It's still work in progress. I'll edit the first post. Thanks, I was confused.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.