[Plugin] unbalanced

[Urlryn]

I'm moving around Movies, Anime and TVShows.  I started with Anime.

 

As for special characters...most don't have anything outside the norm.

They typically will have a period, comma, dash, squared brackets and/or exclamation points.

 

See the screen shots

 

Sample folder listing

 

Sample File Name List:

 

As for the size....I've tried multiple tests. The error happens with selecting 1 folder, Bleach for example, or selecting many of folders.

Selection total > 20GB fails

Selection total < 20GB transfers no problem

 

Folder Selection < 20GB

 

 

 

Completed:

 

 

So now i'll select the same 3 series + add Bleach.  This batch will be > 20GB.

 

Click Proceed and goes right to here:

 

Log only shows this one error:

W: 2017/12/17 08:38:20 server.go:331: closing socket (&{ws:0xc420116870 onMessage:0x6e6730 onClose:0x6e6780}): unexpected end of JSON input

 

 

 

BTW.....Sorry for all the posts!!

 

Looking at the logs saw something else that looked a tad strange....grabbed 7 series that were about 22GB

In the logs I see this:

I: 2017/12/17 09:44:41 planner.go:490: [143.32 MB] /mnt/disk4/media/anime/Denpa Onna to Seishun Otoko (2011)/./Ground Control To Psychoelectric Girl - Special 02 - The Nightmare Sun [1280x720 x264 AAC 2ch] [F327A5D6].mkv
I: 2017/12/17 09:44:41 planner.go:493: ---------------------------------------------------------
I: 2017/12/17 09:44:41 planner.go:494:
I: 2017/12/17 09:44:41 planner.go:485: =========================================================

I: 2017/12/17 09:44:41 planner.go:486: disk(/mnt/cache):items(77)-(22.27 GB):currentFree(321.28 GB)-plannedFree(299.01 GB)

I: 2017/12/17 09:44:41 planner.go:487: ---------------------------------------------------------
I: 2017/12/17 09:44:41 planner.go:490: [343.12 MB] /mnt/disk2/media/anime/Chaos;Child (2017)/./Chaos;Child - 07 - Episode 7 [1280x720 x264 AAC 2ch].mkv
I: 2017/12/17 09:44:41 planner.go:504: =========================================================

I: 2017/12/17 09:44:41 planner.go:505: Bytes To Transfer: 155.89 GB

I: 2017/12/17 09:44:41 planner.go:506: ---------------------------------------------------------
W: 2017/12/17 09:44:48 server.go:331: closing socket (&{ws:0xc420116240 onMessage:0x6e6730 onClose:0x6e6780}): unexpected end of JSON input

 

Why does it say Bytes to Transfer is 155GB? when in the previous line is shows 22.27GB?

Also what is it doing with the cache disk in that same line?....out of curosity

 

Anything else i can test for ya!  Let me know!

 

Urlryn

 

Edited December 19, 2017 by Urlryn

[bmfrosty]

I've had similar problems in 4.0.0 as Urlryn, but my assumption has been different, since I've been less patient.   I've gotten his error:

 

W: 2017/12/17 07:59:25 server.go:331: closing socket (&{ws:0xc4201c4090 onMessage:0x6e6730 onClose:0x6e6780}): unexpected end of JSON input

 

But I've also seen this:

 

W: 2017/12/17 08:00:49 server.go:331: closing socket (&{ws:0xc4200fc240 onMessage:0x6e6730 onClose:0x6e6780}): invalid character ' ' in string escape code

 

I think in my vast anime collection I have some filenames that aren't being escaped correctly for the JSON and are causing the JSON interpreter to dump.

[bmfrosty]

One of my movie directories fails too.  Here are some interesting directory names:

'Master\ Harold'\ ...\ And\ the\ Boys\ (2010)/
*batteries\ not\ included\ (1987)/

Scotland,\ Pa.\ (2001)/

WALL·E\ (2008)/

 

Maybe these help?

 

Would be happy to PM you logs if that would help.

Edited December 17, 2017 by bmfrosty

[Squid]

One of my movie directories fails too.  Here are some interesting directory names:

'Master\ Harold'\ ...\ And\ the\ Boys\ (2010)/
*batteries\ not\ included\ (1987)/
Scotland,\ Pa.\ (2001)/
WALL·E\ (2008)/
 
Maybe these help?
 
Would be happy to PM you logs if that would help.

You really should rename batteries included. The * is an invalid character when dealing with SMB. Whether you use it now or possibly later it is something you're only going to have problems with

[jbrodriguez]

Thanks Urlryn and bmfrosty.

 

I will do some testing to figure this out.

[bmfrosty]

Great.  I finished my operation with rsync, but once your update is available, I'll be happy to retest the smaller share that had the issue.

[DanByrne]

Hi All,

FYI I am getting the same 'unexpected end of JSON input' error on version 4.0.0 - although I havent yet figured out which folders are causing the issue...

[Cessquill]

Hi - is there any way of pausing/cancelling?  I've got three entries queued up in a scatter (freeing up a drive to reformat), and it's got 51h remaining.  However, I've got a different replacement drive precleared waiting to add to the array that I'd like to get in if possible.  Don't mind letting the first of three finish.  Thought that clicking on the "-" icon next to each entry would remove.

[jbrodriguez]

5 hours ago, Cessquill said:

is there any way of pausing/cancelling?

 

Not built-in yet.

 

But you could

killall rsync

and it should stop the operation gracefully.

 

If that doesn't work, repeat the killall rsync and add a 

killall unbalance

 

[Cessquill]

Thanks.  I'll let it finish the first share and try killing it.

[Cessquill]

19 hours ago, jbrodriguez said:

killall rsync

Thanks for your advice.

 

This command gracefully stopped the sync at its current position.

 

I waited for the first command to complete (the first share to transfer).  For anybody else, this is not just when unBALANCE says "Command Finished" (or similar), but when all folders have been removed from the source drive.  Wait for the next share to start, then run the command (can be quite a while if - like me - you were moving 2.5TB).  Then delete the portion that has already transferred from this new share.

[JustinAiken]

Just tried this after using the commandline `diskmv` from the MUSDL plugin for awhile... looks really nice, but doesn't seem to work from me.  Just about every move I try, stops immediately with a `exit status 23 : Partial transfer due to error` 

 

But if I drop to the command line, and manually paste in the rsync command, it goes through..  

 

Actually, watching the output from the current `rsync` command I copy pasted in, I think that `.git` repos break it..

 

 

[jbrodriguez]

On 12/31/2017 at 12:46 PM, JustinAiken said:

But if I drop to the command line, and manually paste in the rsync command, it goes through..

 

Hi JustinAiken, thanks for the feedback !

 

Yes, I've experienced this too.

 

I want to change the way rsync is invoked from unBALANCE, right now it fails in instances where a command line invocation just reports a warning.

[RGauld]

I seem to be getting this every time I start unRAID:

 

"/bin/bash: line 3: /usr/local/emhttp/plugins/unbalance/scripts/stop: No such file or directory"

 

[jbrodriguez]

Hi RGauld, yes, it's a known issue, but harmless.

 

It has to do with the install script.

 

I will eventually get it fixed.

[jbrodriguez]

v4.0.1 (2018.01.11) is out !

 

This is small maintenance release to help figure out the json parsing error.

 

Towards this end, the go websocket library has been replaced with the gorilla's version.

 

Additionally, separate warnings are being logged for transmission and parsing errors.

 

Additional changes are:
- Fix plugin install script (remove error shown on server boot)
- Add date based plugin version
- Other bug fixes and improvements
 

I tested json parsing with all the folder names you sent and couldn't reproduce the error.

What I did is change websocket library and log separate errors as per description above.

If/when you get the error, please post it, hopefully it should help identify the cause.

 

The test was done on Chrome, Safari and Firefox (mac), so I don't rule out it may have to do with some windows/linux browser issue.

 

Also I wasn't able to repro the floating footer error either, but I need to overhaul the front-end code anyway.

 

Let me know how it goes !

[SoAvenger]

I just tried to install unbalance with Community Applications and I get this error;

 

 

plugin: installing: https://raw.githubusercontent.com/jbrodriguez/unraid/master/plugins/unbalance.plg
plugin: downloading https://raw.githubusercontent.com/jbrodriguez/unraid/master/plugins/unbalance.plg
plugin: downloading: https://raw.githubusercontent.com/jbrodriguez/unraid/master/plugins/unbalance.plg ... done
plugin: downloading: "https://github.com/jbrodriguez/unbalance/releases/download/2018.01.11/unbalance-2018.01.11.tgz" ... failed (Invalid URL / Server error response)
plugin: wget: "https://github.com/jbrodriguez/unbalance/releases/download/2018.01.11/unbalance-2018.01.11.tgz" download failure (Invalid URL / Server error response)
 

Is this a Github problem?

[jbrodriguez]

Hi SoAvenger,

 

Mmm, it wasn't supposed to use that url. I'll double check that.

 

But it's fixed in 4.0.2 which is now available.

[SoAvenger]

3 hours ago, jbrodriguez said:

Hi SoAvenger,

 

Mmm, it wasn't supposed to use that url. I'll double check that.

 

But it's fixed in 4.0.2 which is now available.

Installed and putting it to work, Thank You.

[bmfrosty]

I: 2018/01/12 09:07:22 planner.go:493: ---------------------------------------------------------

I: 2018/01/12 09:07:22 planner.go:494:

I: 2018/01/12 09:07:22 planner.go:504: =========================================================

I: 2018/01/12 09:07:22 planner.go:505: Bytes To Transfer: 727.02 GB

I: 2018/01/12 09:07:22 planner.go:506: ---------------------------------------------------------

W: 2018/01/12 09:07:33 server.go:331: closing socket (&{ws:0xc4201ac480 onMessage:0x6e6730 onClose:0x6e6780}): invalid character ' ' in string escape code

I: 2018/01/12 09:07:33 core.go:158: Sending operation

I: 2018/01/12 09:07:43 core.go:153: Sending state

I: 2018/01/12 09:07:43 core.go:148: Sending config

I: 2018/01/12 09:07:43 core.go:163: Sending storage

[jbrodriguez]

bmfrosty, that warning seems to come from v4.0.0.

 

Can you make sure you're on 4.0.2 ?

[ashman70]

Just a quick question, I want to free up a disk to changes it's file system, rather than manually going through folders to figure out what data is on this particular disk, is there a way to just select all the data on say disk 1 without having to manually check?

[sureguy]

Getting an XML Parse error when trying to install manually, using Community Applications I don't get an option to install.  

 

Edit - Rebooted - and installation worked - odd

Edited January 13, 2018 by sureguy
Update

[RGauld]

thanks for the update!  I no longer receive that message when I boot unRAID now...

Boot up screen show no errors, and the unBALANACE interface works great!

Thanks for your help!

 

 

[woble]

There's an issue with the footer where it floats on top of long lists. Apparently was brought up before but never fixed.

 

Easy to fix as follows.

 

1. Change `height` on the `body` class to `min-height`. This will have the footer at the bottom of the viewport if there's plenty of space, otherwise other content will push it down outside the viewport.

2. Remove use of `height100` class. It relies on viewport height and unnecessarily clamps the height of a block that is taller than 100vh.