[Plugin] unbalanced


Recommended Posts

When you get to the page "This site can't be reached", can you do a hard refresh ?

 

In macOS, either

- Command + Shift + R

- Shift + <reload button>

 

In Windows, either

- Ctrl + <reload button>

- Ctrl + F5

 

I developed on my test server.

 

When I tried it on my production server, I had a similar issue, but it was a browser cache issue.

 

Let me know if that works.

Link to comment

unBalance (2018.09.18a - 5.4.0)

unRAID (6.6.0)

 

When I move a folder from one disk to another, an empty SHOW/MOVIE folder is left on the source disk.  How can this be prevented?

 

Directory Structure:     /Media/Series/<FOLDER FOR SHOW>

Split level:    Top twp directories.

 

Note: this behavior started with unBalance 5.4.0, and also occurred on unRAID 6.5.3.

 

If I re-run the move, it will remove the empty folders.  But only one will get removed, so I have to run it one time for each empty folder.

 

But if I navigate to the share via the network, I can only see one folder with the data in it, or an empty folder if the folder was originally empty to start with, which occurs sometimes when a program creates a folder anticipating data.

 

UPDATE:  When I try to move an empty folder this line appears in the logs:

 

"PLANNING: The following items will not be transferred, because there's not enough space in the target disks:"

 

BUT...the target has 2.79TB free.

 

 

Edited by DoItMyselfToo
Link to comment

Hi, can you send the log (/boot/logs/unbalance.log) ?

 

5.4.0 didn't touch the scatter/gather logic, it was just a front-end change.

 

The not enough space means you're trying to transfer more data than the free space on the disk.

 

It will not remove top-level share folders (/mnt/disk<x>/series and such), but it should remove lower level folders, if empty.

Link to comment
On 9/21/2018 at 9:37 AM, jbrodriguez said:

Hi, can you send the log (/boot/logs/unbalance.log) ?

 

5.4.0 didn't touch the scatter/gather logic, it was just a front-end change.

 

The not enough space means you're trying to transfer more data than the free space on the disk.

 

It will not remove top-level share folders (/mnt/disk<x>/series and such), but it should remove lower level folders, if empty.

Log attached.

 

After posting, I read through more posts and saw the updates this version made.  I can only report the timeline as I know it.  As for "not enough space", I know that my target had 2.79TB free, and I was only trying to move 5 or 6 empty folders.  In my case "media" is my top level share.  Media share settings handles whether "media" shows up on any "x" disk.

 

I'm having problems with the lower level folders.  Specifically, when moving folders with content, the folder with content gets moved to the target but an empty folder of the same name is left behind on the source disk.  Also, if I try to move the empty folders, I get the error message that the target has too little space.  But if I then choose a bunch of folders with GB's of data, they move just fine to the same target, but again the empty folders are left on the source disk.

 

One last thing, at some point recently I received a message in unBalance after performing the initial "plan" to update permissions via the Fix Common Problems Docker Safe New Perms, which I did before completing the operation.

 

Thank you for looking at this.

 

Edited by DoItMyselfToo
Link to comment

Hi,

 

Hadn't had the chance to post a reply.

 

There are a couple of things, from checking your logs.

 

There are many instances where the plugin isn't deleting folders, by design, because the rsync operation was flagged with code 23 (which can mean permission issues, datetime issues and honestly pretty much anything).

 

The plugin doesn't delete the source in these cases and shows these commands with a yellow icon and an rmsrc label in the history page, so you can eventually remove the source (rmsrc), after you manually check it's ok to do so.

 

Many commands are being flagged due to the file seemingly not being present:

"rsync: link_stat "/mnt/disk2/media/films-auto/City Lights (1931)/City Lights (1931) - Bluray-720p.mkv" failed: No such file or directory (2)"

 

Are you running mover while unBALANCE is running ? Or perhaps a nzbget/torrent docker that move folders around ?

 

Is the only thing I can think of.

Link to comment

I remembered I had this kind of reports a while back, checked your log again and see you're doing a Scatter/Move.

 

I run an additional "purge" routine, for parent folders that are empty, but it applies only for the Gather operation.

 

I don't remember why that's the case, may it was because it was reported on the Gather operation only.

 

I can't find a reason currently not to do it for the Scatter/Move operation, I will look at some other scenarios, but will most likely add it for Scatter/Move in the next release.

 

In any case, please do remember that unBALANCE should run exclusively.

 

If you have other apps/plugins/etc, moving content around, the results are unpredictable.

Edited by jbrodriguez
Link to comment

 

Hi all,

 

 I'm puzzled. I just added a new drive to my array (formatted it, etc.) and I can move files to/from it just fine in a shell using mc, but somehow, Unbalance is completely unable to offer me this brand new emtpy 4TB drive as a destination for a "gather" operation. I would like to move all files pertaining to a share to this new drive, but alas, nothing but my cache drive is offered as a destination (the other drives are quite full, up to the 512MB set as a limit).

 

 Of course, I tried stopping/restarting unbalance, the array… I even rebooted the whole server, but no luck in getting this new drive as a destination. How can I "force" that as a destination in a gather operation?

Link to comment
4 hours ago, jbrodriguez said:

If not offered, it's almost certain that the size of the entire share is 4Tb+.

Try running 


du -h /mnt/user/<your share name>

 

Thanks a million! That was indeed the case. It was my nextcloud share, and somehow, between the trashbin and a few redundant folders, I had more than 5.1 TB on this share!

 

It's weird Unbalance offered the cache drive as destination then, or didn't warn me there wasn't enough space. But thanks! Mystery solved now!

Edited by denishay
Link to comment

Cool !

 

13 hours ago, denishay said:

offered the cache drive as destination

 

What's the size of your cache drive ?

Is it a pool of drives ?

 

Could you run 

df --block-size=1 /mnt/*

and send me the output (pm if you prefer).

 

It would also help if you could send me /boot/logs/unbalance.log, to further understand.

 

I do take some precautions with the cache drive, but maybe something changed or I had it wrong.

Link to comment
3 hours ago, jbrodriguez said:

What's the size of your cache drive ?

Is it a pool of drives ?

Nope, an single SSD. 240GB, nothing extraordinary.

 

Quote

Could you run 


df --block-size=1 /mnt/*

and send me the output (pm if you prefer).

 

Filesystem          1B-blocks           Used     Available Use% Mounted on
/dev/sdc1        239947935744    97084276736  142863659008  41% /mnt/cache
/dev/md1        1999422144512  1214729420800  784692723712  61% /mnt/disk1
/dev/md2         999716474880   247712878592  752003596288  25% /mnt/disk2
/dev/md3        2999127797760  1427599224832 1571528572928  48% /mnt/disk3
/dev/md4        2999127797760  2593694089216  405433708544  87% /mnt/disk4
/dev/md5        3998833471488  3454259048448  544574423040  87% /mnt/disk5
/dev/md6        3998833471488  2016894824448 1981938647040  51% /mnt/disk6
rootfs             4140683264      691458048    3449225216  17% /mnt
shfs           17235009093632 11051973763072 6183035330560  65% /mnt/user
shfs           16995061157888 10954889486336 6040171671552  65% /mnt/user0
 

 

Quote

It would also help if you could send me /boot/logs/unbalance.log, to further understand.

 

I do take some precautions with the cache drive, but maybe something changed or I had it wrong.

Unfortunately, in between, I had completely uninstalled and reinstalled it in hope of "fixing" what I thought was an issue. So the log is now empty.

Edited by denishay
Link to comment
  • 2 weeks later...

I'm suddenly not having success running this.  I've tried to scatter the files off of one drive to others, and after an hour of work, it gives a message such as this one...

 

skipping:deletion:(rsync command was flagged):(/mnt/disk2/video/Racing/Formula 1/2012/Formula 1 - 2012 - Round 17 - India - Race.mkv)

 

Why is it skipping the deletion?

 

It seems I now have these files on both drives, but they show as one file when I view thru user shares.

 

I tried to move them with MC in putty, but that just hangs MC.

 

I then tried to gather the files to a disk, hoping that would remove them from the disk I'm trying to clear, but that is what actually generated the error above.

 

I don't have any idea at this point how to consolidate these files.  I've tried several times, and the files don't move off of the disk I'm trying to clear.

 

Is there some way to reset the program and get it to actually move then delete the files?

Link to comment
On 10/8/2018 at 11:33 AM, JustinChase said:

Why is it skipping the deletion?

It skips deletion when there was an error with the rsync command, specifically error 23, which is quite generic.

 

When that happens the rsync command is flagged and when you look at the operation in the History tab, it appears with a yellow icon and an 'rmsrc' button on the left.

 

The idea is that you check why it was flagged (permissions, timestamps, etc) and if you want, you manually remove the source with the 'rmsrc' button (some more details here)

 

You have performed some other operations, so you can't replay that specific operation.

 

Why don't you delete the sources manually, not optimal but it should work.

Link to comment
  • jbrodriguez changed the title to [Plugin] unbalanced

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.