[Support] ich777 - Application Dockers

[Kilrah]

  

36 minutes ago, miicar said:

I hope someone can tell me what i am doing wrong?  Or do is LB just not able to see exclusive access shares, and I have to go back to writing manual R-Sync scripts for everything?  I want to use my exclusive shares again and LB.

 

What's "LB"?

If you have exclusive access shares then you map /mnt/poolname/sharename directly instead of /mnt/user/sharename and you're good.

Edited January 2 by Kilrah

[kevco97]

Hello,

i have a question about luckyBackup.

 

If i schedule my backuptask it doesn´t work. But if i manually start the Backuptask it works.

What can i do or what should i check?

Thank you.

Regards
Kevin

[miicar]

1 hour ago, kevco97 said:

If i schedule my backuptask it doesn´t work. But if i manually start the Backuptask it works.

Make sure you guys have "Console Mode" selected in the Schedule profile...that stumped me for a bit too, but it does work on schedule after i checked that.  Make sure you re-"cronIT !!" after making those changes.

 

1 hour ago, Kilrah said:

 What's "LB"?

LuckyBackup....
 

 

1 hour ago, Kilrah said:

If you have exclusive access shares then you map /mnt/poolname/sharename directly instead of /mnt/user/sharename and you're good.

except it doesn't work like that in LB (I'm guessing you didn't test this before replying).  Also, this is the "old" way of doing things since unraid now does it automatically and all locations should be "mnt/user/(sharename)" and unraid decides if its exclusive or FUSE access, since the update!  So my guess is LB needs an update to see this new feature! 

Edited January 2 by miicar
missing info

[Kilrah]

3 minutes ago, miicar said:

So my guess is LB needs an update to see this new feature! 

It doesn't, this affects all containers. Exclusive access works through making a symlink, and there are limitations to where this works transparently.

 

Just map the pool itself, you don't need to rely on the default mounts/can add as many of yours as you want.

Edited January 2 by Kilrah

[miicar]

11 minutes ago, Kilrah said:

Just map the pool itself.

This is not a solution, as sometimes the data is on BOTH the cache and the array (before mover runs)...also, i just tested it again, and i cannot map to the pool itself, unless i change the mount point to "/" use the "mnt/user/mnt/user/(sharename)", but thats giving LB way more access then is needed.  There HAS to be a better way...

Hopefully @ich777 can chime in with their thoughts.

Edited January 2 by miicar
oops

[Kilrah]

3 minutes ago, miicar said:

sometimes the data is on BOTH the cache and the array (before mover runs)

Exclusive shares require no secondary storage set and no files present outside of the designated primary storage so that's not possible, the prerequisite for an exclusive share is no data will ever be outside of the primary storage.

Must be a misunderstanding somewhere.

[miicar]

1 minute ago, Kilrah said:

Exclusive shares require no secondary storage set and no files present outside of the designated primary storage so that's not possible, the prerequisite for an exclusive share is no data will ever be outside of the primary storage.

Must be a misunderstanding somewhere.

you are correct i misspoke (i do understand this...just been up for more hours then humans should over the last week, and my brain stalled)...still the issue remains.

[ich777]

3 hours ago, kevco97 said:

What can i do or what should i check?

Have you read the description from the container where it says you have to tick the box Console Mode for schedules?

[ich777]

1 hour ago, miicar said:

you are correct i misspoke (i do understand this...just been up for more hours then humans should over the last week, and my brain stalled)...still the issue remains.

I'm not too sure about this but have you yet tried R/W Slave in the Docker template?

 

The main issue with symlinks is that they don't work correctly in Docker container most of the times if you pass them through from the host because Docker is isolated and usually don't follow symlinks on the host and since Exclusive shares are symlinks like @Kilrah already pointed out it doesn't work that well most of the times.

Hope that makes some kind of sense to you.

[miicar]

33 minutes ago, ich777 said:

I'm not too sure about this but have you yet tried R/W Slave in the Docker template?

 

I will figure out how to do this, give this a try, and report back.

 

33 minutes ago, ich777 said:

The main issue with symlinks is that they don't work correctly in Docker container most of the times if you pass them through from the host because Docker is isolated and usually don't follow symlinks on the host and since Exclusive shares are symlinks like @Kilrah already pointed out it doesn't work that well most of the times.

Hope that makes some kind of sense to you.

 

I understand what you are saying.  So how can we get it to work better?  Is @kilrah, suggestion of mapping each share (and hoping you remember to update the mapping if things change a little bit), the only way?  That seems silly!
When i did give LB "/" mount point (instead of "/mnt/user/"), the simlinks show up, but as shortcuts (?) only and it doesn't actually transfer the data within (i, admittedly, haven't set each share in "include" section of LB, only "exclude"ed my temp folders i don't want backed up; i was hoping to just be able to exclude a couple folders that shouldn't be backed up and the rest is good...but its never that easy is it).  
I'm trying to backup my whole server share set...everything, and i'm trying to keep it simple so i don't have to teach an assistant rocket science.

thanks for your help @ich777.  sorry for the frustration, i hate it when things are not working properly, or need a micky mouse Band-Aid to work.  

Edited January 2 by miicar

[miicar]

10 hours ago, ich777 said:

have you yet tried R/W Slave in the Docker template?

Well that didn't work! 

But to re-visit what you said about dockers not playing well with exclusive shares.  Why is it that all my other dockers can read exclusive shares; including their install folder being "/mnt/user/appdata/(docker name)" not "/mnt/(pool name)/appdata/(docker name)", like we had to do to bypass FUSE, before the update in unraid?  I changed ALL my dockers (and the folders they look at) back to "user", and haven't had a lick of problems?  

(Ironically enough, LB has "preserve symlinks" as a Command Option...)

Not trying to be difficult...just trying to fix it!  

Edited January 3 by miicar
cleaned up bad grammar

[ich777]

8 hours ago, miicar said:

but as shortcuts (?)

Because they are symlinks (hard links) to another directory, so to speak the real disk.

 

8 hours ago, miicar said:

sorry for the frustration, i hate it when things are not working properly, or need a micky mouse Band-Aid to work.

5 hours ago, miicar said:

Not trying to be difficult...just trying to fix it!  

No worries.

 

5 hours ago, miicar said:

including their install folder being "/mnt/user/(docker name)" not "/mnt/appdata/(docker name)"

I think you mean /mnt/user/(docker name) and /mnt/cache/appdata/(docker name) because the path /mnt/appdata/(docker name) doesn't exist usually.

 

5 hours ago, miicar said:

Why is it that all my other dockers can read exclusive shares;

This is a think I can't answer, my container runs pure Debian with TurboVNC, noVNC and luckyBackup on top of it, nothing special.

Do you run luckyBackup as root?

 

However you should be able to do it like @Kilrah mentioned and map all folders individually but I would strongly recommend that you for mount the real disk which is used as exclusive share like /mnt/yourpoolname/

 

To take it a bit further you've said that if you mount it to /mnt/user it is not working, have you yet tried to mount /mnt instead of / to the container, this should also solve your issues...

 

 

TBH I haven't looked that much into exclusive shares, from where are the hard links created and where they point to because I simply don't have any use case for them.

[Kilrah]

7 hours ago, miicar said:

Why is it that all my other dockers can read exclusive shares; including their install folder being "/mnt/user/(docker name)"

Because you're passing a folder that is deeper than the symlink, so it's unraid dealing with the symlink, not docker.

[miicar]

18 minutes ago, Kilrah said:

Because you're passing a folder that is deeper than the symlink, so it's unraid dealing with the symlink, not docker.

so your saying LB should be able to do that too? 

[Kilrah]

As long as you pass it a folder that's deeper than the symlink yes, but while you never clearly said what paths you were mapping that doesn't seem to be what you want.

Edited January 3 by Kilrah

[miicar]

5 minutes ago, Kilrah said:

As long as you pass it a folder that's deeper than the symlink yes, but that doesn't seem to be what you want.

i just want to be able to say "/mnt/users/" and BU all my shares at once!  like krusader does, for example.  I don't want to forget that i moved things around to a different pool, to fit my needs, and miss backups for that share (or worse, it causes LB to crash and not backup at all).  I'm trying to think long term/bigger picture.

i'll keep poking at this till i figure it out!  thanks for the suggestions.

 

Edited January 3 by miicar

[ich777]

6 minutes ago, miicar said:

so your saying LB should be able to do that too? 

Have you read my comment about passing through /mnt and not /, you should really consider doing it that way and that should work just fine.

 

I also think there is some miss-understanding, luckyBackup can't deal with that. Please try to use /mnt

[miicar]

2 minutes ago, ich777 said:

Have you read my comment about passing through /mnt and not /, you should really consider doing it that way and that should work just fine.

 

I also think there is some miss-understanding, luckyBackup can't deal with that. Please try to use /mnt

haha its 3:30am...not doing that tonight.

thanks for your suggestions, i will figure it out!  i don't give up.

(and if that was the simple solution, not sure if it was me who was misunderstanding here)

Edited January 3 by miicar

[ich777]

4 minutes ago, miicar said:

thanks for your suggestions, i will figure it out!  i don't give up.

What Krusader container are you using (repository)? How do the mounts look there, or what specific directory have you mounted?

 

You have to understand that luckyBackup is just an app running inside that container nothing more and the mounts are passed over by Docker, you even can go crazy and open the console from the container and see where the mounts are pointing to and what directories you see from the command line.

[miicar]

1 minute ago, ich777 said:

What Krusader container are you using (repository)? How do the mounts look there, or what specific directory have you mounted?

 

binhex/arch-krusader
HAHA i think you got me!  I just looked the mount for this...and its indeed "/mnt"!  I cant test it in LB atm as i'm doing a disk rebuild and i don't wanna f tings up while im half asleep, but i will give this a shot and report back.  I kinda feel dum now.

 

2 minutes ago, ich777 said:

You have to understand that luckyBackup is just an app running inside that container nothing more and the mounts are passed over by Docker, you even can go crazy and open the console from the container and see where the mounts are pointing to and what directories you see from the command line.

go crazy you say?  i just might!  im one of those weird people who likes to understand the inner workings of things, to understand the broader use...its a blessing and a curse.

[ich777]

15 minutes ago, miicar said:

to understand the broader use

It's a limitation from Docker, Docker is not allowed to follow a bind mount which points to a symlinked directory which is outside the bind mount and that is also good since Docker is an isolation layer, otherwise you would be able to more easily break out that isolation.

[miicar]

aite...so for the most part using "/mnt" as the mount point works to give me access to all my shares (regardless if they are exclusive or not).  It still gives me "/mnt/user/user/" as a source, which is visually annoying, but i can live with it.  Its actually backing up everything now (as i desire).
SO, now that i got the basic functionality working, time to figure out how to add a secure BU only user and make my remote offsite BU connection a bit more safe.  But thats another story, and i am pretty sure i understand how to pull that off. 

Thank you for pointing me in the right direction @ich777.  (Stupidly enough, i vagally remember pulling my hair out with krusader before about this same issue...with the same solution.)
Maybe there should be a note added to the install guide that gives users a clue how to get here??!!??!!

Edited January 3 by miicar
found a solution

[miicar]

On 1/3/2024 at 4:54 PM, miicar said:

It still gives me "/mnt/user/user/" as a source, which is visually annoying, but i can live with it. 

so, i fixed this by changing the Container Path, and the Host Path of the Shares to "/mnt" only.  Now its presenting itself normal looking in the LB gui!  I also changed the access mode to read only for safety.  I don't want to allow myself the ability to accidently select "Synchronize Source and Destination" and destroy my (current) life's work!  Since my backup location is not on the same server this works...if i was using disks on the same server, i would have to turn it back to read write.  That Synchronize option terrifies me tho, but that's a me problem.

 (update: all it did was present the simlinks.  LB didn't actually follow them and back up their contents.)

 

Just an observation:  When choosing "Only Include", and having one folder in the include list, one would expect that folder to be all the LB would look at, but it doesn't.  It seems to scan the whole "/mnt/user/" (source) folder first... 

So to speed things up for single share backups, point the Source directly to the share (either direct or the more preferred "/mnt/user/(share name)" method), not just its its parent folder.

Edited January 5 by miicar
update

[ich777]

8 hours ago, miicar said:

add a secure BU

Don't know what BU is...

 

8 hours ago, miicar said:

so for the most part using "/mnt" as the mount point works to give me access to all my shares (regardless if they are exclusive or not).

This is working.

 

8 hours ago, miicar said:

Maybe there should be a note added to the install guide that gives users a clue how to get here??!!??!!

I'm not sure because usually if you want to back up everything you'll mount /mnt and not /mnt/user only think about Unassigned Devices.

 

8 hours ago, miicar said:

I also changed the access mode to read only for safety. 

That's up to you.

 

8 hours ago, miicar said:

It seems to scan the whole "/mnt/user/" (source) folder first...

That's expected, this would also be the case if you issue find from the terminal or execute rsync since, as said previously luckyBackup uses rsync as the Backend and luckyBackup is just a fancy GUI for rsync.

[miicar]

21 hours ago, ich777 said:

Don't know what BU is...

BU = Backup

 

 

UPDATE:  Well that didn't work...in fact it removed the previous BU contents of a folder i changed BACK to an exclusive share (it left the empty folder tho to trick me)🤪. So back to the drawing board (or find another solution/manually).  The simlinks show up, but LB doesn't follow them.