Embedding top-level shares into a "container" share


Crad

Recommended Posts

Hi guys,

 

Bit of a niche use case here. I work with huge files and need to maximize speed so I want as many of the currently active files (files being processes) to be on cache.

So this is my share and folder structure along with where I want each folder to be located.

 

  • Photography (unraid share)
    • Incoming (array)
    • Processing (cache)
    • Review (array)
    • Archive (array)

 

So within the Photography share, I only want the "Processing" folder stored on cache. As far as I can tell the only way to achieve this is to create 2 different shares for and call them, say, "Photography Cache" and "Photography Array" and move the folders around respectively.

This approach would be fine but I really want to be able to access the whole list of folders as one structure. Is there a way to use SMB Extras commands to create a rootshare-like folder so I can access all the shares from a single mapped drive? @SpaceInvaderOne has a great video on something very similar (https://youtu.be/cEeEr89dSXc?t=1532)

 

There may be another way of going about this as what I'm trying to do isn't unique. I'm open to ideas.

 

 

Link to comment

Hmmm, so your asking if its possible to NEST Shares. Honestly I don't know how to do that. I wonder if making some Symbolic links would solve this.

 

Folder/ Photography (unraid share) Direct Folder

-- Folder/ Incoming (array) --> Points to its Share Location  ln -s /mnt/user/Incoming Incoming

-- Folder/ Processing (cache) --> Points to its Share location ln -s /mnt/user/Processing Processing

-- Folder/ Review (array) --> Points to its Share location ln -s /mnt/user/Review Review

-- Folder/ Archive (array) --> Points to its Share location ln -s /mnt/user/Archive Archive

 

https://www.freecodecamp.org/news/symlink-tutorial-in-linux-how-to-create-and-remove-a-symbolic-link/

 

 

 

Link to comment

Yes sir.

 

Be careful and do some tests. I basically took what the website said and modified it to fit what your talking about. You need to make sure they are in the folder where you need them and I'm sure somebody else will pipe in. I don't use them often, but was showing you they aren't to hard to create.  

Link to comment

This is how I would approach it:

  1. Create Photography share with cache settings set to "No",
  2. Manually create a folder on the cache i.e. /mnt/cache/Photography/Processing/
  3. Symlink the cache folder to a folder on one of the array disks, in the Photography folder.

New files added to Processing will land on the cache, everything else will land on array.

Please test this thoroughly, however it works as intended for me.

image.png.accb79dc05f64a5e48ed0e444fa07742.png  image.png.7808b623a53635ed028468fe7c10123c.png

in 6.9 onwards you can utilise multiple cache pools to create the Processing folder on a different cache pool for the same effect, while retaining the primary cache for the rest of the photocraphy share... not tested that method but should work also.

 

Also note also you can easily create symlinks in MC using the ctrl+x + s combo while highlighting the folder you want symlink created for, and the other panel with destination.

Link to comment

Thanks @tjb_altf4!

 

I'm trying what you suggested but I'm guess an error message:

 

 

 

image.png.0430b25e0d28e404e15fee62831bf7ae.png

 

 

I've tried several different ways in case I was doing it in the wrong order but can't seem to get it to work.

 

Also, is there any reason I couldn't link over to /mnt/user0/Test instead of /mnt/disk6/Test?

I guess there's no difference where you symlink to since the files will be on the cache, but just wondering for theoretical / understanding.

 

Oh, also for understanding... what's the process for changing / removing symlinks? Is there anything to be aware of? They kind of confuse my brain.

 

EDIT: I was clicking "CTRL+X+S" all at once. Once I hit CTRL+X and then S, it brought up the dialog.

 

Thanks!

 

Edited by Conrad Allan
Link to comment

So I was able to create a symlink eventually and the folder structure appears in windows but when I try to copy to it I get an error:

 

image.png.53744bd1b95db1b94c6be33a0e39676d.png

 

I also get an error if I try to create a folder within that folder using krusader:

 

image.png.7edefe7a4b9c48e6d4d3b25b89f756dc.png

 

For reference, here are a couple of screenshots from MC of what I did and what the structure looks like.

 

image.png.69718e4f2fbf5045f3bfc131dad01207.png

image.png.7b84be5731239423c7560db366e6e433.png

 

 

Any idea why I can't write anything to the symlink folder?

Thanks!

 

 

 

Edited by Conrad Allan
Link to comment
48 minutes ago, Conrad Allan said:

Thanks @tjb_altf4!

 

I'm trying what you suggested but I'm guess an error message:

 

image.png.0430b25e0d28e404e15fee62831bf7ae.png

 

 

I've tried several different ways in case I was doing it in the wrong order but can't seem to get it to work.

The keystrokes you need to do are hold Ctrl, press X, release Ctrl, press S

Holding Ctrl for X+S is a symlink edit shortcut, hence the error.

 

You should also remove the Processing folder on disk6

 

50 minutes ago, Conrad Allan said:

Also, is there any reason I couldn't link over to /mnt/user0/Test instead of /mnt/disk6/Test?

I guess there's no difference where you symlink to since the files will be on the cache, but just wondering for theoretical / understanding.

You will get a symlink already exists error,  as it the content will already be present.

Remember the cache folder you created will exist in the user share as unraid still looks at the cache, its only the mover and initial file creation for the share that is ignoring that folder.

 

1 hour ago, Conrad Allan said:

Oh, also for understanding... what's the process for changing / removing symlinks? Is there anything to be aware of? They kind of confuse my brain.

In MC, just delete the symlink (not the real folder!). 

You can then do a move to the array if needed, or simply change you cache preferences and let mover take care of it

Link to comment
1 hour ago, Conrad Allan said:

Thanks @tjb_altf4 I did eventually figure out the MC issues.

 

Did you see my problem with writing to the symlink folder?

A couple things to try:

  • make sure your user access is setup correctly for the new share, i.e. can you create a folder in the other parts of the share
  • try deleting the symlink and recreating

See if that fixes things

 

Link to comment

Okay I'm lost for options... but found some strange behavior that might guide us through.

 

Nothing I do to a new share will make it work unless I set it to "Yes" for 'use cache'. Then I'm able to write to the folder but I'm pretty sure this isn't using the symlink and is just writing directly to the folder on the cache drive.

 

Other things I've tried

  • Running new permissions on the share in case there's something funky going on there.
  • Restarting Unraid after creating a share and then creating the symlink on that share.

 

Any more thoughts? :(

Edited by Conrad Allan
Link to comment

Aha! Found out the issue which was blocking me from writing to the folder.

 

When creating my work shares I exclude disk1-3 but disk3 was the active disk Unraid wanted to write to because of the high-water setting. So once I just allowed the share to write to all disks, I was then able to write to the symlink folder.

 

However… you might have picked up from the paragraph above that the symlink is now just causing the file to appear on disk3 (I created the symlink using MC and created it on disk6 but Unraid wants to use Disk3). So what is actually happening is the file is simply being added to disk3 and not to the cache drive.

To further test this, I cleared everything and created the symlink on disk3 instead and once again I'm now blocked and cannot write to the folder.

 

My suspicion is that Unraid's high-water setting tells the share to look at Disk3 and if it them finds the symlink, it tries to write to Cache but the share is set to not use cache and it's then blocked. Furthermore, it appears as though trying to write a file to a symlink folder in a share, consequently ignores the "exclude disks" settings in the Unraid share.

 

I can't be sure, but maybe you could test once more on your end?

 

Edited by Conrad Allan
Link to comment
1 hour ago, Conrad Allan said:

Hey @tjb_altf4, did you have any further thoughts on what might be going on for me? Did you try the same setup on your side? I'm wondering if this is actually "desired behaviour" from Unraid rather than a bug on my end.

I tested it quite a bit before posting to double check it worked as intended.

Only thing I can think you should try is creating the symlink on disk 1, to ensure its the first disk it tries to write to.

 

Mine was created on disk2, which was the first disk that participated in the share (for writing)

Link to comment

I'm at a complete loss...

 

disk3 is the first disk the share writes to because of the high-water setting. So I tried creating the symlink from /cache to/disk3 but when I do that I get the error in windows.

 

image.png.ceb129b33085e24b5d22a5eb0cf5ea70.png

 

If I create the symlink on any other disk, it just creates the folder again on disk3 since that's the first disk it's checking for the write.

 

Link to comment
33 minutes ago, Conrad Allan said:

I'm at a complete loss...

 

disk3 is the first disk the share writes to because of the high-water setting. So I tried creating the symlink from /cache to/disk3 but when I do that I get the error in windows.

 

image.png.ceb129b33085e24b5d22a5eb0cf5ea70.png

 

If I create the symlink on any other disk, it just creates the folder again on disk3 since that's the first disk it's checking for the write.

 

Can you screenshot your (Photography) share settings

Edited by tjb_altf4
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.