Crad Posted January 5, 2021 Share Posted January 5, 2021 Hi guys, Bit of a niche use case here. I work with huge files and need to maximize speed so I want as many of the currently active files (files being processes) to be on cache. So this is my share and folder structure along with where I want each folder to be located. Photography (unraid share) Incoming (array) Processing (cache) Review (array) Archive (array) So within the Photography share, I only want the "Processing" folder stored on cache. As far as I can tell the only way to achieve this is to create 2 different shares for and call them, say, "Photography Cache" and "Photography Array" and move the folders around respectively. This approach would be fine but I really want to be able to access the whole list of folders as one structure. Is there a way to use SMB Extras commands to create a rootshare-like folder so I can access all the shares from a single mapped drive? @SpaceInvaderOne has a great video on something very similar (https://youtu.be/cEeEr89dSXc?t=1532) There may be another way of going about this as what I'm trying to do isn't unique. I'm open to ideas. Quote Link to comment
kizer Posted January 5, 2021 Share Posted January 5, 2021 Hmmm, so your asking if its possible to NEST Shares. Honestly I don't know how to do that. I wonder if making some Symbolic links would solve this. Folder/ Photography (unraid share) Direct Folder -- Folder/ Incoming (array) --> Points to its Share Location ln -s /mnt/user/Incoming Incoming -- Folder/ Processing (cache) --> Points to its Share location ln -s /mnt/user/Processing Processing -- Folder/ Review (array) --> Points to its Share location ln -s /mnt/user/Review Review -- Folder/ Archive (array) --> Points to its Share location ln -s /mnt/user/Archive Archive https://www.freecodecamp.org/news/symlink-tutorial-in-linux-how-to-create-and-remove-a-symbolic-link/ Quote Link to comment
Crad Posted January 5, 2021 Author Share Posted January 5, 2021 Thanks @kizer, I've used symlinks a little but was hoping for a solution more native to the unraid UI. However, if there's no other options I will definitely dive into this and give it a go! Quote Link to comment
kizer Posted January 5, 2021 Share Posted January 5, 2021 I was updating my post above when you replied. So it might explain a little more. I've been using unraid since 2009 and honestly I've never seen Nested Shares, but then again things have been evolving a lot lately. Lol Quote Link to comment
Crad Posted January 5, 2021 Author Share Posted January 5, 2021 Oh thanks! is the red text an example of the command to create the symlink? I assume I do that in the unraid terminal? Quote Link to comment
kizer Posted January 5, 2021 Share Posted January 5, 2021 Yes sir. Be careful and do some tests. I basically took what the website said and modified it to fit what your talking about. You need to make sure they are in the folder where you need them and I'm sure somebody else will pipe in. I don't use them often, but was showing you they aren't to hard to create. Quote Link to comment
Crad Posted January 5, 2021 Author Share Posted January 5, 2021 Fantastic, thanks again. Once I get a moment this week I'll try creating some dummy shares and see how I go. Hopefully in the meantime, if there's any other solutions, someone else will pipe in! Quote Link to comment
tjb_altf4 Posted January 6, 2021 Share Posted January 6, 2021 This is how I would approach it: Create Photography share with cache settings set to "No", Manually create a folder on the cache i.e. /mnt/cache/Photography/Processing/ Symlink the cache folder to a folder on one of the array disks, in the Photography folder. New files added to Processing will land on the cache, everything else will land on array. Please test this thoroughly, however it works as intended for me. in 6.9 onwards you can utilise multiple cache pools to create the Processing folder on a different cache pool for the same effect, while retaining the primary cache for the rest of the photocraphy share... not tested that method but should work also. Also note also you can easily create symlinks in MC using the ctrl+x + s combo while highlighting the folder you want symlink created for, and the other panel with destination. Quote Link to comment
Crad Posted January 6, 2021 Author Share Posted January 6, 2021 (edited) Thanks @tjb_altf4! I'm trying what you suggested but I'm guess an error message: I've tried several different ways in case I was doing it in the wrong order but can't seem to get it to work. Also, is there any reason I couldn't link over to /mnt/user0/Test instead of /mnt/disk6/Test? I guess there's no difference where you symlink to since the files will be on the cache, but just wondering for theoretical / understanding. Oh, also for understanding... what's the process for changing / removing symlinks? Is there anything to be aware of? They kind of confuse my brain. EDIT: I was clicking "CTRL+X+S" all at once. Once I hit CTRL+X and then S, it brought up the dialog. Thanks! Edited January 6, 2021 by Conrad Allan Quote Link to comment
Crad Posted January 6, 2021 Author Share Posted January 6, 2021 (edited) So I was able to create a symlink eventually and the folder structure appears in windows but when I try to copy to it I get an error: I also get an error if I try to create a folder within that folder using krusader: For reference, here are a couple of screenshots from MC of what I did and what the structure looks like. Any idea why I can't write anything to the symlink folder? Thanks! Edited January 6, 2021 by Conrad Allan Quote Link to comment
tjb_altf4 Posted January 6, 2021 Share Posted January 6, 2021 48 minutes ago, Conrad Allan said: Thanks @tjb_altf4! I'm trying what you suggested but I'm guess an error message: I've tried several different ways in case I was doing it in the wrong order but can't seem to get it to work. The keystrokes you need to do are hold Ctrl, press X, release Ctrl, press S Holding Ctrl for X+S is a symlink edit shortcut, hence the error. You should also remove the Processing folder on disk6 50 minutes ago, Conrad Allan said: Also, is there any reason I couldn't link over to /mnt/user0/Test instead of /mnt/disk6/Test? I guess there's no difference where you symlink to since the files will be on the cache, but just wondering for theoretical / understanding. You will get a symlink already exists error, as it the content will already be present. Remember the cache folder you created will exist in the user share as unraid still looks at the cache, its only the mover and initial file creation for the share that is ignoring that folder. 1 hour ago, Conrad Allan said: Oh, also for understanding... what's the process for changing / removing symlinks? Is there anything to be aware of? They kind of confuse my brain. In MC, just delete the symlink (not the real folder!). You can then do a move to the array if needed, or simply change you cache preferences and let mover take care of it Quote Link to comment
Crad Posted January 6, 2021 Author Share Posted January 6, 2021 Thanks @tjb_altf4 I did eventually figure out the MC issues. Did you see my problem with writing to the symlink folder? Quote Link to comment
tjb_altf4 Posted January 6, 2021 Share Posted January 6, 2021 1 hour ago, Conrad Allan said: Thanks @tjb_altf4 I did eventually figure out the MC issues. Did you see my problem with writing to the symlink folder? A couple things to try: make sure your user access is setup correctly for the new share, i.e. can you create a folder in the other parts of the share try deleting the symlink and recreating See if that fixes things Quote Link to comment
Crad Posted January 6, 2021 Author Share Posted January 6, 2021 Gave that a go, unfortunately no dice. I can create files and folders on the rest of the share, but I can't do anything within the symlink even after recreating (I created a new folder and symlinked that just in case). Any other thoughts? Quote Link to comment
Crad Posted January 6, 2021 Author Share Posted January 6, 2021 (edited) Okay I'm lost for options... but found some strange behavior that might guide us through. Nothing I do to a new share will make it work unless I set it to "Yes" for 'use cache'. Then I'm able to write to the folder but I'm pretty sure this isn't using the symlink and is just writing directly to the folder on the cache drive. Other things I've tried Running new permissions on the share in case there's something funky going on there. Restarting Unraid after creating a share and then creating the symlink on that share. Any more thoughts? Edited January 6, 2021 by Conrad Allan Quote Link to comment
Crad Posted January 6, 2021 Author Share Posted January 6, 2021 (edited) Aha! Found out the issue which was blocking me from writing to the folder. When creating my work shares I exclude disk1-3 but disk3 was the active disk Unraid wanted to write to because of the high-water setting. So once I just allowed the share to write to all disks, I was then able to write to the symlink folder. However… you might have picked up from the paragraph above that the symlink is now just causing the file to appear on disk3 (I created the symlink using MC and created it on disk6 but Unraid wants to use Disk3). So what is actually happening is the file is simply being added to disk3 and not to the cache drive. To further test this, I cleared everything and created the symlink on disk3 instead and once again I'm now blocked and cannot write to the folder. My suspicion is that Unraid's high-water setting tells the share to look at Disk3 and if it them finds the symlink, it tries to write to Cache but the share is set to not use cache and it's then blocked. Furthermore, it appears as though trying to write a file to a symlink folder in a share, consequently ignores the "exclude disks" settings in the Unraid share. I can't be sure, but maybe you could test once more on your end? Edited January 6, 2021 by Conrad Allan Quote Link to comment
Crad Posted January 10, 2021 Author Share Posted January 10, 2021 Hey @tjb_altf4, did you have any further thoughts on what might be going on for me? Did you try the same setup on your side? I'm wondering if this is actually "desired behaviour" from Unraid rather than a bug on my end. Quote Link to comment
tjb_altf4 Posted January 11, 2021 Share Posted January 11, 2021 1 hour ago, Conrad Allan said: Hey @tjb_altf4, did you have any further thoughts on what might be going on for me? Did you try the same setup on your side? I'm wondering if this is actually "desired behaviour" from Unraid rather than a bug on my end. I tested it quite a bit before posting to double check it worked as intended. Only thing I can think you should try is creating the symlink on disk 1, to ensure its the first disk it tries to write to. Mine was created on disk2, which was the first disk that participated in the share (for writing) Quote Link to comment
Crad Posted January 11, 2021 Author Share Posted January 11, 2021 I'm at a complete loss... disk3 is the first disk the share writes to because of the high-water setting. So I tried creating the symlink from /cache to/disk3 but when I do that I get the error in windows. If I create the symlink on any other disk, it just creates the folder again on disk3 since that's the first disk it's checking for the write. Quote Link to comment
tjb_altf4 Posted January 11, 2021 Share Posted January 11, 2021 (edited) 33 minutes ago, Conrad Allan said: I'm at a complete loss... disk3 is the first disk the share writes to because of the high-water setting. So I tried creating the symlink from /cache to/disk3 but when I do that I get the error in windows. If I create the symlink on any other disk, it just creates the folder again on disk3 since that's the first disk it's checking for the write. Can you screenshot your (Photography) share settings Edited January 11, 2021 by tjb_altf4 Quote Link to comment
Crad Posted January 11, 2021 Author Share Posted January 11, 2021 Sure, here they are. This is the test share I created which I've been doing all the testing on. Quote Link to comment
tjb_altf4 Posted January 11, 2021 Share Posted January 11, 2021 Try resetting the minimum free space (set to 0KB), that is the only difference I can see to my own config that works as you would expect. Quote Link to comment
Crad Posted January 11, 2021 Author Share Posted January 11, 2021 Still nothing. I even tried recreating the share just in case but still didn't work. Should I contact LimeTech support about this potentially? I'm at a loss Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.