I have a fairly large array mainly used as media storage.

 

I am considering to use user shares, but not sure whether they will fit my purpose.

 

Let me illustrate with the example of TV shows my folder structure:

 

Disk1/TV/Show1/Season1/Episode1

Disk1/TV/Show2/Season 1/Episode1

(...)

Disk2/TV/Show3/Season1/Episode1

Disk2/TV/Show4/Season1/Episode1

(...)

 

If I were to create a user share over disk 1 and 2 with folder /TV, I would have a very nice share including all TV shares. However, I think there would still be an issue. I am using Sonarr and Sonarr would start copying part of show1 on disk1 and part of disk2. Also, metadata may be spread across the two disks. This technically shouldn't bother me as the user share displays both together. However, I would like to keep the disks "autonomous" and not spreads episodes of the same show over two disks.

Any thoughts whether what I am trying to accomplish is possible?

User shares contain a configuration called split level. This forces files within the same directory to be kept on the same physical disk. You just say how deep. If you set to 3, all directories that are three levels deep or deeper will be kept together, while directories less than that can be split. Using this feature you can control whether whole shows will remain on the same physical disk, or seasons of a show must remain on the same disk.

 

If you go into the user share configuration and turn on help, you will see a more precise explanation.

 

As one who did not use user shares for many years, I'll say that it is well worth your time to understand how they work and use them!

Wow, this is great. Wasn't aware that this is possible. Assume user shares also make things easier with Sonarr/Radarr as I only have one user share to "point to"?

 

For my example above, I would set the "depth" to "1" or "0"?

Go intro configuration screen and read the help text. It's easy to be off by one. I think you'd want it set to 1 to allow different show to be on different disks, and 2 to allow different seasons to be on different disks.

And related question. Assuming I set the "depth" to "1". How does Unraid deal with the situation when one disk is full and I cannot add to this folder anymore as this would exceed disk space. Would Unraid then move other folders to another disk that I can add more to this folder?

Make sure you saw answer to other question above.

 

The answer to this is "no". If you are keeping shows on the same disk, and that disk runs out of space, you'd start getting out of space errors. You'd have to, using your disk shares, move a show off of the full disk to make room for shows on that disk to have room for new episodes. Just keeping seasons together requires less of this type of maintenance than trying to keep whole shows together.

 

Note the split level, allocation method, and min free settings all come into play to get your user shares tuned. 

Got it, thanks. This worries me a bit. I understand there is an easy way to corrupt my files by copying from a disk to a user share.

 

If I read you correctly, I would not corrupt anything if I copy something from a disk to another disk share (even if both are part of a user share)?

 

So, this use case would require me to set "enable disk share" to "yes". I may kill my whole media collection by a wrong copy interaction. Or doing a mistake setting up Sonarr/Radarr correctly?

 

Reading the instruction felt that setting "disk share" to "auto" would be the safest. But from what you say, this would give me the issue of running out of space?

To switch over, you'd want to change all references to disk shares from your configurations. You could disable or smart disable disk shares, at least initially, to prevent accidental / forgotten configurations from hitting the disk.

 

The user share copy bug only bites when trying to copy the exact same file from a disk to user share (or vice versa). The OS doesn't realize they are the same file, and once it starts writing the new file, the source file is gone. It's not corruption, its data loss. If you try to copy a file on top of itself in Windows or other OSes, you'd get an error saying you can't. With mixed user/disk shares, the OS doesn't realize what you are doing and can't protect you.

 

Its easy to avoid once you know. The simple rule don't copy between disk and user shares is overly broad but keeps you out of trouble. But if you dled a new file to a disk share that was part of a user share, there would be no problem.

Thanks for your help, this is very helpful.

 

I believe I understand what I'd need to avoid to prevent data loss.

 

To be on the safe side, I may still disable disk shares for now.

 

If I understand you correctly: in case I run out of disk space, it is very easy to just disable user share and then copy & paste between disk shares to free up space. And then enable user shares again. This sounds good and safe enough for me. Let me give it a try.

You shouldn't need to disable the user shares. Just enable the disk shares and move files between your disk shares. 

I know. Just thought to be even more on the safe side. You never know what copy activity is happening and then suddenly all is wiped...

1 hour ago, steve1977 said:

Or doing a mistake setting up Sonarr/Radarr correctly?

Extremely unlikely you would ever do this, even accidentally.  You would have to be going out of your way to try and force the user share copy issue (it's not a bug) on Sonarr/Radarr

 

To clarify what @SSD said

42 minutes ago, SSD said:

copy the exact same file from a disk to user share (or vice versa).

what he means is copying from /mnt/disk1/Movies/moviefile.mkv to /mnt/user/Movies/moviefile.mkv

 

Ultimately, many many people mix user shares and disk shares all the time when utilizing downloaders.  But the copy that happens will be /mnt/cache/Downloads/moviefile.mkv to /mnt/user/Movies/moviefile.mkv and there is no problem ever with doing that as the share name is different.

Edited November 25, 20178 yr by Squid

7 minutes ago, Squid said:

user share copy issue (it's not a bug)

 yes it is.  

 

If an Windows or Linux had this "issue", it would absolutely be a bug.

 

Understand it is hard to fix, and benefits far outweigh the risks, but I still call it a bug.

Thanks guys.

 

I understand what the issue/"bug" is. I understand it is rare to happen, but knowing me, it is not unlikely...

 

I am about to finish my setup. Are you sure that I should set the split level to "1" (top-level) instead of "manual". "Manual" sounds what I am looking for?

And related question. Can I have a disk included in the user share, but not write additional files to it? I want the content of diskX/movies to be part of the user share movies, but I don't want to add additional movies to diskX. Is this what "exclude disk" does?

 

And on related note, I am surprised "include disk" and "exclude disk" both exist? If I exclude one disk, but all disks are included: who "wins"?

9 minutes ago, SSD said:

yes it is.

Going to disagree (although I have to admit I always do want to type bug instead of issue), but I'm coming from a different perspective than you.  And I'm sure we would both agree that thinking about what's actually going on it makes sense that the corruption happens.

5 minutes ago, steve1977 said:

And on related note, I am surprised "include disk" and "exclude disk" both exist? If I exclude one disk, but all disks are included: who "wins"?

Sometimes its easier to exclude 1 disk rather than include all but one disk.  As to which one winds in case of a conflict, there are apparently rules in place but the rules have never been published publicly.  But you're correct - use one or the other and you'll never have a conflict.

7 minutes ago, steve1977 said:

Can I have a disk included in the user share, but not write additional files to it? I want the content of diskX/movies to be part of the user share movies, but I don't want to add additional movies to diskX. Is this what "exclude disk" does?

When reading the contents of a share, it doesn't matter if the disk is included / excluded.  All files on all disks are part of the share.  Included / Excluded prevent new files from being writting to that disk

26 minutes ago, Squid said:

All files on all disks are part of the share.  Included / Excluded prevent new files from being writting to that disk

 

This is pretty confusing, but actually works pretty nicely most of the time. But not understanding it is what really gives the user user share copy bug teeth!

 

Let's say that disk1, disk2, and disk3 all have the top level "TV" folder. And that disk1 contains all seasons of "Show X".

 

But inside the TV user share configuration, only disks 2 and 3 are included. (BTW, you should either include or exclude, don't try to do both. I normally include, using certain disks for certain user shares. Call me OCD.).

 

You look at the TV user share. Is "Show X" there?

 

Yes

 

A new episode comes to the user share for "Show X". Who gets the file?

 

The answer is disk1, assuming split level drives it there.

 

Is there any way to stop this and really and truly get disk1 OUT of the TV user share.

 

Yes - there is a setting to globally exclude a disk from the whole user share feature. If you excluded disk1, it would really be excluded from this and any other user shares.

 

So why does this relate to the user share copy bug. A user wanted to remove a disk from a user share. So he went into the user share configuration and excluded the disk, thinking that would exclude the disk from the user share, which we just learned it does not. But thinking it was removed, they now went about copying all of the files from the disk share to the user share, thinking that the user share would then find a new home for them and they'd be off of the disk share. But as we just learned, this would just cause the files from the disk share to be copied directly on top of themselves - instant data loss. The theory is sound, but you'd need to instead completely remove the disk from the user shares to do it this way.

 

But there is a far easier way. First remove the disk from the TV user share in the user share configuration. For example, call it "TVX" vs "TV". This will, BTW, automatically create a user share call TVX. Now you can move everything from the TVX user share to the TV user share. When done, you can remove the now empty TVX folder and the user share will disappear. Done!

 

So why have these include/exclude settings in the user shares if they are basically ignored?

 

Because they control what happens if the split level does not force a file to a particular disk. Let's say you are using "most free", and creating a new TV show folder - "Show Y". And let's just say disk1 has the most free space, and disk3 has the second most. If disk1 is not included in the use share config, disk3 would get the new show.

1 minute ago, SSD said:

user share copy bug

issue 

 

As to everything else, you're correct but I did have to read it twice as all the fine-tuning unRaid allows can really mess with your head sometimes.  

Thanks guys. I have set it up now. Still struggling though:

 

1) I have disk shares mapped within my Win10 VM. Can change the folder name on sub-directory. I cannot change the name of the movie (permission issue). Any thoughts what is driving this permission issue?

 

2) Any thoughts whether "manual" as split-level is actually the best thing for me in stead of top-level? This way, tvshows don't split across disks.

 

3) Why does Unraid still write to disk1 if I exclude disk1 from the local user share?

3 minutes ago, steve1977 said:

I cannot change the name of the movie (permission issue)

Since you had previously mentioned Sonarr / Radarr, did either of those originally create the file?  In the settings for both of those there's an option to set the permissions on new files.  Set it to be 0777.  Old already written files you can fix via Tools - Docker Safe New Permissions or Tools - New Permissions (only run this against the share having problems)

 

6 minutes ago, steve1977 said:

2) Any thoughts whether "manual" as split-level is actually the best thing for me in stead of top-level? This way, tvshows don't split across disks.

TBH, split levels always confuse me so I don't use them ever.  But, that also corresponds with my theory that *if* I ever suffer data loss from say having 3 disks drop dead simultaneously I'd rather lose some of the episodes of a tv show stored on those disks than all episodes of the tv show if it was confined to those disks.

14 minutes ago, steve1977 said:

2) Any thoughts whether "manual" as split-level is actually the best thing for me in stead of top-level? This way, tvshows don't split across disks.

 

See here for some split level examples:

 

https://forums.lime-technology.com/topic/59589-solved-v635-advice-please-on-shares-settings/?do=findComment&comment=584558

 

Thanks. What is the difference between "manual" and "split only top-level". I followed your link and not fully clear to me from there.

18 minutes ago, steve1977 said:

Thanks. What is the difference between "manual" and "split only top-level". I followed your link and not fully clear to me from there.

Manual will split only to disks where that folder (or subfolder) already exists.

41 minutes ago, Squid said:

As to everything else, you're correct but I did have to read it twice as all the fine-tuning unRaid allows can really mess with your head sometimes.  

 

No wonder you were confused. I reread it and realized I omitted an important sentence. I added it in bold to the post above.

 

I think they're a little tricky to get set up at first, but then I almost never need to muck with them. But when I do I have to re-figure it out all over again. 

 

Manual split level must be new. I'll have to look into that!