SSD

This would definitely cause the problem you are seeing. Double check this setting. If you are deeper than the split level, unRAID knows which disk to pick and it will ignore the user share settings like min free even if disk has little or no space available.

I had another idea to partially mitigate this issue. When a drive is removed from a user share, unRAID could automatically rename its user share folder (e.g., append "_X", so "Movies" would become "Movies_X"). unRAID will then create a user share for that new name, making it easy to then copy files from the removed share to the real share. This would mitigate a common (probably the most common) condition giving rise to the bug.

I doubt it would disrupt parity but certainly no harm doing a parity check.. You might also check the filesystem to make sure this didn't confuse RFS.

Thanks Tom. This is the type of analysis that was needed. It is disappointing, but I understand you are using a combination of your logic and FUSE to put the user share feature together, and that some things are possible and others are not. The last thing you mention (user shares look like symlinks) sounds promising, and I'm hopeful that will be a solution. A few suggestions related to the user share solution: 1 - When enabling user shares, pop up a one time informational message explaining the user share feature and this potential gotcha. Users should be warned that copying files from disk shares to user shares, and user shares to disk shares (i think the same thing could happen) should be avoided. 2 - Add add'l info on the user share configuration screen that makes the feature and options more clear. Include a read-only field that indicates what disks are REALLY in the share (for read purposes), and clearly label "include" and "exclude" fields only limit writes of new files. 3 - If a user excludes a disk that is part of the user share, pop up a warning that in order to truly exclude a disk, you would need to rename its root level folder. I am not even sure if that is enough. Would you have to bounce the array afterwards?

Well stated. So functionally it's even worse than copying a file from a folder to the same folder, since neither the copying utility nor the OS natively recognizes that this is happening. It'd be nice if this actually resolves the issue. But I DO think the reality is it's a fairly rare occurrence that can be "resolved" through education and, as Brian suggested, some clear warnings on the User Share page about the possible data loss if copying from a disk share to the parent user share that includes it. You may want to include an option to make a share "Read Only" -- with a note outlining how this would eliminate the possibility of this issue occurring. Clearly there are downsides to that ... but for those who write their new content to disk shares, and don't use any utilities that require write access to the user shares, this would eliminate any possibility of encountering this problem. I don't think this read only approach would work. If the share is read only, and you copy a file from the share to the disk it is sourced from, I still think its get clobbered. tell me if I'm wrong.

I agree philosophically with that statement. But I also think no one would think it's reasonable to copy file from a folder onto itself (i.e. the source and destination are identical). That, in essence, is what's happening when these copies are done ... except that it's not obvious because the references are different [i.e. \\Tower\disk1\MyShare\MyFile and \\Tower\MyShare\MyFile ]. It may be what is happening TECHNICALLY, but the physical implementation of the feature is not something the user could reasonably be expected to understand. The following is not at all clear (IMO) about user shares. This is not explained anywhere that I know of, and most users do not understand #1 and #3: 1 - ALL disks that have a root directory with the name of the user share are included in the user share, even if explicitly excluded or left off of the include list. 2 - If a new file is copied to a user share, it will be written to one of the included (or not excluded disks) following the allocation method and split level setting. 3 - If a file is written to a user share with the same path and filename as already exists on the user share, it will OVERWRITE that file, even if it is on an excluded (or not included) disk. I am not adverse to appending something when overwriting a file, if you can eliminate some of the weird scenarios (i.e., Windows prompts whether to overwrite or rename, user clicks overwrite, and unRAID renames it anyway). There is no Samba fix in the world that will fix this.

I agree that you cannot protect a user against user error. But copying data - no one would expect an attempt to copy data would delete the source. I do not believe this falls under the category of user error. Agree with all but the "no full solution in the next release." I don't expect a full solution in the next BETA, but do expect a full solution in the next final release. But the release notes for the next beta should include a warning.

Agree with point #1. Point #2 - Knowing it is coming from a disk share and is going back to the same directory as the source would make this about the same level of difficulty as some of the options I listed. Not opposed to this though. But I am curious of the interplay with Windows. If you copied a file from Windows to a user share, it would prompt you whether to overwrite it or rename it. If you selected overwrite, but unRAID always renamed it, it would be confusing.

I disagree with this from so many dimensions. 1 - Fixing the issue for over the network access comes with negatives for existing users. And it doesn't solve the problem. It is useless. Writes occur to user shares in my environment from the media boxes, but they provide a useful function to track resume positions. Having my user share be read only when my disk shares are available would not work for me. 2 - This is a bug. The comment "it clearly requires some fairly extensive logic" is something you could not possibly know and I think is BS. I have programmed for most of my career. This is a special case that needs to be coded for. My guess is that it would require Tom crawl into code he hasn't touched in a while. It is a PITA. I get it. But it is a bug and needs to be fixed the right way, and not with a poorly fitting band-aid that only covers part of the wound and leaves user data exposed ON PURPOSE. If its not fixed completely then don't do any code changes. Clearly document and present the risk to all users as they upgrade or enable user shares, along with options and workarounds (like disabling disk shares). I would be disappointed if this is the solution, but less disappointed than a half-assed fix. I was about to post when this popped up. What is an "intelligent user"? There is only ONE way that a user would know to avoid this issue - reading MY posts on the forum. An unknowing "intelligent user" would go into user shares, remove a disk from the share, and then copy data from that disk share to the user share. Perfectly logical. Its what any of us would reasonably do. And we'd lose all our copied data. (I could easily imagine a customer doing this, losing his kids pictures, finding out LimeTech knew this and didn't warn him, and going after LimeTech for the data loss.) It is not intelligence that is needed to avoid the issue! I suggest we let Tom chime in. Our banter is useless without his involvement.

I think we should keep these defect report discussions narrowly focused on the issue at hand. To summarize so far ... 1. Tom's initial suggestion of removing disk shares for disks involved in user shares (which is every disk) was not acceptable because: - it addresses the issue only for users accessing the user over the network, not for users accessing the server directly - many users, including most of the moderators, use their servers in ways that this change would interfere with their normal usage patterns. variations (e.g., making user shares read only) may be viable although restrictive (for example, my media players write resume files to the disks, and making it read only would nullify that feature). But overall, this whole line of solution reasoning really fails as it doesn't address the non-network copying. 2. Detecting an overwriting of the same inode may not be technically possible due to the obfuscation that happens within the user shares 3. An option of detecting an attempt to copy a file from a disk share to a user share and omitting the disk share from the possible destination has been suggested and hopefully Tom will comment on whether that or something similar is possible.

Actually, I do not believe that this fixes the problem. A user can still copy files using midnight commander or mv/cp on the server itself. in fact, I believe this is the the scenario in which data has been lost.

Your method would always cause all of the disk mounts to be unshared, right? Because every directory is a user share, every disk (except an empty one maybe) would be in at least one user share. There are lots of other reasons to want to access the disk shares even if user shares are turned on. Some users backup their servers to a backup server a disk at a time. I am working on some scripts to maintain MD5s on a disk by disk basis so I could do an integrity check if evern a disk needed to be repaired. I would be very against trying to prohibit export of disk shares if user shares are turned on. I think it would cause some of the more advanced users to be upset. "Know what they are doing" requires knowledge of the exposure. It would take some very good warnings to make people, even very strong Linux people, understand the risk here. And I believe they would expect it to be fixed, not just warning them. And mv/cp are not exactly complex Linux commands. I use them all of the time. Here are a couple of options that come to mind ... 1 - Compare the inode of the source file with the inode of the selected destination file, and if they match fail the copy. 2 - If the copy/move is coming from a server disk, exclude that disk from the possible destination list. Sort of like a one-time exclude. So if copying from /mnt/disk2/Movies/batman.iso to /mnt/user/Movies, and the Movies user share included disk1, disk2, and disk3, exclude disk2 from the possible destinations. So the file would be copied to either disk1 or disk3 - duplicating the file. When you think about it, this is what the user wants to have happen in this situation.

WeeboTech, for example If the inode approach won't work, how about this one? Not quite as simple as what I laid out, though. Copying over top of an existing file bypasses the "select a disk" logic. Logic would have to be added to bypass this "always overwrite" model.

I don't like this solution. Many users that use user shares use them only for reading, and manage their writes to the array by manually copying data to their disk of choice rather than using the various allocation methods and split levels. This is what I do.