[6.0-beta6] User Share Copy Bug

OK I understand...I think.

17 minutes ago, jaybee said:

OK I understand...I think.

What exactly were you planning on doing that you suspect may cause a problem?

The "absolutely paranoid and don't want to take ANY chance on losing data" approach to this is to be CERTAIN that the data you plan to move around is BACKED UP on another system

 

But as already noted, as long as you understand how to avoid the issue there shouldn't be any problem with your copies.     If you have ANY doubt that you're doing it right; copy ONE file first (being certain you have a backup of that file on another system) ... and confirm that all worked well.

 

16 hours ago, jonathanm said:

I wouldn't call it a defect

Ok, I have just started to work with 6.x and I was at first confused about why I could not see any disk shares... Found where to enable them after reading up on the "problem" than saw there was a "user share copy bug" I needed to become familiar with.

 

  My take on the whole thing, now I have read this thread, I can see where people could get into big troubles by copying files from Disk to User shares and also in reverse.  It has always been a potential from what I can see, at least back to the 4.x days.  As others, I also ALWAYS sue disk shares to WRITE to my arrays, and often, (always for media players), use the user shares for reading, including in the past, setting the user shares to READ ONLY.  On a server others have access to, I also would set DISK shares to be hidden, but set to full READ/WRITE access.

 

  Now I have my disk shares working again for my 6.5.3 test server, I am looking for a way to have them enabled, but hidden from other users casual browsing... is this possible with 6.X, or did it vanish sometime between 4.x and 6.x?

 

  Also, as some have said, this is NOT a bug really.  It is just a misunderstanding of file system operations.  Just like you want to avoid recursive loops, unless you are cautious with what you are doing, you really should not be playing around too much with virtual directory structures, again without proper thought put into the potential issues that could result.

 

  I think in general the decision to disable disk shares by default was a very good choice.  It will help protect the majority of casual computer users, that are more frequently adopting Unraid as a home server solution.  At the same time, I do not see that Unraid would be very useful to me, or at least fully meet my needs, without the ability to use Disk Shares!

 

  As a side, but related note, I always move files from a networked computer, using the DISK SHARES, (obviously since my user shares are set to read only).  I use a combination of Drag and Drop, Command line, and Batch files.  If moving a file or sets of files to another disk share, I make copies of the files to the new destination first, then I do a full file comparison of the original and new copies.  Then and ONLY then do I delete the original file from the original disk share.  I am a bit paranoid about potential file corruption, so I take the extra steps to insure it does not happen while the files are being moved due to memory, disk, or even communication losses.  If I am moving a file WITHIN a disk share, to another user share that is also on that disk for example, I just do a simple move as it does not do as much to the file in the process, but perhaps I should re-think that one, as some OSes do more than others at that point.

You can set the Disk Shares SMB Export option to “Yes (hidden)” to stop them being visible on the network but still accessible if you know the name.

35 minutes ago, itimpi said:

You can set the Disk Shares SMB Export option to “Yes (hidden)” to stop them being visible on the network but still accessible if you know the name.

Yes!  :-)  Thanks!  Just what I was looking for!  I think that was the last thing I felt I needed before full deployment!

Is copying from disk share (eg /mnt/disk1) to user share (somewhere under /mnt/user) guaranteed to result in data loss?

Would only the copy target be lost, or also the source (data on disk1)?

19 minutes ago, tuxbass said:

Is copying from disk share (eg /mnt/disk1) to user share (somewhere under /mnt/user) guaranteed to result in data loss?

Would only the copy target be lost, or also the source (data on disk1)?

The problem arises from the fact that /mnt/disk and /mnt/user can be different views of the same files.   The effect of the bug is that you end up copying the file to itself which results in it being truncated to zero size (and thus the file contents are completely lost).

What if file is copied from say /mnt/disk1/dir/file to /mnt/user/dir2/dir/file?

39 minutes ago, tuxbass said:

What if file is copied from say /mnt/disk1/dir/file to /mnt/user/dir2/dir/file?

The problem only occurs when you accidentally copy the file to itself.  (e.g something like /mnt/disk1/dir1/file1 to /mnt/user/dir1/file1).   In your example that is not happening so you do not encounter the bug.

I just lost 2 months of files due to this. i just wasn't thinking and made error after error after ERROR!. OMG. I almost did backups first, but i decided to tackle the problem i was having with mover first. Obliterated hundreds of gigs of files on my cache. WTF was I thinking?

 

FML.

"When" will this bug be fixed? Or: Will this ever be fixed?

48 minutes ago, Zonediver said:

"When" will this bug be fixed? Or: Will this ever be fixed?

The question is whether it is even a 'bug' in the strict sense of the word!   The word 'bug' is used in this context to mean that it is not the behaviour the user expects even though the system is behaving as designed.  It is intrinsic to Linux when used with Fuse file systems which Unraid uses to implement User Shares) so barring a major change to Fuse it will probably never get fixed.

 

This is one of the reasons that Disk Shares are disabled by default on Unraid as it reduces the chance of a user encountering it by accident.    However even that does not help when a user is bypassing the User Share system by working directly at the Linux level either via the command line or a docker container.

I am a little confused here
My setup is as follows:
Disks are mapped to my desktop system so I can manage the disks accordingly.

User shares are enabled as well, which only admin can read/write to.

I use these to WRITE so my data is spanned across disk using highwater.

I also use these shares to point plex at.

 

Guests can only READ from /user/ shares
Guest can NOT access /disk/ shares

 

If I stick to writing data via user shares am I safe?
Can I also write data to the disks (mapped drives) that are outside of the /user/ share data scope (i.e. personal files)?

 

I ask this as well because I am reorganizing my shares. 
I am making a new share and MOVING data (media files) from shares to this new share so the new share uses highwater across 4 disks.
Is this safe to do?
Thanks

4 minutes ago, bombz said:

Can I also write data to the disks (mapped drives) that are outside of the /user/ share data scope (i.e. personal files)?

Yes, you just can't copy from a disk share to a user share, if copying from outside the array you can do it to both types of share.

2 minutes ago, johnnie.black said:

Yes, you just can't copy from a disk share to a user share, if copying from outside the array you can do it to both types of share.

OK, so as I have /user/ shares setup that I am deleting after the data has been moved, the best method is (from a windows PC)
access the mapped /disk/ 
Cut data from disk -> paste to /user/share?

I wish i could move this data right on UnRaid itself, however my command line skills are lacking to move data from /disk/ to /user/

1 minute ago, bombz said:

Cut data from disk -> paste to /user/share?

That's exactly what you can't do, move/copy from a disk share to user share.

1 minute ago, johnnie.black said:

That's exactly what you can't do, move/copy from a disk share to user share.

Oh, ok hm
I am really confused then on how to re-organize this using the highwater setting

via windows:
Can I move data from
/user/tv-sd/
/user/tvhd/
to
/user/TV/

any /user/share that exists, only write that data to that share
any data outside the /user/share scope, write to the /disk/ manually?

19 minutes ago, bombz said:

Can I move data from
/user/tv-sd/
/user/tvhd/
to
/user/TV/

Yes, that's user share to user share.

 

19 minutes ago, bombz said:

any /user/share that exists, only write that data to that share
any data outside the /user/share scope, write to the /disk/ manually?

Don't understand what you mean here.

OK.

I have all 11 disk mapped within windows. I use this to manage the disks accordingly. I can hit any disk and see the data

the disk shares are exported (hidden) for admin user only
This gives me the ability to go to:
/mapped drive letter/disk10/photos
or
/mapped driver letter/disk8/personal files

and drop data to the specific disk. Keep in mind these 'folders' are not part of any user share at all

 

Then, I also have user shares with included disk(s):

disk 1 + 2 + 4 + 8 - set to high water

these user shares are generally for media files, or any files added to plex, etc.

 

moving forward,

when I write data (tv/movies/iso/etc.) I should use only the /user/<sharenamehere>

when I write data, outside the above scope (pictures, apps, personal documents, etc.) I should be able to use /mapped driver letter/disk8/personal files

 

Currently I have some user shares that I am going to combine

/user/tv-sd/

/user/tv-hd/

/user/tv-cooking/

 

I have made a new /user/share:
/user/tv/

Right now, I am moving data from the 3 shares to the one share. Once moved, I am eliminating the 3 shares. Basically I am consolidating data and re-organizing my shares.

 

I suppose I want to be sure I am thinking about this the correct way so I am not setting myself up for data loss at all. I do believe I am on the right path, however the UnRaid community are some of the best people to give opinions and provide outlooks at times questions come up.

Thanks :-)   

 

3 minutes ago, bombz said:

and drop data to the specific disk. Keep in mind these 'folders' are not part of any user share at all

All top level folders are user shares.

 

But again what you're doing is not a problem, moving/copying from user share to user share is OK.

1 minute ago, johnnie.black said:

All top level folders are user shares.

 

But again what you're doing is not a problem, moving/copying from user share to user share is OK.

OK wonderful.
When I was using:
/mapped driver letter/disk8/ -> user/tv/ 
you had stated that was a 'neddy-no-no' 

I didn't move much data at that time or media i really didn't care about. Via Windows, I then opened 2x windows 
/user/tv-sd/
&
user/tv/

Did a cut + paste, and data is starting to move.

I really think this would be much faster via command line, however, I need to brush up on my unix skills 😞

1 minute ago, bombz said:

/mapped driver letter/disk8/ -> user/tv/ 
you had stated that was a 'neddy-no-no' 

It is, because that's a disk share, can't do that.