nuticles Posted February 8, 2018 Share Posted February 8, 2018 First of all I want to thank the community a ton. I have found every answer I have needed for the past year on the forums without needing to post once! Until now.... A little background for everyone, I have successfully migrated from old hardware (i5 3570 / 16 GB / 5 x 2 TB HGST drives) on to my new box (2x Xeon 2650s, 64 GB 5 x 8 TB HGST Drives). I was able to move my appdata, get all of my dockers over to the new box as well as migrate from Crashplan Home to Pro. I credit these forums for everything from build, docker migration and finally data migration all being flawless. The good news, I have multiple copies of my data 3...2...1... backup scheme including an up-to-date synch between the old (i5) and new (xeon) boxes. So all is not lost. The bad news. I somehow nailed all of my data. A big part of the problem is I get addicted to these projects and stay up way too late and am not fresh, regardless. Here is what I did: 1. Installed SSD Cache. 2. Did not realize appdata share was set to "cache prefer" be default. 3. Noticed the yellow triangle before the appdata share and data copied on the cache drive. 4. Stopped array, turned appdata share to "cache no". 5. Reformatted SSD. 6. Started Array. 7. Eventually noticed "User0" which had never been there before. 8. Get irritated thinking I have duplicate data. 9. rm -rd /mnt/user0 -- before reading, or doing anything smart -- mind you its 2:00 AM at this point. 10. Watch all of my Shares slowly disappear from the GUI. 11. Rebuild new box from scratch. 12. Start data migration from old back to new. 13. Post here. So with all of that jargon out of the way (and I hope its not too long). Can someone help me understand the relationship between cache, mover and user0? I'm now very afraid to put cache back on once the data is migrated TLDR: - How does mover work with user0? If I understand properly, user0 represent files which are not in cache or to be moved to cache. - Any idea why mover created user0 after appdata was set to "cache no"? All other shares were "cache no." I assume once I turned on cache I started the process and was simply too late. - Why did the shares and data under user vaporize? Is it because I wiped user0, thus the files on user were cleaned as they were assumed to have been moved to cache? Thanks a ton in advance. I really want to move appdata and dockers to SSD in order to improve performance, specifically on plex. The library is growing and the tumbs can sometimes load kinda slow. Thank you! Link to comment
SSD Posted February 8, 2018 Share Posted February 8, 2018 1 hour ago, nuticles said: First of all I want to thank the community a ton. I have found every answer I have needed for the past year on the forums without needing to post once! Until now.... A little background for everyone, I have successfully migrated from old hardware (i5 3570 / 16 GB / 5 x 2 TB HGST drives) on to my new box (2x Xeon 2650s, 64 GB 5 x 8 TB HGST Drives). I was able to move my appdata, get all of my dockers over to the new box as well as migrate from Crashplan Home to Pro. I credit these forums for everything from build, docker migration and finally data migration all being flawless. The good news, I have multiple copies of my data 3...2...1... backup scheme including an up-to-date synch between the old (i5) and new (xeon) boxes. So all is not lost. The bad news. I somehow nailed all of my data. A big part of the problem is I get addicted to these projects and stay up way too late and am not fresh, regardless. Here is what I did: 1. Installed SSD Cache. 2. Did not realize appdata share was set to "cache prefer" be default. 3. Noticed the yellow triangle before the appdata share and data copied on the cache drive. 4. Stopped array, turned appdata share to "cache no". 5. Reformatted SSD. 6. Started Array. 7. Eventually noticed "User0" which had never been there before. 8. Get irritated thinking I have duplicate data. 9. rm -rd /mnt/user0 -- before reading, or doing anything smart -- mind you its 2:00 AM at this point. 10. Watch all of my Shares slowly disappear from the GUI. 11. Rebuild new box from scratch. 12. Start data migration from old back to new. 13. Post here. So with all of that jargon out of the way (and I hope its not too long). Can someone help me understand the relationship between cache, mover and user0? I'm now very afraid to put cache back on once the data is migrated TLDR: - How does mover work with user0? If I understand properly, user0 represent files which are not in cache or to be moved to cache. - Any idea why mover created user0 after appdata was set to "cache no"? All other shares were "cache no." I assume once I turned on cache I started the process and was simply too late. - Why did the shares and data under user vaporize? Is it because I wiped user0, thus the files on user were cleaned as they were assumed to have been moved to cache? Thanks a ton in advance. I really want to move appdata and dockers to SSD in order to improve performance, specifically on plex. The library is growing and the tumbs can sometimes load kinda slow. Thank you! Good you have backups! User and user0 are part of the logical filesystem that support user shares called FUSE. These shares are not duplicated data, but just an alternate view of the data on your disks. If you delete data from /mnt/user or /mnt/user0, it is same as deleting data from the disks. It had been a while but I think the only difference between user and user0 is that user0 does not include data on the cache. Normally you would just ignore it, and if you wanted to access a user share called Movie, for example, you would refer to /mnt/user/Movie. That would appear as a combination of all files on all disks inside their Movies folders. But /mnt/user0/Movie would be virtually same without files on the cache. Mover uses this. They are not duplicates but links to the disk files. Deleting then deletes the fundamental data. So you have basically deleted all of your data on all of your disks. Lacking backups, the situation would be dire. You would be looking at filesystem utilities, 3rd party tools, our commercial solutions. Assuming you are using xfs, which is a very good filesystem, but not one that undeletes easily or at all. I once deleted some data and decided I would attempt to recover it as a sort of experiment to see if it were possible. I had no luck. If you were using RFS, undelete is much more likely. RFS is old and has lots of problems, but undeleting is a strength. Not sure about if you were using BTRFS. Maybe others do have advice. You might Google for "xfs undelete" and read the hits. But I believe you'll be using your backups to recover. Here is a sample link I found on xfs undelete. https://www.linuxquestions.org/questions/linux-software-2/how-do-i-undelete-files-on-an-xfs-filesystem-718787/ I used a trial of a tool that did nothing. Forget is name. Spoke to the author who admitted the the xfs internal structure made undelete very unlikely. There are paid services that might be best chance of recovery. Could have been disaster. Be careful deleting! Lesson learned I'm sure. Link to comment
S80_UK Posted February 8, 2018 Share Posted February 8, 2018 20 minutes ago, SSD said: It had been a while but I think the only difference between user and user0 is that user0 does not include data on the cache. That is correct. Pleased to see I am not the only one that keeps backups! Link to comment
nuticles Posted February 8, 2018 Author Share Posted February 8, 2018 5 hours ago, SSD said: User and user0 are part of the logical filesystem that support user shares called FUSE. These shares are not duplicated data, but just an alternate view of the data on your disks. If you delete data from /mnt/user or /mnt/user0, it is same as deleting data from the disks. It had been a while but I think the only difference between user and user0 is that user0 does not include data on the cache. Normally you would just ignore it, and if you wanted to access a user share called Movie, for example, you would refer to /mnt/user/Movie. That would appear as a combination of all files on all disks inside their Movies folders. But /mnt/user0/Movie would be virtually same without files on the cache. Mover uses this. They are not duplicates but links to the disk files. Deleting then deletes the fundamental data. So you have basically deleted all of your data on all of your disks. Thank you for confirming my hunch about user versus user0. I've read it explained a lot of ways, and so far yours was the best and easiest for me to understand. Once I thought about it as two different views in a "logical filesystem" versus the actual files themselves it really clicked. Dropping the FUSE breadcrumb helped as well, this will give me something more to while the data moves back over the next 3 days!! I am too paranoid not to have multiple data backups. I did go the XFS route. I'm not sure why, but in cases of mass file loss I do not have a ton of confidence in undelete so despite it being much faster than a restore, I typically go to backup. 5 hours ago, S80_UK said: That is correct. Pleased to see I am not the only one that keeps backups! Yes thank God for backups! I love the 3, 2, 1 philosophy. It will also take me 30 to 60 days after finalizing the new box before I am willing to tear down the old. For me it was worth buying another license to run these in parallel for a while. Even after the 60 or so days, I may strip the old box down to 1 drive and use it as a "dev" or "QA" box. So one or two more questions on this issue: - What triggers the creation of the user0? You need an "AND" condition. You need a cache drive configured AND a share set to a cache favorable setting (yes, prefer, only). But if only cache is installed OR a share is set cache favorable it will not. - If for some reason you tried cache but didn't like it, how do you get rid of user0 and ensure mover (or whatever process) will not create user0 again? I doubt I'll go backwards from cache, but I am more curios where I went wrong. Thanks a ton for the support! Link to comment
SSD Posted February 8, 2018 Share Posted February 8, 2018 3 hours ago, nuticles said: - What triggers the creation of the user0? You need an "AND" condition. You need a cache drive configured AND a share set to a cache favorable setting (yes, prefer, only). But if only cache is installed OR a share is set cache favorable it will not. If user shares are enabled, user and user0 are there. They could be identical. Quote - If for some reason you tried cache but didn't like it, how do you get rid of user0 and ensure mover (or whatever process) will not create user0 again? I doubt I'll go backwards from cache, but I am more curios where I went wrong. If user shares are enabled, user and user0 are there. Quote Thanks a ton for the support! Welcome! When I first responded I had missed that part about you having backups. Was feeling so bad for you! Glad you have the backups and hope the loss is none or very minor. Could have been a very expensive lesson! User shares are a powerful yet quirky feature. You should read up on the user share copy bug so you know to avoid it. Link to comment
nuticles Posted February 8, 2018 Author Share Posted February 8, 2018 1 hour ago, SSD said: If user shares are enabled, user and user0 are there. They could be identical. User0 was not there from the beginning, only after cache was turned on. Unless its hidden and I somehow missed it. Either I did something wrong or something triggered it. Unfortunately I am more nervous now! What did I do....? "Tower-New" (below) is the box I had issues with and since started from scratch. As you can see, there is no user0 and I do have a "server" user share created myself. "Tower" (below) which is the old box I have been using for years with user shares also shows no user0. Also with the same "server" user share I created. 1 hour ago, SSD said: User shares are a powerful yet quirky feature. You should read up on the user share copy bug so you know to avoid it. Is this what you are referring to? The fact that you cannot copy between disk and user shares or risk data loss? Thats scary, fortunately I almost exclusively use user shares.... I am glad I know this now. Phew.... https://lime-technology.com/forums/topic/60961-is-there-a-simple-way-to-clean-up-user-shares/?tab=comments#comment-598522 Link to comment
JorgeB Posted February 8, 2018 Share Posted February 8, 2018 user0 only exists when there's a cache device/pool. Link to comment
nuticles Posted February 8, 2018 Author Share Posted February 8, 2018 7 minutes ago, johnnie.black said: user0 only exists when there's a cache device/pool. Thanks! Not that I'll ever monkey with it again, but if you decide against cache and turn it back off is there a way to remove user0? I'm asking more because I am curious how I went wrong.... its really not super relevant, as I have the most important answers! Link to comment
JorgeB Posted February 8, 2018 Share Posted February 8, 2018 4 minutes ago, nuticles said: but if you decide against cache and turn it back off is there a way to remove user0? AFAIK it will only go way if you unassign the cache device. Link to comment
nuticles Posted February 8, 2018 Author Share Posted February 8, 2018 14 minutes ago, johnnie.black said: AFAIK it will only go way if you unassign the cache device. Interesting... maybe I'll run a test and confirm. Although after this nightmare I may not want to monkey around. Thanks for the support! Link to comment
trurl Posted February 9, 2018 Share Posted February 9, 2018 I'm pretty sure user, user0, and their subfolders which make up the user shares, are dynamically created when the array is started. The top level folders on cache and array disks are automatically user shares, with default settings unless the settings exist in /boot/config/shares, which is where they are stored from the webUI. Link to comment
SSD Posted February 9, 2018 Share Posted February 9, 2018 Sorry, I misspoke. I have a cache but have no user shares configured to cache writes. And user0 still appears. Guess I never checked if user0 exists with no cache drive at all. I am confused with this preoccupation with doing away with user0. It is a logical filesystem, not a physical one. It takes little or no resources. I just went to user0 and did an "ls", and there was a long pause and my disks spun up. This indicates, as @trurl just mentioned that this is dynamically built. I do not believe there is any reason to want it to go away. I could argue that making it hidden would be a good idea to avoid confusion like it did in this case. Link to comment
nuticles Posted February 11, 2018 Author Share Posted February 11, 2018 On 2/8/2018 at 8:51 PM, SSD said: Sorry, I misspoke. I have a cache but have no user shares configured to cache writes. And user0 still appears. Guess I never checked if user0 exists with no cache drive at all. I am confused with this preoccupation with doing away with user0. It is a logical filesystem, not a physical one. It takes little or no resources. I just went to user0 and did an "ls", and there was a long pause and my disks spun up. This indicates, as @trurl just mentioned that this is dynamically built. I do not believe there is any reason to want it to go away. I could argue that making it hidden would be a good idea to avoid confusion like it did in this case. Please don't apologize, anyone who takes the time out of their busy lives to contribute to someone else's learning and or problem solving is great. That said, is there any way I can thank you and other members who responded? I see there is reputation, how can I add to yours and the other folks who contributed? Ok, now on to the whole User0 thing. Some background on me first of all, as much as I have been in IT for years between a childhood hobby. This is the part I should deny, my first PC was an Apple //e. Yes I dated myself. There are several factors here which is why I became obsessed with User0: This one is huge -- it was 3 AM! - I was tired and went with the "fire, aim, ready" order. All typical diligence was put aside, stupid, I know. Typically with any change I follow the carpenter's method "measure twice cut once." I didn't read (aka measure) one lick on user0, I just nuked it (aka cut). I am a linux n00b -- I did not realize it was a logical file system, I thought I created some bizarre data duplication. Once I make a mistake, I tend to examine the crap out of it. In my mind the true mistake is not learning from it! I am slightly IT OCD. I like to think of things being "Clean" or "Lean." This is probably too many years of running Windows because its so damn bloated over time. In hindsight, this mistake was one of the best unRaid learning experiences. That said, I hope I don't screw up this bad again! TLDR: Thanks for the help. How do I add forum reputation to those what helped? When I screw up I need to examine why thoroughly! Link to comment
SSD Posted February 11, 2018 Share Posted February 11, 2018 45 minutes ago, nuticles said: How do I add forum reputation to those what helped? There is a heart icon on the bottom right of each post. Click it and you will add reputation. You are in similar company. Many of the unRAID folks here have a lot of years of experience under our belts. TRS-80 was my first computer experience when I was in high school. Link to comment
nuticles Posted February 11, 2018 Author Share Posted February 11, 2018 Just now, SSD said: There is a heart icon on the bottom right of each post. Click it and you will add reputation. You are in similar company. Many of the unRAID folks here have a lot of years of experience under our belts. TRS-80 was my first computer experience when I was in high school. I love it! I remember when my father bought a "Trash 80" for his small business. That's when I first discovered the text-based game "Planetfall," among others. Poor little Floyd Link to comment
S80_UK Posted February 11, 2018 Share Posted February 11, 2018 Hand soldered my ZX80 in around April 1980, one of the first of that generation... Link to comment
nuticles Posted February 11, 2018 Author Share Posted February 11, 2018 42 minutes ago, S80_UK said: Hand soldered my ZX80 in around April 1980, one of the first of that generation... You are a baller! I always botch hand soldering anything smaller than car stereo wiring. When I was a teen I'd solder to compliment the crimping... I was insane back then. Or when building subwoofer boxes I'd solder to the woofer and the plate which connects to the outside of the box. Anything smaller I'd make a mess.... amps, circuit boards, etc.. nope... ruined Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.