Jump to content
markswam

Updated to 6.6.2. All shares gone.

19 posts in this topic Last Reply

Recommended Posts

Hi all,

 

I'm sorry in advance if this comes across a little panicky. I'm doing my best to remain calm, but I've got over 10TB of data that I'm worried about losing at the moment.

 

So today, when I got back from work, I was prompted that there was an update to unRAID available. Naturally, I installed it and rebooted. That's when my problem kicked in. As soon as my system came back online, I noticed that I couldn't access any of my server's share's through my desktop's mapped network drive. Only the "flash" folder was visible. I decided to load up the web UI to check on it, and my Shares tab is completely empty. All of my shares are gone, apparently.

 

I checked to make sure that the data still exists on the drives, and it does. I can still see all of my files and folders through /mnt/disk*, but /mnt/user is completely empty.

 

My next thought was "okay, let's just try adding the shares back." So I tried that, only to be told "Share $share has been deleted" immediately after clicking "Done." It doesn't matter if the share is one that existed previously, or one that is completely unique. I can't add anything to the machine.

 

Attached is my diagnostics zip file and a couple of screenshots. Hopefully someone here can help me out and get me access to my data once more.

 

I have all of my data backed up elsewhere as well, but given the speeds and data caps my I$P imposes on me, it would take almost a year to download it all again, and I'd really rather avoid that.

 

I'm sorry, but I wasn't tailing the syslog to a file before the initial shutdown. I know I should be doing that, but I just didn't think of it.

 

1475776667_ss(2018-10-15at05_49.47).thumb.png.2f76aeefef640c5fd383fb2979356803.png

 

1165644741_ss(2018-10-15at05_50.05).thumb.png.cb26fdd6f9c5546aa8c4cd23bd7d960f.png

 

160562177_ss(2018-10-15at05_52.42).thumb.png.4215b869063a7402cec1a2570cdb00ac.png

 

1114780000_ss(2018-10-15at06_11.36).thumb.png.b0765011e52e77be377109618d2ea581.png

 

tower-diagnostics-20181015-1727.zip

Edited by markswam
Added more screenshots

Share this post


Link to post

What version did you upgrade from? 

Share this post


Link to post

I'm not entirely sure, but I want to say either 6.6.0 or 6.6.1. I know that I've updated at least once within the last month.

Edited by markswam

Share this post


Link to post

I've always done the updating through the web interface, since I don't know the steps for any other way off the top of my head.

Share this post


Link to post

Update: Something appears to be deleting every single folder in /mnt/user upon startup. I manually created /mnt/user/testFolder using mkdir, verified that the folder is there using ls, rebooted, and the folder is now gone.

Share this post


Link to post

This is happening to many users - the fuse file system which supports the 'shares' under /mnt/user/ appears to be failing. If you check the forums you'll see that many people have this issue.

My limited observations lead me to believe that people with newer hardware seem to be more affected.

Share this post


Link to post

In your observations, have you found anyone who's figured out a fix for this?

Share this post


Link to post
6 minutes ago, markswam said:

In your observations, have you found anyone who's figured out a fix for this?

This only seems to affect users who have NFS access to shares enabled (as you do according to your diagnostics).  I have only SMB access to shares and have never experienced the problem in any 6.6.x version.  I am currently running 6.6.2 on both servers.

 

It doesn't matter if you also have AFP or SMB, the problem is NFS.  It kills /mnt/user and so no shares are visible to AFP or SMB either.

 

The only permanent solution (until there is a kernel fix for this) at this time is to roll back to unRAID 6.5.3

Edited by Hoopster

Share this post


Link to post

There's more discussion about similar issue here:

 

I personally think that Linux support for Ryzen/Epyc is about where it would normally be - as in a few years behind. I expect that kernel 4.19+ will sort out some of the AMD issues that people are experiencing.

Share this post


Link to post

Well, alrighty. I'll try that. 

 

Just to make sure, is the process still just copying and overwriting bzroot, bzimage, and bzroot-gui on the flash drive?

Share this post


Link to post
11 minutes ago, markswam said:

Well, alrighty. I'll try that. 

 

Just to make sure, is the process still just copying and overwriting bzroot, bzimage, and bzroot-gui on the flash drive? 

There are a few extra files that now need to be copied.

 

If the last version you had installed prior to upgrading to a 6.6.x version was 6.5.3, you can roll back to 6.5.3 from the GUI via Tools>Update OS>Restore.

 

If 6.5.3 was not your previous version, you can download it from unraid.net and copy over all of the following files (this is what is stored in the "previous" folder on the flash) The most important are the bz* files:

 

image.png.b1ab2b6214887de84713838445195524.png

Edited by Hoopster

Share this post


Link to post

Further update:

 

The previous version I had was 6.6.1. I tried rolling back to that, just to see what would happen, and everything is back the way it was before.

 

That said, I'm still going to roll back to 6.5.3, for stability's sake.

Share this post


Link to post
2 minutes ago, markswam said:

Further update:

 

The previous version I had was 6.6.1. I tried rolling back to that, just to see what would happen, and everything is back the way it was before.

  

That said, I'm still going to roll back to 6.5.3, for stability's sake.

 

Because after a reboot, everything is generally fine for a few minutes or hours.  With any 6.6.x version, if you have NFS access enabled, shares will eventually disappear when FUSE crashes and takes /mnt/user with it.

 

It's hard to say if this happens on *every* 6.6.x system with NFS enabled, but, those who have reported the problem have all had NFS enabled on a 6.6.x system.

Share this post


Link to post

@markswam Also, if you have the Network Stats plugin installed, uninstall it and install the latest version.  It was causing some share-related issues as well and has been updated by the author.

Edited by Hoopster

Share this post


Link to post

Thank you Hoopster. There's something amiss with newer hardware and linux kernels. It's not only AMD as I've seen a similar issue with intel recently.

Definitely do not panic, one of the huge advantages to unraid is that all the data is on disks in a simple manner. No stripes, nothing complicated. The key thing is we have /mnt/disk# 

Rolling back is a great idea and so good this sorted things out. watch updates for kernel 4.19 or greater 

Del

 

Share this post


Link to post
11 hours ago, johnnie.black said:

The problem appears to be the nerdpack plugin overwriting glibc with an older version.

@markswam you should check this possibility. There are other reports of nerd pack plugins causing shares to disappear in 6.6.2 as noted in the 6.6.2 release thread:

 

 

The NFS issue may still be a problem, but, whether or not it will be an issue for you may be masked by the problem johnnie.black mentioned based on your diagnostics.

Share this post


Link to post
On 10/15/2018 at 11:54 PM, johnnie.black said:

The problem appears to be the nerdpack plugin overwriting glibc with an older version.

 

Talking with @eschultz about this he said:

 

"NerdPack (and probably DevPack too) puts packages in /boot/config/plugins/NerdPack/packages/6.6 folder.  NerdPack is using 'upgradepkg --install-new' AFAIK so i'm a bit puzzled how glibc is getting overwritten."

Share this post


Link to post



 
Talking with [mention=1875]eschultz[/mention] about this he said:
 
"NerdPack (and probably DevPack too) puts packages in /boot/config/plugins/NerdPack/packages/6.6 folder.  NerdPack is using 'upgradepkg --install-new' AFAIK so i'm a bit puzzled how glibc is getting overwritten."


NerdPack is fairly benign and doesn't include any packages that are present in unRAID. It's the DevPack that's the problem. I put it out there on request as a repository for compiling on a dev unRAID system not for daily use. It uses just installpkg. Going forword I think I'll strip out all the packages inherent to unRAID, include only the headers and add - headers tag to them. Something may not compile right but it won't hose the system. Basically all that is needed is the headers you guys strip out to save space for the preinstalled unRAID packages.
  • Like 2

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now