This really has me stumped.  I am unable to run diagnostics, even from the console.  The GUI has text strewn through it, docker and vm menus are missing, but oddly still working, actually behind the scenes it is still running but I'm not very happy about it.

 

Background:

I moved the server.  Oh and I Licenced it, it's a second machine and I was on trial mode.  OK that shouldn't be too bad, but I also decided to move my SSD's from a fixed bay to a hot swap bay.  That also isn't too bad, and they are still working so I don't think it's that.  I've done a memory test, but not an extensive memory test.  I've ruled out ZFS I believe and besides the file system is working fine.

 

Symptoms I have seen include mostly USB related / looking challenges.  I've reflashed and copied the config quite a few times now.

Because it's a new USB I tried the old USB too.  That's turned out with the same issue.  I've used a config backup I made before this happened.

 

I've downgraded from the beta to stable and it still happens.  Check out these screenshots - weird text where the shouldn't be.  Even getting told that VM's won't start because my hardware doesn't support VT-d, which it does and the VM's are running in the background.  I'm sorry but I can't post diagnostics -see output below.

 

I hope my hardware hasn't died - but would have thought that would show up in either disk or memory related errors, which it hasn't.  And not that the docker and virtual machines are showing up as running (and are running fine when I connect from my laptop).

 

Any ideas?  I have to admit, I'm stumped.

 

What am I missing?DiagnosticErrors.rtf

 

 

Edited October 23, 20205 yr by Marshalleq

Are you using a USB 2.0 port? Full diags might also give more clues, or this after the problems start.

Yes, it usually runs on USB3.0, but for this I'm using USB2 to be safe.  I half thought it was ZFS again, but known good versions of ZFS and precompiled kernels with ZFS are also triggering the problems.  I am unable to provide diagnostics, they don't run, not even in console which is what the attached file above is about.

 

I tried syslog, but nothing is getting logged.  It seems the system can't create (or thinks it can't create) files.  I can create files in the console though.  So I think it's broken too, which is what I was expecting to be honest.

Edited October 23, 20205 yr by Marshalleq

I unplugged one disk and it seems to have brought it back to life.  I'm just going to pretend I fixed it for tonight.  Too scared to look at it again for now.

Have you installed ZFS on this server too?

I know such problems from my testing with ZFS.

If a drive is pulled out from a running ZFS system/pool or errors out, this is a pretty standard behavior, after a restart everything works again normally.

 

This doesn't happen only on Unraid other systems are effected by this too (I have to search my Unraid-Kernel-Helper thread for this, I think I did some research about this).

 

I then modified the Kernel Helper to execute 'zfs export -a' if the array is stopped this should eject all imported zpools.

 

But I can't tell you if this is really the problem in your case only want to let you know...

I am assuming that the one disk, had some ZFS config on it, that was interfering with the rest.  The thing I don't understand is why it was playing havoc with the rest of the whole system.  I'm relatively new to ZFS so I guess what do I know, but logically if the system is on a USB in a path that has nothing to do with the ZFS volume, this kind of extenuating circumstance should not happen.  Until I know for sure I'll consider this my first ZFS drama.

 

The awesome thing is that throughout, no ZFS data was lost, all pools were active and running.  Tested multiple times - even did a scrub.

 

Anyway, I am using kernel helper again as it's easier to keep control of ZFS versions and I did notice the nice ZFS export notice at the bottom of the screen - nice job!