Long story short,

 

Woke up this morning and noticed my Unraid server was pingable, but unresponsive.  No GUI, no shares responding, nor response from keyboard/monitor also plugged up to it.  Did a hard reset.  Now it's stuck at "Array Starting - Mounting Disks."

 

After an hour, I started a diagnostics to bring here for help, but it also has been stuck for the last hour and not completing.  Looking for some next steps on direction to go to get it back up.

 

Thanks in advance!

So, I've done some more digging.  It seems the ZFS cache pool is the culprit.  It won't start the array with the either disk from the cache pool assigned.  If I unassign both cache drives, turn off docker and vms, and start the array, it comes up fine.

 

So, at this point, I'll need someone to point me in the right direction, LOL.  I went to ZFS for the compression feature, but I'm at a disadvantage on what to do to troubleshoot why it won't start.  Diagnostics will not complete if I try to start the array with the ZFS cache pool a part of it.

I was lucky enough to catch an error this time when it tried to mount the disks to start the array as I was tinkering with it.  This is on the monitor directly connected to it while in Safe Mode and I tried to start the array with my two cache disks (mirrored) were attempting to mount as well.

 

VERIFY3(size <= rt->rt_space) failed (281442900086784 <= 1880903680)

PANIC at range_tree.c:436:range_tree_remove_impl()

Hey, I think we are having the exact same issue. I've posted some syslog info on my post as well. Any ideas on how to proceed?

This is what I found that seems to apply to me I think.

 

 

Perfect thanks for this link! I just went ahead and erased my cache drives completely (there wasnt much on there to begin with to be honest, the important stuff was on my drives) and my server is up and running perfectly so that does seem to be the issue.

Well, moving everything off the cache in safe mode by mounting it in read only mode to a disk in the array worked.  Ran a full memtest (passed) and restarted the system with auto start array turned off and removed the cache disks, erased the cache, then built a new one and it worked fine.  Copied everything back and found everything was owned by root:root (docker permissions was totally borked because of this).  At least that is fixable and a much better place to be than where I was.

 

I've seen some other people (including the one I posted above and possibly DingusKahn) with the same issue.  I'm questioning if going to ZFS to get compression on my cache was such a good idea (don't get me wrong, performance and compression are great) but only time will tell.  I guess if it happens again I'll roll it back to XFS.  This was no fun, haha.

On 8/7/2023 at 10:51 AM, Athelstan said:

Well, moving everything off the cache in safe mode by mounting it in read only mode to a disk in the array worked.  Ran a full memtest (passed) and restarted the system with auto start array turned off and removed the cache disks, erased the cache, then built a new one and it worked fine.  Copied everything back and found everything was owned by root:root (docker permissions was totally borked because of this).  At least that is fixable and a much better place to be than where I was.

 

I've seen some other people (including the one I posted above and possibly DingusKahn) with the same issue.  I'm questioning if going to ZFS to get compression on my cache was such a good idea (don't get me wrong, performance and compression are great) but only time will tell.  I guess if it happens again I'll roll it back to XFS.  This was no fun, haha.

How did you put it in read-only mode? When I add the cache disks to the array it says they will be wiped when the array is brought online

..

1 hour ago, kintamanate said:

When I add the cache disks to the array it says they will be wiped when the array is brought online

You definitely don't want to add your cache disks to the array. Probably some misunderstanding.

 

Please describe your exact problem and attach your diagnostics to your NEXT post in this thread.

On 9/17/2023 at 10:27 PM, trurl said:

You definitely don't want to add your cache disks to the array. Probably some misunderstanding.

 

Please describe your exact problem and attach your diagnostics to your NEXT post in this thread.

 

Yes, you are correct I did misunderstand. I have have create a new topic so I don't take over this one.