After upgrade to 6.12.9 - array stuck at mounting disk

After downgrading to 6.12.8 the array seemed to still hang on mounting disk. From settings, I change the VM Manager and Docker Manager to "NO" so they are not set to start. After doing that, the array started. I suspect me changing VM and Docker manager to "no" had no bearing. I think it was just a coincidence in timing.
However, I did try 6.12.9 again with the VM and Docker manager auto start set to NO and the array still will not mount.

UPDATE: As I was writing this and collecting more diagnostics, the array started. I believe it took around 15 mins to mount disk and start.
 

tower-diagnostics-20240401-1138.zip

I did 1 more reboot with 6.12.9 and the array finally started after 26 mins of uptime. I have attached another diagnostics and will downgrade to 6.12.8 for now.

tower-diagnostics-20240401-1234.zip

You cache pool is taking a long time to mount, don't see any disk or filesystem errors, but unlikely the issue is v6.12.9, and according to you it also took a long time to mount with 6.12.8, possibly one of the disks has an issue and is extra slow, or the zfs pool has some problem, or a lot of metadata/small files making it extra slow.

 

You can retest by manually importing the pool, it should take the same amount of time, stoop the array and type:

 

zpool import cache

time how long it rakes to get the cursor back, then export the pool before array start

zpool export cache

 

2 hours ago, bigernie71 said:

I did 1 more reboot with 6.12.9 and the array finally started after 26 mins of uptime. I have attached another diagnostics and will downgrade to 6.12.8 for now.

tower-diagnostics-20240401-1234.zip

Thanks for the feedback JorgeB.
The initial start after downgrading from 6.12.9 to 6.12.8 did take some time.

I stopped the array (running 6.12.8) and then started it again and it took 2-3 mins.

 

I stopped the array and performed the actions you suggested (zpool import cache). It took about 1 min or less.

I performed a reboot running 6.12.8 and the array started at server uptime of 3 mins.

Please retest with v6.12.9

6.12.9
First reboot - array started at server uptime 6 mins (not sure why but much better)

Second reboot - array started at server uptime 6 mins
Third reboot - testing a theory, array started at server uptime 6 mins

Im not sure what happened with the first and second install of 6.12.9 but this seems to be a non issue after the 3rd install. Probably not needed but I uploaded my diagnostics I pulled after the 3rd reboot. Thanks for the assist, very much appreciated!

tower-diagnostics-20240402-0837.zip

So it's resolved?

Yes, thank you.