So not sure if this a bug or not but I did the procedure without issue and everything was fine. Decided to just to do 2 pools, and named them both differently. Specifically not "cache". After starting the array I went to the shares tab to update my shares. All shares defaulted to the first pool and I changed a couple to use the new pool. The problem lies that these shares that say they are using the first pool are not actually using that first pool which can be confirmed by the share cfg file which still had "cache" as the pool the share was using.
So my thought is, that if the config file doesn't have a valid cache pool then in the GUI it shows the first available. But this isn't correct. I had to in the GUI, switch to a different pool then the first one to get the save button to not be greyed out. And I assume I could have manually edited the cfg files to use the right pool.
This is obviously probably an edge case and not sure of a solution to propose since shares only show up after array is started. But IMO, the config and GUI should match even though it might be incorrect. Because the way it was I almost assumed those shares were going to use the new pool without issue but there was a lot of issues.