The FAQ mentions in bold that:
You only had one cache slot.
Unassign current cache, start the array, stop the array, set cache slots to 2, assign old cache (there can't be an "all data on this device will be deleted" warning), start array, check old cache is mounting correctly, stop array, now you can replace.