If your Nextcloud docker is functional and you can get into it for updates yes. If the container is not working you might need to revert the container to whatever version of 23 you have on the Nextcloud software side. On my system I upgraded to I believe version 26.0.6 then just updated the container and it updated both the rest of the way.
One note: If you backup the nextcloud appdata folder before you do any updating, if the update fails you can just restore the Nextcloud appdata folder and it should revert back to where you was before updating.