Ok, I figured it out. By default the system can't even change clocks at all. I ended up having to set the kernel option
amdgpu.ppfeaturemask=0xfff7ffff
and once booted with that setting I launch the container with the various flags to change clock, memory, and voltage and it appears to work.
Radeon VII, stock memory is 1.00G and 1.80G or something for clock. I can only assume that the voltage is changed too, not sure where to check that. Also the later drivers don't seem to work because of some kernel stuff only in 5.13 and later so you have to use the 6.10 pre-releases, I think. Also also the 21.40.1 drivers have a sleep infinity in the install step so that's why it never gets passed the install step. I've submitted an issue on github for that so hopefully he fixes it quick.