For anyone who is also seeking the answer:
Use self signed certificates on your docker registry, then copy the crt file to the following folder in your UNRAID:
/etc/docker/certs.d/myregistrydomain.com:5000/ca.crt
For detail instruction you can checkout https://docs.docker.com/registry/insecure/#use-self-signed-certificates