I'm having this same issue. MacOS 12.1, Wifi, SMB, 144 files, total size 883 MB. Just attempting to initiate the transfer too probably 10mins, then the actual transfer is about 1-15MB/s.
Folder discovery is pretty much impossible to use on the mac. I often just go to my windows machine and find the files I need, and then push them to a smaller directory just so I can open that on my mac.
I've tried the "Enhanced macOS interoperability" setting, but that broke my connection to the NAS from my windows machine. Just wondering if anyone else came to any conclusions here.