Although unRAID is based on Linux, this will not easily done without a lot of command line work. You have to work out how to get this to be handled correctly at both the share (samba) and Linux levels. It might be possible by manipulating the permissions at the Linux Level on the folders to stop users without appropriate permission being able to get into folders but there is no built-in support for this so you would be on your own in getting it working. Also since unRAID runs from RAM and is loaded ‘fresh’ each time you boot the system you then have to do additional work to reinstate into RAM the files (e.g. etc/groups) needed to maintain the groups.
Since unRAID’s primary market seems to be home users there has not been much demand for such capability. You could consider raising a feature request for such a capability to be added but I have no idea if it would be considered something Limetech would want to put in the effort to implement, and if they did what the timescales might be.