Correct, just stopping and restarting the array appears to be enough to flush cache, at least the results were consistent with repeat runs, some of them after rebooting, when I changed release used.
Multiple directories, these last tests I did this morning used just part of the small files to make it go faster, the smaller ones which I noticed were the ones causing the more significant slowdown, 11.5k files in 74 folders totaling about 50MB, so very small files, and not something most people would ever use with Unraid, I was just curious after the other results, I also tested with direct_io on/off and there was no difference for almost all the tests, hence my earlier question.
On the other hand, user shares performed very well with large files, always about the same as using disk shares, on all 3 releases I tested (v6.7.2, v6.8.3 and -beta30).