muko

Hi again,
sorry for the double post. I did some further testing today. What I did:
1) I moved all my drives to yet an other PC. Exept of the two HDD the NVME cache and the USB drive every pice of hardware was different.
Result was that the issue didnt. change. Still one drive was very slow.
2) I then createt usb drive with a fresh unraid test version install. I used a NVME drive that I had laying around as my array and put everything back in my "old" PC. Just installed the unassigned divece plugin and diskspeed.  
The result was both drives at full speed.
3) I replaced the fresh USB drive with my "production" one reinserted my cache drive and voila the error is back.
 
Interesting to add here that again the error moved from one drive to the other comparing the situation before step 1 and after. For me it seems that it realy comes down to the unraid installation ?!?
 
An other thing I noticed during doin all of that. Starting the array takes like ages. So I would also exclude a lot of processes that are started after the array is running.
 
Edit: Ok I think I now identified the error. However, I still don't fully understand it. Maybe you can help me out.. I noticed that ZFS scrup was running. When I stoped that, we're back at full speed. I still don't understand why it's sometimes the one and sometimes the other drive? Can anyone explain that for me?
 
Edit 2: I think I have the full picture now. I just made an other little experiment. I started scrub and did the speed test. I figured that my main drive was slow. Parity at full speed. I did that test several times. Next I shuted down the PC and switched the sata cable between the two drives. After reboot I started scrub again and I did an other speed test. Now my party was slow. 
Here comes the thing: Disk Speed Scans the SATA controllers once. It won't recocnize if you switch SATA cable. So after switching my main drive was shown as parity and vice versa. After hitting that rescan button the main drive speed test is the slower one again. My goodness, that cost me some of my life. I was about to buy some new drives ;(