June 16, 201313 yr There have been many discussions regarding the benefits, or lack thereof, in having a faster parity drive. With some of the new test utilities devised by members I have some numbers to prove it's worth if you use your unRAID server as heavily as I do. I have 2 identical N54L Microservers. Each is running ESX with 16GB of ram. 1GB is allocated to unRAID 5.0-RC15 First test was matching WD 2TB 5400 RPM data data and parity drives. I grabbed some statistics from another thread to capture here. The following tests were performaned. test with writes to exceed ram in serial. Same test with writes to exceed ram in parallel. I then swapped the parity drive with a modern Seagate 3TB 7200 RPM drive. One of the fastest I've seen so far. The tests were issued again. N54L, unRAID 5.0 RC-15 under ESX 5.1 with 1GB ram allocated to unRAID. 2TB 5400 RPM Parity Drive root@unRAID1:~# ls -1 /dev/disk/by-id | grep -v part scsi-1ATA_WDC_WD20EARX-00PASB0_WD-WCAZAC296206@ scsi-1ATA_WDC_WD20EARX-00PASB0_WD-WCAZAC993717@ scsi-1ATA_WDC_WD20EARX-00PASB0_WD-WCAZAJ255820@ scsi-1ATA_WDC_WD20EARX-00PASB0_WD-WCAZAJ271733@ usb-Generic-_USB3.0_CRW_-0_000000002318-0:0@ root@unRAID1:~# df -vH Filesystem Size Used Avail Use% Mounted on /dev/sdf1 32G 1.9G 30G 6% /boot tmpfs 135M 160k 135M 1% /var/log /dev/md1 2.1T 34M 2.1T 1% /mnt/disk1 /dev/md2 2.1T 2.2G 2.0T 1% /mnt/disk2 /dev/md3 2.1T 1.3T 756G 63% /mnt/disk3 shfs 6.1T 1.3T 4.8T 21% /mnt/user These are all the same model drive 2TB 5400 RPM. This gives me a real good platform and test to see how a faster parity drive will perform. root@unRAID1:~# /boot/bin/diskspeed.sh diskspeed Version 1.6 <c> 2013 by Pourko Balkanski diskspeed[1998]: Testing: version=5.0-rc15 /mnt/disk1 /mnt/disk2 /mnt/disk3 diskspeed[1998]: Options: bs=64k count=30000 parallel=yes dropcache=no forcesync=no diskspeed[1998]: Syncing filesystems diskspeed[1998]: Performing write tests (parallel=yes)... diskspeed[1998]: /mnt/disk1: write 1966080000 bytes (2.0 GB) copied, 248.136 s, 7.9 MB/s diskspeed[1998]: /mnt/disk3: write 1966080000 bytes (2.0 GB) copied, 256.887 s, 7.7 MB/s diskspeed[1998]: /mnt/disk2: write 1966080000 bytes (2.0 GB) copied, 259.428 s, 7.6 MB/s real 4m19.471s user 0m0.600s sys 0m30.280s diskspeed[1998]: /mnt/disk1: read 1966080000 bytes (2.0 GB) copied, 15.9243 s, 123 MB/s diskspeed[1998]: /mnt/disk2: read 1966080000 bytes (2.0 GB) copied, 17.177 s, 114 MB/s diskspeed[1998]: /mnt/disk3: read 1966080000 bytes (2.0 GB) copied, 17.7904 s, 111 MB/s real 0m17.830s user 0m0.100s sys 0m7.890s N54L, unRAID 5.0 RC-15 under ESX 5.1 with 1GB ram allocated to unRAID. 3TB 7200RPM Parity Drive root@unRAID1:~# ls -1 /dev/disk/by-id | grep -v part scsi-1ATA_ST3000DM001-9YN166_W1F191JR@ scsi-1ATA_WDC_WD20EARX-00PASB0_WD-WCAZAC993717@ scsi-1ATA_WDC_WD20EARX-00PASB0_WD-WCAZAJ255820@ scsi-1ATA_WDC_WD20EARX-00PASB0_WD-WCAZAJ271733@ usb-Generic-_USB3.0_CRW_-0_000000002318-0:0@ root@unRAID1:~# df -vH Filesystem Size Used Avail Use% Mounted on /dev/sdf1 32G 1.9G 30G 6% /boot tmpfs 135M 185k 135M 1% /var/log /dev/md1 2.1T 34M 2.1T 1% /mnt/disk1 /dev/md2 2.1T 2.2G 2.0T 1% /mnt/disk2 /dev/md3 2.1T 1.3T 756G 63% /mnt/disk3 shfs 6.1T 1.3T 4.8T 21% /mnt/user Disk /dev/sda: 1073 MB, 1073741824 bytes Disk /dev/sdb: 3000.6 GB, 3000592982016 bytes Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes Disk /dev/sde: 2000.4 GB, 2000398934016 bytes Disk /dev/sdf: 31.9 GB, 31914983424 bytes root@unRAID1:~# free -l total used free shared buffers cached Mem: 1033116 347316 685800 0 31948 208852 Low: 898004 266364 631640 High: 135112 80952 54160 -/+ buffers/cache: 106516 926600 Swap: 0 0 0 root@unRAID1:~# parallel=no /boot/bin/diskspeed.sh diskspeed Version 1.6 <c> 2013 by Pourko Balkanski diskspeed[2003]: version=5.0-rc15 mem=1008 diskspeed[2003]: Testing: /mnt/disk1 /mnt/disk2 /mnt/disk3 diskspeed[2003]: Options: bs=64k count=30000 parallel=no dropcache=no forcesync=no diskspeed[2003]: Syncing filesystems diskspeed[2003]: Performing write tests (parallel=no)... diskspeed[2003]: /mnt/disk1: write 1966080000 bytes (2.0 GB) copied, 61.5472 s, 31.9 MB/s diskspeed[2003]: /mnt/disk2: write 1966080000 bytes (2.0 GB) copied, 65.2266 s, 30.1 MB/s diskspeed[2003]: /mnt/disk3: write 1966080000 bytes (2.0 GB) copied, 69.9301 s, 28.1 MB/s real 3m16.792s user 0m0.230s sys 0m28.080s diskspeed[2003]: /mnt/disk1: read 1966080000 bytes (2.0 GB) copied, 16.1127 s, 122 MB/s diskspeed[2003]: /mnt/disk2: read 1966080000 bytes (2.0 GB) copied, 15.9703 s, 123 MB/s diskspeed[2003]: /mnt/disk3: read 1966080000 bytes (2.0 GB) copied, 17.361 s, 113 MB/s real 0m49.518s user 0m0.080s sys 0m9.420s root@unRAID1:~# parallel=yes /boot/bin/diskspeed.sh diskspeed Version 1.6 <c> 2013 by Pourko Balkanski diskspeed[2066]: version=5.0-rc15 mem=1008 diskspeed[2066]: Testing: /mnt/disk1 /mnt/disk2 /mnt/disk3 diskspeed[2066]: Options: bs=64k count=30000 parallel=yes dropcache=no forcesync=no diskspeed[2066]: Syncing filesystems diskspeed[2066]: Performing write tests (parallel=yes)... diskspeed[2066]: /mnt/disk1: write 1966080000 bytes (2.0 GB) copied, 149.387 s, 13.2 MB/s diskspeed[2066]: /mnt/disk2: write 1966080000 bytes (2.0 GB) copied, 157.978 s, 12.4 MB/s diskspeed[2066]: /mnt/disk3: write 1966080000 bytes (2.0 GB) copied, 159.558 s, 12.3 MB/s real 2m39.617s user 0m0.120s sys 0m26.280s diskspeed[2066]: /mnt/disk1: read 1966080000 bytes (2.0 GB) copied, 15.6154 s, 126 MB/s diskspeed[2066]: /mnt/disk2: read 1966080000 bytes (2.0 GB) copied, 16.7625 s, 117 MB/s diskspeed[2066]: /mnt/disk3: read 1966080000 bytes (2.0 GB) copied, 19.6811 s, 99.9 MB/s real 0m19.706s user 0m0.130s sys 0m7.820s Where it shows the performance boost is in the parallel writes diskspeed[1998]: /mnt/disk1: write 1966080000 bytes (2.0 GB) copied, 248.136 s, 7.9 MB/s diskspeed[1998]: /mnt/disk3: write 1966080000 bytes (2.0 GB) copied, 256.887 s, 7.7 MB/s diskspeed[1998]: /mnt/disk2: write 1966080000 bytes (2.0 GB) copied, 259.428 s, 7.6 MB/s vs diskspeed[2066]: /mnt/disk1: write 1966080000 bytes (2.0 GB) copied, 149.387 s, 13.2 MB/s diskspeed[2066]: /mnt/disk2: write 1966080000 bytes (2.0 GB) copied, 157.978 s, 12.4 MB/s diskspeed[2066]: /mnt/disk3: write 1966080000 bytes (2.0 GB) copied, 159.558 s, 12.3 MB/s While it's not much, every little bit helps when your server is a busy one. How this affects your parity sync and generate speeds, that's a test for another time. I failed to capture the ending statistics, so I'll have to do that another day. At the very least, this is parity generate for the new 3TB drive. I'll come back at another time when I put the 2TB drive back it. It's a bit of work to replace drives in the unRAID VM. 3TB Parity Generate Jun 15 19:53:20 unRAID1 kernel: md: sync done. time=30923sec
June 17, 201313 yr Nice thread. Can we see some "real world" results. What is the speed difference between: a) Writing a 10gb file to the server. b) Doing a full parity sync/calculation. That would be great.
June 17, 201313 yr Author Nice thread. Can we see some "real world" results. What is the speed difference between: a) Writing a 10gb file to the server. b) Doing a full parity sync/calculation. That would be great. I will consider it. It's going to take a very long time though. I have the following drives for parity. 2TB 5400 matching drive. 2TB 7200 Seagate. 3TB 7200 Seagate. Also Parity check vs parity generate can be different. It's also HIGHLY system dependent. In my older system, with the ARC-1200, it was faster to create the parity then check it. When I get some time and a good way of capturing the information, I'll post results.
June 17, 201313 yr Author Nice thread. Can we see some "real world" results. What is the speed difference between: a) Writing a 10gb file to the server. b) Doing a full parity sync/calculation. That would be great. ALSO FWIW, what I posted below are REAL world results. I've always been a proponent of faster parity drives. While it may not affect parity sync/create speeds by much, it surely helps a little for regular drive writes when using multiple drives (simultaneously). The numbers below prove it.
June 17, 201313 yr Nice thread. Can we see some "real world" results. What is the speed difference between: a) Writing a 10gb file to the server. b) Doing a full parity sync/calculation. That would be great. ALSO FWIW, what I posted below are REAL world results. I've always been a proponent of faster parity drives. While it may not affect parity sync/create speeds by much, it surely helps a little for regular drive writes when using multiple drives. The numbers below prove it. I don;t think anyone will disagree, when writing concurrently/simultaneously to multiple data drives, a faster parity drive will result in more throughput than a slower one. It is not when using multiple data drives, it is when writing (at the same time) multiple data drives where it helps.
June 17, 201313 yr Author It is not when using multiple data drives, it is when writing (at the same time) multiple data drives where it helps. Correct, sorry, that was obvious to me.
June 17, 201313 yr Author Cool! Have you ever tested parity drive setup as Raid 10? I tried RAID0, it showed improvement, but it was a small improvement. 3-6 MB/s using the silicon image port multiplier chipset. I think it was about 10MB/s using the ARC-1200 in my setup with hybrid RAID0/RAID1. i.e. SAFE mode. The outer tracks were RAID0 for the parity. The inner tracks were RAID1 for my cache drive. I enabled hardware caching on the parity drive. It was a big improvement for my server at the time. Frankly, the best 'burstable' improvement were kernel tunings which let me burst at a very high speed using the the buffer cache to hold a decent amount of data. You can read about it here. http://lime-technology.com/forum/index.php?topic=25431.30
Archived
This topic is now archived and is closed to further replies.