michael123 Posted October 22, 2009 Author Share Posted October 22, 2009 I not at home, I will be back next week, so I can post syslog here I am happy to hear that I can speed this box up I really appreciate your help, friends Quote Link to comment
WizADSL Posted October 23, 2009 Share Posted October 23, 2009 I am aware that dd read/write is affected by block size, does anyone know what block size samba uses? Quote Link to comment
michael123 Posted October 27, 2009 Author Share Posted October 27, 2009 Hi henris finally got back home :-) My syslog is attached (renamed it to .TXT) Michael Quote Link to comment
henris Posted October 28, 2009 Share Posted October 28, 2009 I didn't spot anything odd in your syslog however there are people in forums with x-times better skills in analysing syslogs. The only thing which comes to my mind is the AHCI vs IDE mode in bios. I wasn't able to see any reference to AHCI in your syslog. To my understanding there should have been something like: Sep 29 17:16:35 Tower kernel: scsi1 : ahci Sep 29 17:16:35 Tower kernel: scsi2 : ahci Sep 29 17:16:35 Tower kernel: scsi3 : ahci Sep 29 17:16:35 Tower kernel: scsi4 : ahci Sep 29 17:16:35 Tower kernel: scsi5 : ahci Sep 29 17:16:35 Tower kernel: scsi6 : ahci Sep 29 17:16:35 Tower kernel: ata1: SATA max UDMA/133 abar m8192@0xfcf76000 port 0xfcf76100 irq 28 Sep 29 17:16:35 Tower kernel: ata2: SATA max UDMA/133 abar m8192@0xfcf76000 port 0xfcf76180 irq 28 Sep 29 17:16:35 Tower kernel: ata3: SATA max UDMA/133 abar m8192@0xfcf76000 port 0xfcf76200 irq 28 Sep 29 17:16:35 Tower kernel: ata4: SATA max UDMA/133 abar m8192@0xfcf76000 port 0xfcf76280 irq 28 Sep 29 17:16:35 Tower kernel: ata5: SATA max UDMA/133 abar m8192@0xfcf76000 port 0xfcf76300 irq 28 Sep 29 17:16:35 Tower kernel: ata6: SATA max UDMA/133 abar m8192@0xfcf76000 port 0xfcf76380 irq 28 Instead your syslog listed: Oct 24 20:53:23 VAULT kernel: scsi1 : sata_mv Oct 24 20:53:23 VAULT kernel: scsi2 : sata_mv Oct 24 20:53:23 VAULT kernel: scsi3 : sata_mv Oct 24 20:53:23 VAULT kernel: scsi4 : sata_mv Oct 24 20:53:23 VAULT kernel: ata1: SATA max UDMA/133 mmio m1048576@0xfe600000 port 0xfe622000 irq 16 Oct 24 20:53:23 VAULT kernel: ata2: SATA max UDMA/133 mmio m1048576@0xfe600000 port 0xfe624000 irq 16 Oct 24 20:53:23 VAULT kernel: ata3: SATA max UDMA/133 mmio m1048576@0xfe600000 port 0xfe626000 irq 16 Oct 24 20:53:23 VAULT kernel: ata4: SATA max UDMA/133 mmio m1048576@0xfe600000 port 0xfe628000 irq 16 I'm not sure if this means AHCI is disabled or not (wise men/women help out here). I have AHCI enabled and I would assume new system would also so it is worth a check. Enabling AHCI does not automatically mean increase in performance but it's still worth a try. There should not be any side effects using AHCI. Quote Link to comment
michael123 Posted October 28, 2009 Author Share Posted October 28, 2009 thanks, henris as my box came from Lime, I assume then Tom shall come to the rescue, right? How can I ping them here? Quote Link to comment
lionelhutz Posted October 30, 2009 Share Posted October 30, 2009 Wow, that sucks. I can speed up unraid easily, if that is the logic, why the f do we spend literally hours clearing drives? The only time unraid should be "reading" the data drive is if the sector has already been written too, ie a file is being overwritten. Otherwise we know the byte is all zeroes, we need to read the parity, xor the new byte being written. Job done and only one read operation. Did I miss something? OK, think of a drive that has been in use for a while. When you delete a file unRAID does not zero out the actual media of the drive. Remember the drive basically just removes a file from the allocation table when it is deleted. So, if you write to the drive again you could be writing to a cleared area or to an area that is not longer cleared. This happens below the unRAID control level so unRAID has no knowledge of this. The reason to clear a new drive is that a drive with all zero data will not affect the parity calculation. If you create a brand new array and add say 5 data and the parity drive it will not clear the drives but rather just do a parity calculation. The clearing is only to add a new drive to an existing arrar without breaking parity. Peter Quote Link to comment
michael123 Posted November 7, 2009 Author Share Posted November 7, 2009 henris thank you for your benchmarking tips, it was very helpful as I am not a Linux guy What I received is following ***READ *** root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/dev/null 73184338+1 records in 73184338+1 records out 37470381528 bytes (37 GB) copied, 609.037 s, 61.5 MB/s *** WRITE *** root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/mnt/disk4/Movies/Rush.mkv 73184338+1 records in 73184338+1 records out 37470381528 bytes (37 GB) copied, 3073.51 s, 12.2 MB/s Often "dd" performs better when some kind of block size is specified: root@Tower:~# ########### READS ########### root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches root@Tower:~# dd if=/mnt/disk1/test/testfile of=/dev/null 1421820+0 records in 1421820+0 records out 727971840 bytes (728 MB) copied, 9.52896 s, 76.4 MB/s root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches root@Tower:~# dd if=/mnt/disk1/test/testfile of=/dev/null 1421820+0 records in 1421820+0 records out 727971840 bytes (728 MB) copied, 9.75895 s, 74.6 MB/s root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches root@Tower:~# dd if=/mnt/disk1/test/testfile of=/dev/null bs=1K 710910+0 records in 710910+0 records out 727971840 bytes (728 MB) copied, 7.08991 s, 103 MB/s root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches root@Tower:~# dd if=/mnt/disk1/test/testfile of=/dev/null bs=1K 710910+0 records in 710910+0 records out 727971840 bytes (728 MB) copied, 7.09987 s, 103 MB/s root@Tower:~# ########### WRITES ########### root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches root@Tower:~# dd if=/mnt/disk1/test/testfile of=/mnt/disk4/test/file1 1421820+0 records in 1421820+0 records out 727971840 bytes (728 MB) copied, 55.6483 s, 13.1 MB/s root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches root@Tower:~# dd if=/mnt/disk1/test/testfile of=/mnt/disk4/test/file2 1421820+0 records in 1421820+0 records out 727971840 bytes (728 MB) copied, 55.118 s, 13.2 MB/s root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches root@Tower:~# dd if=/mnt/disk1/test/testfile of=/mnt/disk4/test/file3 bs=1K 2776+1 records in 2776+1 records out 727971840 bytes (728 MB) copied, 45.5878 s, 16.0 MB/s root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches root@Tower:~# dd if=/mnt/disk1/test/testfile of=/mnt/disk4/test/file4 bs=1k 694+1 records in 694+1 records out 727971840 bytes (728 MB) copied, 45.1853 s, 16.1 MB/s Notice how the reads jumped from ~75 MB/s to ~100 MB/s? Similar for the writes. Purko There is a serious performance boost (250%) in recent beta version (I do not see however that block size has any influence in my case) VAULT login: root Linux 2.6.31.5-unRAID. root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/dev/null 73184338+1 records in 73184338+1 records out 37470381528 bytes (37 GB) copied, 644.946 s, 58.1 MB/s root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/dev/null bs=1K 36592169+1 records in 36592169+1 records out 37470381528 bytes (37 GB) copied, 642.689 s, 58.3 MB/s root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/dev/null bs=16K 2287010+1 records in 2287010+1 records out 37470381528 bytes (37 GB) copied, 644.15 s, 58.2 MB/s root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/mnt/disk4/Movies/Rush.mkv 73184338+1 records in 73184338+1 records out 37470381528 bytes (37 GB) copied, 1309.42 s, 28.6 MB/s root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/mnt/disk4/Movies/Rush2.mkv bs=1K 36592169+1 records in 36592169+1 records out 37470381528 bytes (37 GB) copied, 1766.9 s, 21.2 MB/s root@VAULT:~# Quote Link to comment
Kaygee Posted November 8, 2009 Share Posted November 8, 2009 There is a serious performance boost (250%) in recent beta version (I do not see however that block size has any influence in my case) root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/mnt/disk4/Movies/Rush.mkv 73184338+1 records in 73184338+1 records out 37470381528 bytes (37 GB) copied, 1309.42 s, 28.6 MB/s root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/mnt/disk4/Movies/Rush2.mkv bs=1K 36592169+1 records in 36592169+1 records out 37470381528 bytes (37 GB) copied, 1766.9 s, 21.2 MB/s Surely that is 33% improvement in writes by specifying a 1k stripe? Quote Link to comment
michael123 Posted November 8, 2009 Author Share Posted November 8, 2009 Its more regression, than improvement 21MB/s vs 28MB/s Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.