I not at home, I will be back next week, so I can post syslog here

I am happy to hear that I can speed this box up

I really appreciate your help, friends

 

I am aware that dd read/write is affected by block size, does anyone know what block size samba uses?

Hi henris

 

finally got back home :-)

My syslog is attached (renamed it to .TXT)

 

Michael

 

I didn't spot anything odd in your syslog however there are people in forums with x-times better skills in analysing syslogs. The only thing which comes to my mind is the AHCI vs IDE mode in bios. I wasn't able to see any reference to AHCI in your syslog. To my understanding there should have been something like:

Sep 29 17:16:35 Tower kernel: scsi1 : ahci
Sep 29 17:16:35 Tower kernel: scsi2 : ahci
Sep 29 17:16:35 Tower kernel: scsi3 : ahci
Sep 29 17:16:35 Tower kernel: scsi4 : ahci
Sep 29 17:16:35 Tower kernel: scsi5 : ahci
Sep 29 17:16:35 Tower kernel: scsi6 : ahci
Sep 29 17:16:35 Tower kernel: ata1: SATA max UDMA/133 abar m8192@0xfcf76000 port 0xfcf76100 irq 28
Sep 29 17:16:35 Tower kernel: ata2: SATA max UDMA/133 abar m8192@0xfcf76000 port 0xfcf76180 irq 28
Sep 29 17:16:35 Tower kernel: ata3: SATA max UDMA/133 abar m8192@0xfcf76000 port 0xfcf76200 irq 28
Sep 29 17:16:35 Tower kernel: ata4: SATA max UDMA/133 abar m8192@0xfcf76000 port 0xfcf76280 irq 28
Sep 29 17:16:35 Tower kernel: ata5: SATA max UDMA/133 abar m8192@0xfcf76000 port 0xfcf76300 irq 28
Sep 29 17:16:35 Tower kernel: ata6: SATA max UDMA/133 abar m8192@0xfcf76000 port 0xfcf76380 irq 28

 

Instead your syslog listed:

Oct 24 20:53:23 VAULT kernel: scsi1 : sata_mv
Oct 24 20:53:23 VAULT kernel: scsi2 : sata_mv
Oct 24 20:53:23 VAULT kernel: scsi3 : sata_mv
Oct 24 20:53:23 VAULT kernel: scsi4 : sata_mv
Oct 24 20:53:23 VAULT kernel: ata1: SATA max UDMA/133 mmio m1048576@0xfe600000 port 0xfe622000 irq 16
Oct 24 20:53:23 VAULT kernel: ata2: SATA max UDMA/133 mmio m1048576@0xfe600000 port 0xfe624000 irq 16
Oct 24 20:53:23 VAULT kernel: ata3: SATA max UDMA/133 mmio m1048576@0xfe600000 port 0xfe626000 irq 16
Oct 24 20:53:23 VAULT kernel: ata4: SATA max UDMA/133 mmio m1048576@0xfe600000 port 0xfe628000 irq 16

 

I'm not sure if this means AHCI is disabled or not (wise men/women help out here). I have AHCI enabled and I would assume new system would also so it is worth a check. Enabling AHCI does not automatically mean increase in performance but it's still worth a try. There should not be any side effects using AHCI.

thanks, henris

 

as my box came from Lime, I assume then Tom shall come to the rescue, right?

How can I ping them here?

Wow, that sucks. I can speed up unraid easily, if that is the logic, why the f do we spend literally hours clearing drives?

 

The only time unraid should be "reading" the data drive is if the sector has already been written too, ie a file is being overwritten. Otherwise we know the byte is all zeroes, we need to read the parity, xor the new byte being written. Job done and only one read operation. Did I miss something?

 

 

 

OK, think of a drive that has been in use for a while. When you delete a file unRAID does not zero out the actual media of the drive. Remember the drive basically just removes a file from the allocation table when it is deleted. So, if you write to the drive again you could be writing to a cleared area or to an area that is not longer cleared. This happens below the unRAID control level so unRAID has no knowledge of this.

 

The reason to clear a new drive is that a drive with all zero data will not affect the parity calculation. If you create a brand new array and add say 5 data and the parity drive it will not clear the drives but rather just do a parity calculation. The clearing is only to add a new drive to an existing arrar without breaking parity.

 

Peter

henris

 

thank you for your benchmarking tips, it was very helpful as I am not a Linux guy

 

What I received is following

 

***READ ***

 

root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/dev/null

73184338+1 records in

73184338+1 records out

37470381528 bytes (37 GB) copied, 609.037 s, 61.5 MB/s

 

 

*** WRITE ***

 

root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/mnt/disk4/Movies/Rush.mkv

73184338+1 records in

73184338+1 records out

37470381528 bytes (37 GB) copied, 3073.51 s, 12.2 MB/s

 

Often "dd" performs better when some kind of block size is specified:

root@Tower:~# ########### READS ###########
root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches
root@Tower:~# dd if=/mnt/disk1/test/testfile of=/dev/null
1421820+0 records in
1421820+0 records out
727971840 bytes (728 MB) copied, 9.52896 s, 76.4 MB/s
root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches
root@Tower:~# dd if=/mnt/disk1/test/testfile of=/dev/null
1421820+0 records in
1421820+0 records out
727971840 bytes (728 MB) copied, 9.75895 s, 74.6 MB/s
root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches
root@Tower:~# dd if=/mnt/disk1/test/testfile of=/dev/null bs=1K
710910+0 records in
710910+0 records out
727971840 bytes (728 MB) copied, 7.08991 s, 103 MB/s
root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches
root@Tower:~# dd if=/mnt/disk1/test/testfile of=/dev/null bs=1K
710910+0 records in
710910+0 records out
727971840 bytes (728 MB) copied, 7.09987 s, 103 MB/s
root@Tower:~# ########### WRITES ###########
root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches
root@Tower:~# dd if=/mnt/disk1/test/testfile of=/mnt/disk4/test/file1
1421820+0 records in
1421820+0 records out
727971840 bytes (728 MB) copied, 55.6483 s, 13.1 MB/s
root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches
root@Tower:~# dd if=/mnt/disk1/test/testfile of=/mnt/disk4/test/file2
1421820+0 records in
1421820+0 records out
727971840 bytes (728 MB) copied, 55.118 s, 13.2 MB/s
root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches
root@Tower:~# dd if=/mnt/disk1/test/testfile of=/mnt/disk4/test/file3 bs=1K
2776+1 records in
2776+1 records out
727971840 bytes (728 MB) copied, 45.5878 s, 16.0 MB/s
root@Tower:~# sync ; echo 3 > /proc/sys/vm/drop_caches
root@Tower:~# dd if=/mnt/disk1/test/testfile of=/mnt/disk4/test/file4 bs=1k
694+1 records in
694+1 records out
727971840 bytes (728 MB) copied, 45.1853 s, 16.1 MB/s

 

Notice how the reads jumped from ~75 MB/s to ~100 MB/s?

Similar for the writes.

 

Purko

 

 

 

There is a serious performance boost (250%) in recent beta version (I do not see however that block size has any influence in my case)

 

VAULT login: root
Linux 2.6.31.5-unRAID.
root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/dev/null
73184338+1 records in
73184338+1 records out
37470381528 bytes (37 GB) copied, 644.946 s, 58.1 MB/s
root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/dev/null bs=1K
36592169+1 records in
36592169+1 records out
37470381528 bytes (37 GB) copied, 642.689 s, 58.3 MB/s
root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/dev/null bs=16K
2287010+1 records in
2287010+1 records out
37470381528 bytes (37 GB) copied, 644.15 s, 58.2 MB/s
root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/mnt/disk4/Movies/Rush.mkv
73184338+1 records in
73184338+1 records out
37470381528 bytes (37 GB) copied, 1309.42 s, 28.6 MB/s
root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/mnt/disk4/Movies/Rush2.mkv bs=1K
36592169+1 records in
36592169+1 records out
37470381528 bytes (37 GB) copied, 1766.9 s, 21.2 MB/s
root@VAULT:~#

 

 

 

 

 

 

There is a serious performance boost (250%) in recent beta version (I do not see however that block size has any influence in my case)

 

root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/mnt/disk4/Movies/Rush.mkv

73184338+1 records in

73184338+1 records out

37470381528 bytes (37 GB) copied, 1309.42 s, 28.6 MB/s

root@VAULT:~# dd if="/mnt/disk3/Movies/Music/Test.mkv" of=/mnt/disk4/Movies/Rush2.mkv bs=1K

36592169+1 records in

36592169+1 records out

37470381528 bytes (37 GB) copied, 1766.9 s, 21.2 MB/s

 

Surely that is 33% improvement in writes by specifying a 1k stripe?

 

Its more regression, than improvement

21MB/s vs 28MB/s