August 11, 200916 yr I've recently bought a Gigabyte ma74gm-s2 board, and I am getting aroung 50kbs/s parity check speeds. Comparing that to the chart of other configurations, using the same mobo, (where others have been getting 80kb/s) I am left wondering what I'm missing... Any ideas (if you're a happy owner of a Gigabyte ma74gm-s2)?
August 11, 200916 yr I assume you mean MB/s. I have this motherboard with 6 SATA drives on the mb and one on a PCI card. I typically get 55-60 MB/s average for a full parity check with peak speed of 75-80 MB/s. There are a lot of factors affecting parity check speed besides the motherboard. I don't think 50 MB/s is too bad but if it's your average speed and all the drives are SATA on the motherboard, then perhaps one or more drives is holding back the array's speed. You can check the throughput per drive by starting a console session and running: hdparm -tT. Make sure none of the drives have the 1.5 Gbps jumper installed. Check the SMART record to see if errors are showing up (perhaps a weak cable connection).
August 11, 200916 yr Author I actually meant KB/s I am doing another parity check now, and it's hovering around 47-53 KB/sec Should I be getting speeds in the MB/s range? if I do a hdparm -tT on my drives, I get around 74-78 MB/sec I am running 4 Segate SATA drives at 250 gigs each, and one Segate SATA at 500 gig (as parity), all on the motherboard SATA connectors.
August 11, 200916 yr Something is definitely wrong if you're only getting KB/s for your parity checks.
August 11, 200916 yr Author Pulling files over the gigabit network is just shy of 20MB/s (From UnRaid to my linux box over NFS) Pushing files over gigabit network is just shy of 10MB/s (To unRaid from linux box over NFS) Where do I start diagnosing what the problem is?
August 11, 200916 yr I actually meant KB/s I am doing another parity check now, and it's hovering around 47-53 KB/sec Should I be getting speeds in the MB/s range? if I do a hdparm -tT on my drives, I get around 74-78 MB/sec I am running 4 Segate SATA drives at 250 gigs each, and one Segate SATA at 500 gig (as parity), all on the motherboard SATA connectors. I get 50,000 kb/s to 60,000kb/s I do a monthly parity check and here is the message from my syslog. Jul 1 06:50:31 Tower kernel: md: sync done. time=24370sec rate=60120K/sec I would suggest looking for the md: sync done in your syslog and posting it so there is no confusion. I would also suggest posting the syslog here as one of your drives might be in PIO mode instead of DMA mode. This would slow everything down.
August 11, 200916 yr Your transfer speeds to and from the array don't look bad, not great but acceptable, so that indicates your networking and the parity drive and the drives you tested by reading and writing are OK. Since the parity check uses ALL drives, there must be another drive that is affecting the speeds so much. Try testing write speed to every one of your data drives. It may be helpful to Obtain a SMART report for each of your drives. But as he said, the syslog would be most helpful. Please see the Troubleshooting link in my sig, for help in capturing and posting it.
August 11, 200916 yr Lines which stand out. Looks like hdc is a PATA?? but model number says SATA. How is it wired on the motherboard? How is it configured on your bios? It seems to be setting it self to PIO4 instead of UDMA If this is truely PIO mode it will slow anything that includes access to this drive. do hdparm -iI /dev/hdc and post results. Also there seems to be another kernel issue. There may be an IRQ issue on the board. You may have to try one of the kernel options to tune how this is handled. I do not remember the kernel boot options, but I do remember on some boards I had to use them in dealing with the ethernet and IRQ issues. Aug 4 19:39:01 Tower kernel: hdc: ST3500320AS, ATA DISK drive Aug 4 19:39:01 Tower kernel: hdc: host max PIO4 wanted PIO255(auto-tune) selected PIO4 Aug 4 19:39:01 Tower kernel: hdc: UDMA/100 mode selected Aug 4 19:39:01 Tower kernel: hdc: max request size: 512KiB Aug 4 19:39:01 Tower kernel: hdc: 976773168 sectors (500107 MB), CHS=60801/255/63 Aug 4 19:39:01 Tower kernel: hdc: cache flushes supported Aug 4 19:39:01 Tower kernel: hdc: hdc1 Aug 5 09:44:18 Tower kernel: ------------[ cut here ]------------ Aug 5 09:44:18 Tower kernel: WARNING: at net/sched/sch_generic.c:226 dev_watchdog+0xf8/0x178() Aug 5 09:44:18 Tower kernel: Hardware name: GA-MA74GM-S2 Aug 5 09:44:18 Tower kernel: NETDEV WATCHDOG: eth0 (r8169): transmit timed out Aug 5 09:44:18 Tower kernel: Modules linked in: md_mod ide_gd_mod sata_promise atiixp ide_core ahci libata r8169 [last unloaded: md_mod] Aug 5 09:44:18 Tower kernel: Pid: 0, comm: swapper Not tainted 2.6.29.1-unRAID #2 Aug 5 09:44:18 Tower kernel: Call Trace: Aug 5 09:44:18 Tower kernel: [<c0120e07>] warn_slowpath+0x74/0x8a Aug 5 09:44:18 Tower kernel: [<c011a353>] ? enqueue_task_fair+0x96/0x9d Aug 5 09:44:18 Tower kernel: [<c01190ed>] ? enqueue_task+0xd/0x18 Aug 5 09:44:18 Tower kernel: [<c011c0fb>] ? try_to_wake_up+0x12b/0x136 Aug 5 09:44:18 Tower kernel: [<c011a2b5>] ? __enqueue_entity+0x9c/0xa4 Aug 5 09:44:18 Tower kernel: [<c011a2b5>] ? __enqueue_entity+0x9c/0xa4 Aug 5 09:44:18 Tower kernel: [<c022291b>] ? strlcpy+0x17/0x48 Aug 5 09:44:18 Tower kernel: [<c02e9283>] dev_watchdog+0xf8/0x178 Aug 5 09:44:18 Tower kernel: [<c0128542>] ? __mod_timer+0x91/0x9c Aug 5 09:44:18 Tower kernel: [<c012868c>] ? mod_timer+0x21/0x27 Aug 5 09:44:18 Tower kernel: [<c02e3a25>] ? neigh_periodic_timer+0x11f/0x130 Aug 5 09:44:18 Tower kernel: [<c02e918b>] ? dev_watchdog+0x0/0x178 Aug 5 09:44:18 Tower kernel: [<c01281af>] run_timer_softirq+0x105/0x158 Aug 5 09:44:18 Tower kernel: [<c0124a48>] __do_softirq+0x84/0x121 Aug 5 09:44:18 Tower kernel: [<c0124b1a>] do_softirq+0x35/0x3a Aug 5 09:44:18 Tower kernel: [<c0124d97>] irq_exit+0x38/0x3a Aug 5 09:44:18 Tower kernel: [<c0104a69>] do_IRQ+0x67/0x7e Aug 5 09:44:18 Tower kernel: [<c01033a7>] common_interrupt+0x27/0x2c Aug 5 09:44:18 Tower kernel: [<c0108068>] ? default_idle+0x2d/0x42 Aug 5 09:44:18 Tower kernel: [<c0108257>] c1e_idle+0xcb/0xd0 Aug 5 09:44:18 Tower kernel: [<c01019dd>] cpu_idle+0x50/0x64 Aug 5 09:44:18 Tower kernel: [<c0336813>] rest_init+0x53/0x55 Aug 5 09:44:18 Tower kernel: ---[ end trace e0a5c5348c17bea7 ]---
August 11, 200916 yr Author Lines which stand out. Looks like hdc is a PATA?? but model number says SATA. How is it wired on the motherboard? How is it configured on your bios? It seems to be setting it self to PIO4 instead of UDMA If this is truely PIO mode it will slow anything that includes access to this drive. do hdparm -iI /dev/hdc and post results. Ok Here's the results from the hdparm -iI on hdc This is my 500gig parity drive. Strange that it would register itself as PATA drive, since it is a SATA drive. I'll take a look at what the BIOS shows. # hdparm -iI /dev/hdc /dev/hdc: Model=ST3500320AS, FwRev=SD15, SerialNo=9QM99MWR Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% } RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4 BuffType=unknown, BuffSize=0kB, MaxMultSect=16, MultSect=16 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=976773168 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120} PIO modes: pio0 pio1 pio2 pio3 pio4 DMA modes: mdma0 mdma1 mdma2 UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5 udma6 AdvancedPM=no WriteCache=enabled Drive conforms to: unknown: ATA/ATAPI-4,5,6,7 * signifies the current active mode ATA device, with non-removable media Model Number: ST3500320AS Serial Number: 9QM99MWR Firmware Revision: SD15 Transport: Serial Standards: Used: unknown (minor revision code 0x0029) Supported: 8 7 6 5 Likely used: 8 Configuration: Logical max current cylinders 16383 16383 heads 16 16 sectors/track 63 63 -- CHS current addressable sectors: 16514064 LBA user addressable sectors: 268435455 LBA48 user addressable sectors: 976773168 device size with M = 1024*1024: 476940 MBytes device size with M = 1000*1000: 500107 MBytes (500 GB) Capabilities: LBA, IORDY(can be disabled) Queue depth: 32 Standby timer values: spec'd by Standard, no device specific minimum R/W multiple sector transfer: Max = 16 Current = 16 Recommended acoustic management value: 254, current value: 0 DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5 udma6 Cycle time: min=120ns recommended=120ns PIO: pio0 pio1 pio2 pio3 pio4 Cycle time: no flow control=120ns IORDY flow control=120ns Commands/features: Enabled Supported: * SMART feature set Security Mode feature set * Power Management feature set * Write cache * Look-ahead * Host Protected Area feature set * WRITE_BUFFER command * READ_BUFFER command * DOWNLOAD_MICROCODE SET_MAX security extension * 48-bit Address feature set * Device Configuration Overlay feature set * Mandatory FLUSH_CACHE * FLUSH_CACHE_EXT * SMART error logging * SMART self-test * General Purpose Logging feature set * 64-bit World wide name * Write-Read-Verify feature set * WRITE_UNCORRECTABLE_EXT command * {READ,WRITE}_DMA_EXT_GPL commands * SATA-I signaling speed (1.5Gb/s) * Native Command Queueing (NCQ) * Phy event counters * Software settings preservation * SMART Command Transport (SCT) feature set * SCT Long Sector Access (AC1) * SCT Error Recovery Control (AC3) * SCT Features Control (AC4) * SCT Data Tables (AC5) unknown 206[12] (vendor specific) Security: Master password revision code = 65534 supported not enabled not locked not frozen not expired: security count supported: enhanced erase 104min for SECURITY ERASE UNIT. 104min for ENHANCED SECURITY ERASE UNIT. Logical Unit WWN Device Identifier: 5000c50010a4b1a4 NAA : 5 IEEE OUI : c50 Unique ID : 010a4b1a4 Checksum: correct and doing a hdparm -tT shows very slow access speeds. # hdparm -tT /dev/hdc /dev/hdc: Timing cached reads: 978 MB in 2.00 seconds = 488.97 MB/sec Timing buffered disk reads: 24 MB in 3.10 seconds = 7.74 MB/sec
August 11, 200916 yr Ok Here's the results from the hdparm -iI on hdc This is my 500gig parity drive. Strange that it would register itself as PATA drive, since it is a SATA drive. I'll take a look at what the BIOS shows. hdparm shows it is in udma5. So that is not the issue. At least from what I can tell. and doing a hdparm -tT shows very slow access speeds. # hdparm -tT /dev/hdc /dev/hdc: Timing cached reads: 978 MB in 2.00 seconds = 488.97 MB/sec Timing buffered disk reads: 24 MB in 3.10 seconds = 7.74 MB/sec This is pretty slow. Anciently slow. There might be some form of ide emulation mode in effect. Notice how it was detected as /dev/hdc and not an /dev/sd? device. is it in some kind of raid compatible mode? http://www.gigabyte.com.tw/Products/Motherboard/Products_Spec.aspx?ClassValue=Motherboard&ProductID=2813&ProductName=GA-MA74GM-S2 Aug 4 19:39:01 Tower kernel: ata1: SATA max UDMA/133 abar m1024@0xfe02f000 port 0xfe02f100 irq 22 Aug 4 19:39:01 Tower kernel: ata2: SATA max UDMA/133 abar m1024@0xfe02f000 port 0xfe02f180 irq 22 Aug 4 19:39:01 Tower kernel: ata3: SATA max UDMA/133 abar m1024@0xfe02f000 port 0xfe02f200 irq 22 Aug 4 19:39:01 Tower kernel: ata4: SATA max UDMA/133 abar m1024@0xfe02f000 port 0xfe02f280 irq 22 ... Aug 4 19:39:01 Tower kernel: atiixp 0000:00:14.1: IDE controller (0x1002:0x439c rev 0x00) Aug 4 19:39:01 Tower kernel: ATIIXP_IDE 0000:00:14.1: PCI INT A -> GSI 16 (level, low) -> IRQ 16 Aug 4 19:39:01 Tower kernel: atiixp 0000:00:14.1: not 100%% native mode: will probe irqs later Aug 4 19:39:01 Tower kernel: ide0: BM-DMA at 0xfa00-0xfa07 Aug 4 19:39:01 Tower kernel: ide1: BM-DMA at 0xfa08-0xfa0f Aug 4 19:39:01 Tower kernel: Probing IDE interface ide0... Aug 4 19:39:01 Tower kernel: Probing IDE interface ide1... Aug 4 19:39:01 Tower kernel: hdc: ST3500320AS, ATA DISK drive ... Aug 4 19:39:01 Tower kernel: hdc: host max PIO4 wanted PIO255(auto-tune) selected PIO4 Aug 4 19:39:01 Tower kernel: hdc: UDMA/100 mode selected ... Aug 4 19:39:01 Tower kernel: sata_promise 0000:03:06.0: version 2.12 Aug 4 19:39:01 Tower kernel: sata_promise 0000:03:06.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20 Aug 4 19:39:01 Tower kernel: scsi5 : sata_promise Aug 4 19:39:01 Tower kernel: scsi6 : sata_promise Aug 4 19:39:01 Tower kernel: scsi7 : sata_promise Aug 4 19:39:01 Tower kernel: scsi8 : sata_promise Aug 4 19:39:01 Tower kernel: ata5: SATA max UDMA/133 mmio m4096@0xfdcff000 ata 0xfdcff380 irq 20 Aug 4 19:39:01 Tower kernel: ata6: SATA max UDMA/133 mmio m4096@0xfdcff000 ata 0xfdcff280 irq 20 Aug 4 19:39:01 Tower kernel: ata7: SATA max UDMA/133 mmio m4096@0xfdcff000 ata 0xfdcff200 irq 20 Aug 4 19:39:01 Tower kernel: ata8: SATA max UDMA/133 mmio m4096@0xfdcff000 ata 0xfdcff300 irq 20 Aug 4 19:39:01 Tower kernel: ata5: SATA link down (SStatus 0 SControl 300) Aug 4 19:39:01 Tower kernel: ata6: SATA link down (SStatus 0 SControl 300) Aug 4 19:39:01 Tower kernel: ata7: SATA link down (SStatus 0 SControl 300) Aug 4 19:39:01 Tower kernel: ata8: SATA link down (SStatus 0 SControl 300) Aug 4 19:39:01 Tower kernel: ide-gd driver 1.18 Aug 4 19:39:01 Tower kernel: hdc: max request size: 512KiB Aug 4 19:39:01 Tower kernel: hdc: 976773168 sectors (500107 MB), CHS=60801/255/63 Aug 4 19:39:01 Tower kernel: hdc: cache flushes supported Aug 4 19:39:01 Tower kernel: hdc: hdc1 I notice that it is initialized after the first 4 250gb drives. where on the motherboard is the parity drive? does it have ahci enabled in the bios? Also noticed that there is a promise controller in your machine? (mobo?) Have you tried switching this drive to that controller?
August 11, 200916 yr Author This is pretty slow. Anciently slow. There might be some form of ide emulation mode in effect. Notice how it was detected as /dev/hdc and not an /dev/sd? device. is it in some kind of raid compatible mode? Not to my knowledge. And I don't know how it got detected as /dev/hdc as opposed to /dev/sdx I notice that it is initialized after the first 4 250gb drives. where on the motherboard is the parity drive? does it have ahci enabled in the bios? Also noticed that there is a promise controller in your machine? (mobo?) Have you tried switching this drive to that controller? This drive (500 gig) is on one of the six SATA mobo connectors. I have not checked if it has ahci enabled. I will when I get home tonight. The promise controller is left over from earlier experiments on getting faster speeds. It's the TX4 controller. I have tried moving only one of the 250gig drives to that controller in an effort to see if it sped up the parity building... It made no difference. I will try the parity (500 gig) drive on that controller.
August 11, 200916 yr It's really great to see another experienced user tackling user syslogs! Concerning the PIO/UDMA thing, I've noticed that both are set up initially, and a best mode selected for each, but then the better one (the UDMA mode of course) is actually 'activated'. The SATA drivers have been suppressing this info more than the older IDE drivers. (I know that technically they aren't really 'drivers', but it is easier to refer to them that way for many users.) Aug 5 09:44:18 Tower kernel: NETDEV WATCHDOG: eth0 (r8169): transmit timed out The issue with this 'cut' section, including the line above and the subsequent 'Call Trace', has occurred 3 to 5 times before, and is always related to the Realtek chipset and r8169 module. If you see it once, you will probably see it again, and it will be very disruptive. Here is one sample, and you can search watchdog r8169 for more. I believe the best advice has been to purchase an Intel PRO/1000 card and disable the onboard Realtek NIC. Since you have a Promise card on the PCI bus, it would be better to purchase a PCI Express version.
August 11, 200916 yr Author The issue with this 'cut' section, including the line above and the subsequent 'Call Trace', has occurred 3 to 5 times before, and is always related to the Realtek chipset and r8169 module. If you see it once, you will probably see it again, and it will be very disruptive. What does "very disruptive" mean?
August 11, 200916 yr This is a screen-capture of the example bios screen from the manual on your MB that is used to set up the SATA controllers. The settings I highlighted in RED are probably what you want to put into place. Notice they are NOT the defaults. Also note the default is to operate in "native IDE mode" (you can probably translate that to be POLLED I/O IDE emulation... and slow...) I'd check the BIOS entries when you get home and change them from the defaults. Odds are you will find your parity calc speeds vastly different. Oh yes, before you do anything, make sure you note which drive is which (print a screen-shot of your devices page) because when you change the BIOS, odds are very high the disks will end up on different "devices" and you will need to re-assign them to their correct slots in the array before it will start. Once you have them assigned properly, the "Start" button should get up up and on-line without having to re-calculate parity again. Joe L.
August 11, 200916 yr The issue with this 'cut' section, including the line above and the subsequent 'Call Trace', has occurred 3 to 5 times before, and is always related to the Realtek chipset and r8169 module. If you see it once, you will probably see it again, and it will be very disruptive. What does "very disruptive" mean? You will have a temporary loss of networking each time, possibly affecting file transfers, more visibly affecting media streaming, as stutter. Longer stoppages could cause timeouts resulting in cancellations of transfers and streams, but I don't know enough about this problem, to know how badly affected the users are that are experiencing it. I don't believe you will have data corruption, just possible delays and dropouts in playback. I believe it is more likely in periods of heavier network traffic. Affected users could say more, if they see this.
August 12, 200916 yr Author This is a screen-capture of the example bios screen from the manual on your MB that is used to set up the SATA controllers. The settings I highlighted in RED are probably what you want to put into place. Notice they are NOT the defaults. Also note the default is to operate in "native IDE mode" (you can probably translate that to be POLLED I/O IDE emulation... and slow...) I'd check the BIOS entries when you get home and change them from the defaults. Odds are you will find your parity calc speeds vastly different. Oh yes, before you do anything, make sure you note which drive is which (print a screen-shot of your devices page) because when you change the BIOS, odds are very high the disks will end up on different "devices" and you will need to re-assign them to their correct slots in the array before it will start. Once you have them assigned properly, the "Start" button should get up up and on-line without having to re-calculate parity again. Joe L. Thanks for the advice. Made the changes you mentioned in the BIOS, and the parity drive which was running at 7MB/s now does 101MB/s (when tested with hdparm -tT) Now I feel kinda stupid by mentioning this, but the numbers I was quoting for parity speed were in fact KB/sec, however for some reason I was dropping the digits after the coma... I think I assumed it was a decimal point. So in fact I was getting 50,000 KB/s before... Now I'm getting as high as 75,000 KB/s for parity check. You will have a temporary loss of networking each time, possibly affecting file transfers, more visibly affecting media streaming, as stutter. Longer stoppages could cause timeouts resulting in cancellations of transfers and streams, but I don't know enough about this problem, to know how badly affected the users are that are experiencing it. I don't believe you will have data corruption, just possible delays and dropouts in playback. I believe it is more likely in periods of heavier network traffic. Affected users could say more, if they see this. I had noticed that when copying large amounts of files, the KDE progress indicator would occasionally tell me the transfer was stalled. It would resume shortly after, but this would happen repeatedly throughout a transfer of files... I will look into getting an Intel 1000 NIC to remedy this issue. I have attached a new copy of the syslog with everything (the drive part anyway) working better.
August 12, 200916 yr Thanks for the advice. Made the changes you mentioned in the BIOS, and the parity drive which was running at 7MB/s now does 101MB/s (when tested with hdparm -tT) Now I feel kinda stupid by mentioning this, but the numbers I was quoting for parity speed were in fact KB/sec, however for some reason I was dropping the digits after the coma... I think I assumed it was a decimal point. So in fact I was getting 50,000 KB/s before... Now I'm getting as high as 75,000 KB/s for parity check. I knew this, which is why I wanted to see your syslog and md sync done message. It was still low anyway. You will have a temporary loss of networking each time, possibly affecting file transfers, more visibly affecting media streaming, as stutter. Longer stoppages could cause timeouts resulting in cancellations of transfers and streams, but I don't know enough about this problem, to know how badly affected the users are that are experiencing it. I don't believe you will have data corruption, just possible delays and dropouts in playback. I believe it is more likely in periods of heavier network traffic. Affected users could say more, if they see this. I had noticed that when copying large amounts of files, the KDE progress indicator would occasionally tell me the transfer was stalled. It would resume shortly after, but this would happen repeatedly throughout a transfer of files... I will look into getting an Intel 1000 NIC to remedy this issue. The pauses are normal if you are sending very large files. You will see a burst of traffic, 20-40MB/s.. Then a pause while the cache is being flushed and parity updated, Then it will continue again. There is only so much data that can be in memory before it is forced to be written to disk. When you do this test again, watch the lights on the network vs the disk. If the disk lights go solid while the network is pausing then this is the issue. You can also load a program called vnstat to monitor the network over a telnet session. If you telnet session is hanging for a long time while the transfer pauses, then the issue could be the network card/driver.
Archived
This topic is now archived and is closed to further replies.