Just a quick stupid question 

 

I currently have all 1TB data drives.

I replaced one those for a 3TB drive and it's currently rebuilding.

Will the rebuild speed up once it's past the 1TB mark? Since there should just be 0's now..

 

Cheers

No, it still "computes" the values for every byte, even though the outcome is obvious.

 

It can, not because it's only zeros, but if your system has a CPU or controller bottleneck with all disks, it won't have (or have less) a bottleneck with just 2 disks.

No, it still "computes" the values for every byte, even though the outcome is obvious.

 

Thanks, that's what I was afraid of... 

True ... hadn't considered that you may have so many disks that your controller could be slowing things down.  If that's the case, as Johnnie noted the rebuild will speed up because that bottleneck will disappear.

 

It can, not because it's only zeros, but if your system has a CPU or controller bottleneck with all disks, it won't have (or have less) a bottleneck with just 2 disks.

 

The AOC-SAT2-MV8's should be fine with all the disks, but I'll keep an eye on it and will see if it improves..

Just out of curiosity what's your current rebuild speed, when it's reading all disks?

It can, not because it's only zeros, but if your system has a CPU or controller bottleneck with all disks, it won't have (or have less) a bottleneck with just 2 disks.

 

The AOC-SAT2-MV8's should be fine with all the disks, but I'll keep an eye on it and will see if it improves..

 

Agree -- the PCI-X slots should give you at least 800MB/s of bandwidth for the MV8's (depends on whether you've got them plugged in to a 100MHz or 133MHz slot) ... and I'd assume your 1TB drives have relatively low platter densities (certainly not the 1TB density of modern drives), so that's probably able to maintain full speed on those disks.  If not, it still shouldn't be too much of a bottleneck.

 

Just out of curiosity what's your current rebuild speed, when it's reading all disks?

 

on the main unraid screen it's reporting 82.5MB/s at position 331GB.

the dynamics stats plugin is reporting 737.7 MB/s read and 82.3 MB/s write

 

That speed seems reasonable for what I'd expect the platter densities to be on your older 1TB drives (probably 500MB).    And it's below what the MV8's should be capable of, so they're almost certainly NOT a bottleneck.

 

Given that, I would not expect to see any change after the 1TB point, as your Core 2 Quad CPU should definitely NOT be a bottleneck either.

 

That speed seems reasonable for what I'd expect the platter densities to be on your older 1TB drives (probably 500MB).    And it's below what the MV8's should be capable of, so they're almost certainly NOT a bottleneck.

 

Given that, I would not expect to see any change after the 1TB point, as your Core 2 Quad CPU should definitely NOT be a bottleneck either.

 

I am going to replace all the 1TB drives with 3TB WD Red's at a rate of about 2 every 2 weeks. Once they are all gone I'll add another one for the 2nd parity...

Note that once you have all WD Reds the MV8's WILL be a bottleneck in parity checks and rebuilds, since the Reds have 1TB platters, and can easily hit transfer rates in the 150MB/s range ... well above what an MV8 can sustain for 8 drives.    Won't be a problem in normal use, as all of the attached drives won't be transferring data; but during rebuilds or parity checks the MV8's will indeed be slowing things down a bit.

 

Probably not a big deal -- I wouldn't change out your controllers just for that, but just wanted your expectations to be realistic ... when you do parity checks with an all-3TB Red system your times will be a bit slower than what others are reporting -- and the reason is the MV8s.

 

on the main unraid screen it's reporting 82.5MB/s at position 331GB.

 

That's exactly the max speed of the SAT2 in a 100Mhz slot with 8 disks, so you are probably hitting a bottleneck, though like Gary pointed out, that's a reasonable speed for older disks and it probably wouldn't be much higher with a faster controller.

 

Even so, speed will increase noticeably after the 1TB mark, because the 3Tb disks are much faster at that position, should speed up to about 125MB/s, then it will start to slowly decrease again as it goes through the inner slower sectors, and should finish at about 80MB/s again.

Note that once you have all WD Reds the MV8's WILL be a bottleneck in parity checks and rebuilds, since the Reds have 1TB platters, and can easily hit transfer rates in the 150MB/s range ... well above what an MV8 can sustain for 8 drives.    Won't be a problem in normal use, as all of the attached drives won't be transferring data; but during rebuilds or parity checks the MV8's will indeed be slowing things down a bit.

 

Probably not a big deal -- I wouldn't change out your controllers just for that, but just wanted your expectations to be realistic ... when you do parity checks with an all-3TB Red system your times will be a bit slower than what others are reporting -- and the reason is the MV8s.

 

Since I have 2 of the MV8's, I will spread them equally between the 2 controllers as I'm not up to that many drives yet

on the main unraid screen it's reporting 82.5MB/s at position 331GB.

 

That's exactly the max speed of the SAT2 in a 100Mhz slot with 8 disks, so you are probably hitting a bottleneck, though like Gary pointed out, that's a reasonable speed for older disks and it probably wouldn't be much higher with a faster controller.

 

Even so, speed will increase noticeably after the 1TB mark, because the 3Tb disks are much faster at that position, should speed up to about 125MB/s, then it will start to slowly decrease again as it goes through the inner slower sectors, and should finish at about 80MB/s again.

 

Yes, I just moved one of the MV8's to the 100Mhz slot to see if that would make a difference.

I have 2 133 and 2 100 Mhz slots that apparently each share a bus.. I'll put both back on the 133Mhz slots after to see if there is any improvement..

IIRC performance is worse if both are on the 133Mhz slots, because like you said they share the bus, but I never tested that, I have one of those boards but only one SAT2 controller.

IIRC performance is worse if both are on the 133Mhz slots, because like you said they share the bus, but I never tested that, I have one of those boards but only one SAT2 controller.

 

If I am reading this correctly, then they are both currently running at 133Mhz? (I have the speed set to auto for slots 5/6 and 1/2 in the BIOS)

 

Output from: lspci -vv

 

02:01.0 SCSI storage controller: Marvell Technology Group Ltd. MV88SX6081 8-port SATA II PCI-X Controller (rev 09)
        Subsystem: Marvell Technology Group Ltd. MV88SX6081 8-port SATA II PCI-X Controller
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr+ Stepping- SERR+ FastB2B+ DisINTx-
        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 32, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 16
        Region 0: Memory at d8200000 (64-bit, non-prefetchable) [size=1M]
        Region 2: I/O ports at 2000 [size=256]
        Capabilities: [40] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [60] PCI-X non-bridge device
                Command: DPERE- ERO- RBC=512 OST=4
                Status: Dev=02:01.0 64bit+ 133MHz+ SCD- USC- DC=simple DMMRBC=512 DMOST=4 DMCRS=8 RSCEM- 266MHz- 533MHz-
        Kernel driver in use: sata_mv
        Kernel modules: sata_mv

03:01.0 SCSI storage controller: Marvell Technology Group Ltd. MV88SX6081 8-port SATA II PCI-X Controller (rev 09)
        Subsystem: Marvell Technology Group Ltd. MV88SX6081 8-port SATA II PCI-X Controller
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr+ Stepping- SERR+ FastB2B+ DisINTx-
        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 32, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 16
        Region 0: Memory at d8c00000 (64-bit, non-prefetchable) [size=1M]
        Region 2: I/O ports at 3000 [size=256]
        Capabilities: [40] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [60] PCI-X non-bridge device
                Command: DPERE- ERO- RBC=512 OST=4
                Status: Dev=03:01.0 64bit+ 133MHz+ SCD- USC- DC=simple DMMRBC=512 DMOST=4 DMCRS=8 RSCEM- 266MHz- 533MHz-
        Kernel driver in use: sata_mv
        Kernel modules: sata_mv

Note that once you have all WD Reds the MV8's WILL be a bottleneck in parity checks and rebuilds, since the Reds have 1TB platters, and can easily hit transfer rates in the 150MB/s range ... well above what an MV8 can sustain for 8 drives.    Won't be a problem in normal use, as all of the attached drives won't be transferring data; but during rebuilds or parity checks the MV8's will indeed be slowing things down a bit.

 

Probably not a big deal -- I wouldn't change out your controllers just for that, but just wanted your expectations to be realistic ... when you do parity checks with an all-3TB Red system your times will be a bit slower than what others are reporting -- and the reason is the MV8s.

 

Since I have 2 of the MV8's, I will spread them equally between the 2 controllers as I'm not up to that many drives yet

 

That won't matter as much as you might think -- as long as you have ANY of the older 1TB drives (even just one) the speed of your rebuilds/parity checks will be limited by that drive.    Of course once you get past the 1TB point, then it will make a difference, because the MV8 won't be a bottleneck unless it has more than 5 of the 3TB drives attached to it.

 

If I am reading this correctly, then they are both currently running at 133Mhz? (I have the speed set to auto for slots 5/6 and 1/2 in the BIOS)

 

Don't know how to interpret those, but AFAIK the bottom slots only work at 66 or 100Mhz max, I tested with a single SAT2 with bios set to auto and it worked at 133Mhz in one of the top slots and 100Mhz in one of the bottom slots.

If I am reading this correctly, then they are both currently running at 133Mhz? (I have the speed set to auto for slots 5/6 and 1/2 in the BIOS)

 

Don't know how to interpret those, but AFAIK the bottom slots only work at 66 or 100Mhz max, I tested with a single SAT2 with bios set to auto and it worked at 133Mhz in one of the top slots and 100Mhz in one of the bottom slots.

 

I believe this is the important part:

 

Capabilities: [60] PCI-X non-bridge device
                Command: DPERE- ERO- RBC=512 OST=4
                Status: Dev=02:01.0 64bit+ 133MHz+ SCD- USC- DC=simple DMMRBC=512 DMOST=4 DMCRS=8 RSCEM- 266MHz- 533MHz-

 

And it shows 133Mhz for both adapters.. maybe someone else can chime in here..

Not sure why it's reporting that, but it's clear from the motherboard manual that the top 2 slots are 133MHz and the bottom 2 slots are 100MHz.  It's VERY unlikely they're operating at a different speed than that 

Not sure why it's reporting that, but it's clear from the motherboard manual that the top 2 slots are 133MHz and the bottom 2 slots are 100MHz.  It's VERY unlikely they're operating at a different speed than that 

 

I was surprised as well.. But in the BIOS I can select 133/100/Auto for the top and bottom slots. It's currently in Auto and I don't have anything else in any of the other slots..

 

Do you have your cards split between the two sockets?  i.e. one in one of the top slots and one in a bottom slot?

 

Do you have your cards split between the two sockets?  i.e. one in one of the top slots and one in a bottom slot?

 

Yes, that's how it is currently set up

I was surprised as well.. But in the BIOS I can select 133/100/Auto for the top and bottom slots. It's currently in Auto and I don't have anything else in any of the other slots..

 

That is strange and it made me curious, but after a quick test I can confirm that the bottom slots work at 100Mhz, regardless of the bios setting (or what lspci reports, if that's what it is reporting).

 

SAT2 on the bottom slots has a max usable bandwidth of ~670MB/s, on the top slots has ~880MB/s, divide that by the number of disks you have connected and that will be your max parity check speed.