Slow Write Speeds? SuperMicro AOC-SASLP-MV8 & 5.0-rc6-r8168

[lungnut]

I've recently upgraded my server to make the jump to 3TB drives. Here's my new hardware:

 

  Motherboard: SUPERMICRO MBD-X9SCM-F-O

  CPU: Intel Core i3-2120 3.3GHz

  RAM: Kingston 4GB DDR3 1333*

  SATA Controllers: SUPERMICRO AOC-SASLP-MV8 (X 2)

  Case: Norco 4220

  *This is a single stick right now. I bought it thinking I was buying 2 X 2GB but received a single stick instead. I will be buying a matching 4GB stick soon.

 

After some initial bumps, I'm up and running. I first replaced my 2TB Parity drive with a 3TB drive. Then successfully added two new 3TB data drives and one 2TB drive (the old parity drive) to the array.

 

The first order of business was to transfer 1.5TB worth of data. While the transfer was ultimately successful, I only managed 13MB/s write speeds. Now, it's important to note that I am not using a cache drive at the moment (I have some very large transfers to make over the next few days), but I still expected higher write speeds than this. My read speeds are better, but still slower than I expected (I copied a 38GB file from my server at roughly 30MB/s).

 

I am attaching my syslog and I would love some help. I know it's probably a BIOS setting or something relatively simple, but I really need some guidance as I'm not a super-savvy tech person.

 

Thanks so much!

 

--------------------

EDIT: I edited the title to reflect that the issue is much more likely with the controller cards (AOC-SASLP-MV8) than the motherboard itself.

lungnut_syslog.txt

[Joe L.]

nothing looked odd.  There are no BIOS settings for network speed.  The connection that was established was at 1000MB/s.

 

Aug 16 20:16:18 Tower kernel: e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: Rx/Tx

Were you performing the initial parity calc at the same time as the transfers?  (that would slow things down a lot)

Is the source PC also 1000MB/s?    Are you going through a 100MB/s router? (or switch)

 

Joe L.

[bcbgboy13]

I suspect it could be the combination of the Unraid version you use (underlying kernels) and the two SASLP cards present. Some combinations do not work very well.

 

Second it depend on how you transferred this data - directly to a disk or to the user share. In the second case you may be using your older drives and if they are almost full the speed will be affected.

 

Third - five of your older 1.5TB Seagate HDs have "bad" firmware:

 

Aug 16 20:16:15 Tower kernel: ata7.00: WARNING: device requires firmware update to be fully functional

Aug 16 20:16:15 Tower kernel: ata7.00:          contact the vendor or visit http://ata.wiki.kernel.org

 

This may cause problems down the road - check their SMART status carefully.

[lungnut]

nothing looked odd.  There are no BIOS settings for network speed.  The connection that was established was at 1000MB/s.

 

Aug 16 20:16:18 Tower kernel: e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: Rx/Tx

Were you performing the initial parity calc at the same time as the transfers?  (that would slow things down a lot)

Is the source PC also 1000MB/s?    Are you going through a 100MB/s router? (or switch)

 

Joe L.

 

My network is all 1000MB/s, and parity had completed.

 

--------------------

EDIT: I edited the title to reflect that the issue is much more likely with the controller cards (AOC-SASLP-MV8) than the motherboard itself.

[lungnut]

I suspect it could be the combination of the Unraid version you use (underlying kernels) and the two SASLP cards present. Some combinations do not work very well.

 

Hmm...I definitely don't mind trying different versions, but two questions: 1) Are there any particular previous versions I should try and/or avoid? 2) Do you think that by the time v5 goes "final" this particular issue will be resolved? (It seems these cards are very popular with the community).

 

Second it depend on how you transferred this data - directly to a disk or to the user share. In the second case you may be using your older drives and if they are almost full the speed will be affected.

 

I definitely use user shares, but for these particular transfers, I went direct to one of the new disks (using the user share folder as a destination).

 

Third - five of your older 1.5TB Seagate HDs have "bad" firmware:

 

Aug 16 20:16:15 Tower kernel: ata7.00: WARNING: device requires firmware update to be fully functional

Aug 16 20:16:15 Tower kernel: ata7.00:          contact the vendor or visit http://ata.wiki.kernel.org

 

This may cause problems down the road - check their SMART status carefully.

 

Thanks for pointing this out. I will keep an eye on these (they have been in my system for quite a while at this point), and I will earmark them for "first to be replaced".

 

--------------------

EDIT: I edited the title to reflect that the issue is much more likely with the controller cards (AOC-SASLP-MV8) than the motherboard itself.

[Helmonder]

Well.... we have largely the same setup and I agree you should be getting higher write speeds:

 

http://lime-technology.com/forum/index.php?topic=21989.msg195633#msg195633

 

35 Bytes/per second is what I am getting when writing to a user share without cache drive and there are similar results with others..

 

Motherboard and extender card are not the issue, I am using the same... You have two SASLP cards, if that is the issue then I am interested because I will need expansion some time next year..

 

It was suggested drives might be the cause... Make a testing user share that only has rights for your newest, greatest and emptiest drive... If that gives the same results then drives are not the issue either..

 

Also try again in a day or so... Just to make sure that your new setup was not working in the background (scanning your itunes library or something like that)..

 

 

[dgaschk]

Quote

Third - five of your older 1.5TB Seagate HDs have "bad" firmware:

 

Aug 16 20:16:15 Tower kernel: ata7.00: WARNING: device requires firmware update to be fully functional

Aug 16 20:16:15 Tower kernel: ata7.00:          contact the vendor or visit http://ata.wiki.kernel.org

 

This may cause problems down the road - check their SMART status carefully.

 

Thanks for pointing this out. I will keep an eye on these (they have been in my system for quite a while at this point), and I will earmark them for "first to be replaced".

 

This may be causing the performance issue.

[lungnut]

Quote

Third - five of your older 1.5TB Seagate HDs have "bad" firmware:

 

Aug 16 20:16:15 Tower kernel: ata7.00: WARNING: device requires firmware update to be fully functional

Aug 16 20:16:15 Tower kernel: ata7.00:          contact the vendor or visit http://ata.wiki.kernel.org

 

This may cause problems down the road - check their SMART status carefully.

 

Thanks for pointing this out. I will keep an eye on these (they have been in my system for quite a while at this point), and I will earmark them for "first to be replaced".

 

This may be causing the performance issue.

 

Even if I wasn't writing to them? I was writing directly to one of my brand new WD Green 3TB drives.

 

--------------------

EDIT: I edited the title to reflect that the issue is much more likely with the controller cards (AOC-SASLP-MV8) than the motherboard itself.

[Helmonder]

And the parity drive ?  Is that one ok ?

[lungnut]

Well.... we have largely the same setup and I agree you should be getting higher write speeds:

 

http://lime-technology.com/forum/index.php?topic=21989.msg195633#msg195633

 

35 Bytes/per second is what I am getting when writing to a user share without cache drive and there are similar results with others..

 

Motherboard and extender card are not the issue, I am using the same... You have two SASLP cards, if that is the issue then I am interested because I will need expansion some time next year..

 

It was suggested drives might be the cause... Make a testing user share that only has rights for your newest, greatest and emptiest drive... If that gives the same results then drives are not the issue either..

 

I was writing directly to one of those said "newest, greatest and emptiest drives." But, in thinking of the way that my system is setup, I realized that the hard drive that I was writing to is connected to one of my SASLP cards, whereas the other new 3TB drive is connected to my motherboard's SATA port. I will try that drive for my next transfer and see if I get a better result (SASLP Drive vs. Mobo Drive).

 

Also try again in a day or so... Just to make sure that your new setup was not working in the background (scanning your itunes library or something like that)..

 

The system was definitely not being used in any other way other than the transfer (I can see the read/write status of each of the drives in the WebUI)

 

--------------------

EDIT: I edited the title to reflect that the issue is much more likely with the controller cards (AOC-SASLP-MV8) than the motherboard itself.

 

 

[lungnut]

And the parity drive ?  Is that one ok ?

 

Sorry if I'm being dense, but what do you mean is it okay? It's green-balled in the WebUI, and it is a brand-new WD Green 3TB Drive (obviously not the fastest drive around, but should be plenty fast enough for Unraid, yeah?).

 

--------------------

EDIT: I edited the title to reflect that the issue is much more likely with the controller cards (AOC-SASLP-MV8) than the motherboard itself.

[lungnut]

Update to my speed issue, the problem does seem to be with my SASLP cards. If I write to a drive on one of those cards, my write speeds are the intial reported 13MB/s. If I write to a drive on my mobo's SATA port, I am getting 28MB/s, which isn't fantastic, but certainly better. And this is a pretty fair test, as the drives in question are both WD Green 3TB drives, brand new and (initially) empty.

 

The good news is that when I do add my cache drive, it will also be connected to a mobo port (as is my parity drive). But I would like to resolve this issue with the SASLP cards. As another user suggested, I will try previous versions of v5.

 

--------------------

EDIT: I edited the title to reflect that the issue is much more likely with the controller cards (AOC-SASLP-MV8) than the motherboard itself.

[Helmonder]

It is specifically with -multiple- cards then... I do not experience this issue on the same setup with one card.

[mvdzwaan]

Update to my speed issue, the problem does seem to be with my SASLP cards. If I write to a drive on one of those cards, my write speeds are the intial reported 13MB/s. If I write to a drive on my mobo's SATA port, I am getting 28MB/s, which isn't fantastic, but certainly better. And this is a pretty fair test, as the drives in question are both WD Green 3TB drives, brand new and (initially) empty.

 

Which firmware are you running on your SASLP card ?

[lungnut]

Update to my speed issue, the problem does seem to be with my SASLP cards. If I write to a drive on one of those cards, my write speeds are the intial reported 13MB/s. If I write to a drive on my mobo's SATA port, I am getting 28MB/s, which isn't fantastic, but certainly better. And this is a pretty fair test, as the drives in question are both WD Green 3TB drives, brand new and (initially) empty.

 

Which firmware are you running on your SASLP card ?

 

They came with .21

[mvdzwaan]

Update to my speed issue, the problem does seem to be with my SASLP cards. If I write to a drive on one of those cards, my write speeds are the intial reported 13MB/s. If I write to a drive on my mobo's SATA port, I am getting 28MB/s, which isn't fantastic, but certainly better. And this is a pretty fair test, as the drives in question are both WD Green 3TB drives, brand new and (initially) empty.

 

Which firmware are you running on your SASLP card ?

 

They came with .21

 

That should be ok. The .15 did not officially support 3tb

[lungnut]

As I said I would, I am now writing 107GB of data to a 2TB drive with 500+ GB of free space using the other SASLP card. This transfer is going just 7.5MB/s.

 

So in summary, I have two fully-populated AOC-SASLP-MV8's (.21 firmware):

• Transfers to a hard drive on one of the SASLP's measure 13MB/s (a brand new 3TB drive)
  
• Transfers to a hard drive on the other SASLP measure 7.5MB/s (a 2TB drive with 500GB of free space)
  
• Transfers to a hard drive on the mobo's SATA ports measure 28MB/s (a brand new 3TB drive)
  

After the current transfer completes, I suppose I will try another iteration of Unraid v5 and see if that helps, as suggested by bcbgboy13. I will definitely update this thread whenever I do.

 

In the meantime--does anyone have any suggestions as to which former 5.0 I should use?

 

Thanks!

 

[Joe L.]

As I said I would, I am now writing 107GB of data to a 2TB drive with 500+ GB of free space using the other SASLP card. This transfer is going just 7.5MB/s.

 

So in summary, I have two fully-populated AOC-SASLP-MV8's (.21 firmware):

• Transfers to a hard drive on one of the SASLP's measure 13MB/s (a brand new 3TB drive)
  
• Transfers to a hard drive on the other SASLP measure 7.5MB/s (a 2TB drive with 500GB of free space)
  
• Transfers to a hard drive on the mobo's SATA ports measure 28MB/s (a brand new 3TB drive)
  

After the current transfer completes, I suppose I will try another iteration of Unraid v5 and see if that helps, as suggested by bcbgboy13. I will definitely update this thread whenever I do.

 

In the meantime--does anyone have any suggestions as to which former 5.0 I should use?

 

Thanks!

Write speeds to a mostly full drive vs. a mostly empty drive will be vastly different.  outer cylinders on a mostly empty drive are written to much faster than inner cylinders on a mostly full drive. It can easily be a 2-to-1 difference in write speed.

 

Joe L.

[lungnut]

Write speeds to a mostly full drive vs. a mostly empty drive will be vastly different.  outer cylinders on a mostly empty drive are written to much faster than inner cylinders on a mostly full drive. It can easily be a 2-to-1 difference in write speed.

 

Joe L.

 

Makes sense, and since 7.5MB/s and 13MB/s falls within your 2:1 ratio, my SASLP's are probably both functioning equally poorly.

[lungnut]

So, I have switched from 5.0-rc6-r8168 to 5.0-rc5 and have almost doubled my write speed in at least one of my test scenarios. I can't offer any explanations as to why because I am far from adept at these types of things. I will say this though: despite the 100% improvement, I think my system is capable of being faster still.

 

Here is a little more detail:

 

As I said I would, I am now writing 107GB of data to a 2TB drive with 500+ GB of free space using the other SASLP card. This transfer is going just 7.5MB/s.

 

So in summary, I have two fully-populated AOC-SASLP-MV8's (.21 firmware):

• Transfers to a hard drive on one of the SASLP's measure 13MB/s (a brand new 3TB drive)
  
• Transfers to a hard drive on the other SASLP measure 7.5MB/s (a 2TB drive with 500GB of free space)
  
• Transfers to a hard drive on the mobo's SATA ports measure 28MB/s (a brand new 3TB drive)
  

 

The first test (the 13MB/s one) is what I'm replicating right now and my sustained transfer speeds have been between 24MB/s and 26MB/s. I'm really curious as to what 5.0-rc5 will do for me in the second test scenario, but that will have to wait until tomorrow, after this transfer completes.

 

Anyway, as always, thanks to everyone. One thing I've learned with Unraid is that anytime I've had these bumps in the road, I somehow (with the help of the forum, of course) fumble my way around until it all works out and then I don't have to even worry about my server for months and sometimes years. That's the place I'm looking forward to getting to right now!

 

[bcbgboy13]

I believe RC5 uses older kernel 3.0.xx and RC6 is using the latest one being 3.4.xx

 

 

[tyrindor]

RC5 is going to have more serious issues. You need RC6 to avoid SASLP driver issues, your drives will probably randomly drop out of the array on RC5. May take a few days for it to happen. unRAID is not in a good place right now, I sent email out to them over a month ago and got no response yet. RC7 is MIA, probably waiting for linux kernel updates.

 

My SAS2LP cards (6 of them, 2 servers) are all much slower in RC6, however it's simply not possible to downgrade. You have to choose between reliability or speed right now, until the linux kernel fixes these SASLP/SAS2LP issues.

[BobPhoenix]

RC5 is going to have more serious issues. You need RC6 to avoid SASLP driver issues, your drives will probably randomly drop out of the array on RC5. May take a few days for it to happen. unRAID is not in a good place right now, I sent email out to them over a month ago and got no response yet. RC7 is MIA, probably waiting for linux kernel updates.

 

My SAS2LP cards (6 of them, 2 servers) are all much slower in RC6, however it's simply not possible to downgrade. You have to choose between reliability or speed right now, until the linux kernel fixes these SASLP/SAS2LP issues.

Does the B14 unRAID version have the same problems with SASLP cards?

[pras1011]

I have the same speeds problems with test2 and the SASLP and then going back to RC5 resolves the problem.

[Marky]

I'm using 2 SASLP cards and have had speed issues with all the RC's i've tried. Parity checks at just 17MB/sec is pathetic.

 

Gone back to B14 and turned off any drive spindowns (which i hate) but it does make it reliable.

 

Parity checks on B14 are ~50MB/sec.

 

So system is capable of decent performance.

 

Mark