4+4... 2x4... and other 8TB algebra

pkn · March 23, 2015

Updated the Addonics AD2HDDHP6G tests post, adding all the findings, and updated the start post adding links to testing posts.

pkn · March 23, 2015

First RAID card I tried was 3Ware AMCC 9550SX-12SI SATA PCI-X RAID 12 Port.

4x2TB RAID-0 - created volume is seen by BIOS and unRAID under name 1AMCC_F30JMEVD00005F00260B

No User Capacity reported, Partition size: 7,812,456,396 KB (K=1024)

In pair with 8TB Archive data drive, unRAID says "Disk in parity slot is not biggest."

The card demonstrated sync and free-falling sync rate of 11-12 MB/s writing to 4x2TB RAID-0 pool. Placing card in PCI-X slot with ZCR and playing with card options did not help the speed, so card goes back into the box. Bad card or I don't know how to set it up? Anyway, unusable because of being incredibly slow.

pkn · March 23, 2015

Second card Addonics AD4SA6GPX2 SATA-III PCIe 2X 4-port, Marvell 88SE9230 chipset.

It showed free-falling speed of 340+ MB/s writing to 4x2TB RAID-0 pool parity, so I had big expectations, but... The card creates both 4x2TB and 4TB+4TB RAID-0 pools no problems, however unRAID says: Disk in parity slot is not biggest. That is when I assign 8TB RAID-0 pool for parity, and single 8TB Archive drive for data.

Looking for making parity bigger-than-8TB, I run for "single large drive" option in Addonics card, wanting to make 4+4+2=10TB parity, but this option is grayed out no matter what I do. Quick research revealed that marketing description on their website says one thing, but the user manual says "This adapter supports RAID 0 (2, 3 or 4 drives), RAID 1 (2 drives), RAID 10 (4 drives), and “HyperDuo” (hybrid HDD/SDD mode)." So, the "single large drive" option was a lie. Black mark to Addonics for false advertisement.*

For speed test of pure 8TB Archives I assigned one 8TB Archive for parity and another one for data, both connected to the motherboard SATA ports, and launched sync build. The speeds I've seen, as reported by unRAID main page, are as follows, in MB/s:

200 at 0 GB

200 at 50 GB

200 at 150 GB

200 at 350 GB

195 at 600 GB

195 at 950 GB

130 at 6 TB

110 at 7 TB

100 at 7.5 TB

90 at 8 TB

_______________________________

* - I've got into communication with Addonics tech support, the final answer was:

"Hello Konstantin

That controller BIOS only supports RAID 0,1,10. However using your OS should allow you create a Spanned RAID mode."

pkn · March 23, 2015

Next: "on-bracket" SYBA SY-PCI40037 1:5 SATA II Dip Switch RAID, JMicron JMB393 chipset, $45 at eBay.

This is 5:1 port (1 host port, 5 secondary drives ports) card with chipset and architecture similar to Addonics "on-drive", but being full size card it mounts on regular bracket instead. Card has power-on and activity LEDs for each of 5 drives. It does not use motherboard slot. No drivers, no BIOS setup, no OS, only externally accessible dip switches on bracket for RAID mode control. Card can do RAID-0 / 1 / 3 / 5 / 10 / LARGE / CLONE.

Testing was done aiming to 4+4TB=8TB parity.

Created volume is seen by BIOS and unRAID under name JMicron_H_W_RAID0_blahblah for RAID0, and JMicron_H_W_LARGE_blahblah for LARGE.

Test 1:

4+4TB RAID-0 via SYBA SY-PCI40037 on-bracket, 8TB Archive as data via mobo ===> unRAID says: Disk in parity slot is not biggest. unRAID reported parity User Capacity: 8,001,456,963,584 bytes [8.00 TB]. Free-falling sync speed ~205-200 MB/s. Activity LEDs confirm it's RAID-0, not LARGE.

Test 2:

4+4TB LARGE via SYBA SY-PCI40037 on-bracket, 8TB Archive as data via mobo ===> unRAID says: Disk in parity slot is not biggest. unRAID reported parity User Capacity: 8,001,456,963,584 bytes [8.00 TB]. Free-falling sync speed ~185-180 MB/s. Activity LEDs confirm it's LARGE, not RAID-0.

So, same chipset and same results as Addonics "on-drive" - card silently does size truncation in both RAID-0 and LARGE and there is no way to disable it.

Test 3:

4x2B RAID-0 via SYBA SY-PCI40037 on-bracket, User Capacity: 8,001,389,854,720 bytes [8.00 TB] - smaller than 4+4TB. It seems, that card does size truncation on each disk, not volume.

Anyway, the card offers interesting option for situations when your motherboard has no useable expansion slots, although you will need port-multiplier compatible host SATA port for it to work as port multiplier.

For our 8TB parity algebra... well, no 4+4TB=8, no 4x2TB=8, but can be used for 3x3TB... just like any other card.

pkn · March 24, 2015

Okay... updated testing posts, and starting post. Added conclusions to start post - conclusions are preliminary, of course, but general situation is more or less clear to me now.

To recap, I (hopefully) don't need this hardware for another month or two, so I'll continue playing with.

If any of you guys want me to do some particular check/test - I'm all ears

garycase · March 24, 2015

By the way, I got an answer to my e-mail to Addonics r.e. the truncation problem with their drive-mounted AD2HDDHP6G card. Totally useless response => Just said "... you probably didn't set the jumpers correctly. I've attached a file that details the settings for you." ... and had the PDF attached that shows the jumper settings (same one that's posted on their web site which you've already been using).

I "politely" let them know that the issue had nothing to do with jumpers, but doubt any follow-up response will be any more useful.

pkn · March 24, 2015

Yeah... canned response. I hate canned responses. To add, their response to my request was not canned, but was no better either - they simply admitted that it does not work...

pkn · March 30, 2015

Next: 4+4TB RAID-0 parity pool via Areca ARC-1210 4 Port PCI-e x8 SATA II RAID fw 1.49 in PCIe x8 slot. Firmware 1.49 guarantees that controller understands HDDs > 2.2 TB. Same as with 4x2TB test, I had to disable size truncation in Areca card setup.

8TB Archive as data drive in motherboard SATA port.

unRAID accepted.

unRAID reported parity-via-4+4TB-Areca: User Capacity: 8,001,573,355,520 bytes [8.00 TB]

Data drive 8TB Archive: User Capacity: 8,001,563,222,016 bytes [8.00 TB]

Created volume is seen by unRAID under name 20004d927fffff800

"Free falling" speed of writing (parity sync with single 160GB data drive, after the 160GB mark) by Areca ARC-1210 into 4+4TB=8TB parity RAID-0 pool is about 330 MB/s.

Spindown via unRAID seems to be not working Needs to be set via Areca card setup.

hypyke · March 30, 2015

So I am trying to understand the speeds here. Considering pkn's last post it SOUNDS like speeds are great with the archive drives as long as you use a non truncating raid card and two 4tbs for parity.

What is the "free-falling" speed mean. 330MB/s sounds insane to me but I am only hardware and my parity and rebuilds top out around 80MB/s

c3 · March 30, 2015

What is the "free-falling" speed mean. 330MB/s sounds insane to me but I am only hardware and my parity and rebuilds top out around 80MB/s

That is the speed at which the parity drive is writing without any other drives to read and calc parity.

hypyke · March 30, 2015

What is the "free-falling" speed mean. 330MB/s sounds insane to me but I am only hardware and my parity and rebuilds top out around 80MB/s

That is the speed at which the parity drive is read without any other drives to read and calc parity.

Ok, so are there some sample parity check and rebuild times using the 8tb drives With a 4+4R0 Parity? That would be my main concern.

pkn · March 30, 2015

What is the "free-falling" speed mean. 330MB/s sounds insane to me but I am only hardware and my parity and rebuilds top out around 80MB/s

"Free falling" speed of writing (I still think it's writing, not reading) is 4+4TB=8TB parity sync with single 160GB data drive, after the 160GB mark. It has nothing to do with real array operation, it's just a simple test how fast the RAID-0 pool operates without any other load.

pkn · March 30, 2015

... Ok, so are there some sample parity check and rebuild times using the 8tb drives With a 4+4R0 Parity? That would be my main concern.

Here. Those are 8TB Archive parity, and 4x2TB=8TB parity RAID-0 pool tests, but results for 4+4TB should be similar.

pkn · March 30, 2015

Yeah, and while the last (for today) test is cooking, to close the loop on updating Areca cards firmware. Well it was a while since I last time felt myself such an idiot. On the other hand - it's always good to learn something... even that your are an idiot...

The point was that I disabled password in McBIOS (card setup), and then was trying to feed to McRAID (web- based manager) non-empty default password. It's either this, or that. When I finally came to using same password in both places, updating firmware via McRAID was a breeze. I luuuv Areca! :-*

c3 · March 30, 2015

What is the "free-falling" speed mean. 330MB/s sounds insane to me but I am only hardware and my parity and rebuilds top out around 80MB/s

"Free falling" speed of writing (I still think it's writing, not reading) is 4+4TB=8TB parity sync with single 160GB data drive, after the 160GB mark. It has nothing to do with real array operation, it's just a simple test how fast the RAID-0 pool operates without any other load.

Yup, you're right, sync is writing, check is reading, unless error found and correct requested.

pkn · March 30, 2015

Next: 4+4TB RAID-0 parity pool via Areca ARC-1110 4 Port PCI-X SATA II RAID fw 1.49 in PCI-X 100/133 MHz slot. Firmware updated to 1.49, which makes controller to understand HDDs > 2.2 TB. Capacity truncation disabled.

8TB Archive as data drive in motherboard SATA port.

unRAID accepted.

unRAID reported parity-via-4+4TB-Areca: User Capacity: 8,001,573,355,520 bytes [8.00 TB]

Data drive 8TB Archive: User Capacity: 8,001,563,222,016 bytes [8.00 TB]

Created volume is seen by unRAID under name 20004d927fffff800

"Free falling" speed of writing (parity sync with single 160GB data drive, after the 160GB mark) by Areca ARC-1110 into 4+4TB=8TB parity RAID-0 pool is about 205 MB/s.

heffe2001 · April 1, 2015

Created volume is seen by unRAID under name 20004d927fffff800

So both cards ended up using the same volume name? I'm going to be setting up 2 4tb WD reds as parity with a 1200 card hopefully this weekend, and adding a 8tb archive to my system (well, most likely replacing several smaller drives, trying to get as few platters in my box as possible to cover the size I need).

SSD · April 1, 2015

Next: 4+4TB RAID-0 parity pool via Areca ARC-1110 4 Port PCI-X SATA II RAID fw 1.49 in PCI-X 100/133 MHz slot. Firmware updated to 1.49, which makes controller to understand HDDs > 2.2 TB. Capacity truncation disabled.

8TB Archive as data drive in motherboard SATA port.

unRAID accepted.

unRAID reported parity-via-4+4TB-Areca: User Capacity: 8,001,573,355,520 bytes [8.00 TB]

Data drive 8TB Archive: User Capacity: 8,001,563,222,016 bytes [8.00 TB]

Created volume is seen by unRAID under name 20004d927fffff800

"Free falling" speed of writing (parity sync with single 160GB data drive, after the 160GB mark) by Areca ARC-1110 into 4+4TB=8TB parity RAID-0 pool is about 205 MB/s.

See THIS for instructions on how to get the Areca controllers to use standard names.

pkn · April 1, 2015

Created volume is seen by unRAID under name 20004d927fffff800

So both cards ended up using the same volume name?

Yes, when they were installed in turns. I'm pretty sure the names will differ somehow if both cards will be installed simultaneously. I will test this when I have a chance.

pkn · April 1, 2015

See THIS for instructions on how to get the Areca controllers to use standard names.

Thanks, I keep an eye on it, great thread and good work. I don't really need standard names as I'm planning to use Areca cards for RAID-0 pools only, but it's nice to have, and especially the smart reports.

SSD · April 1, 2015

See THIS for instructions on how to get the Areca controllers to use standard names.

Thanks, I keep an eye on it, great thread and good work. I don't really need standard names as I'm planning to use Areca cards for RAID-0 pools only, but it's nice to have, and especially the smart reports.

The standard naming may resolve your duplicate drive name issue. I expect that RAID volume names would be unique after the switch.

pkn · April 2, 2015

Next: Data disk rebuild test number... err... number Ultimate.

Data disk rebuild tests number one and number two were done in "simulation" (no real data, and no real parity) mode. So, to remove (from myself ) any doubts I've done it again with everything real.

Setup: 4+4TB=8TB parity (RAID-0 parity pool via Areca ARC-1110 4 Port PCI-X SATA II RAID fw 1.49 in PCI-X 100/133 MHz slot) and two 8TB Archive data disks.

1. I filled data disk1 with files. 99% filled.

2. I run parity sync. Parity sync took same 15.06 hours, average speed ~140 MB/s. I was not watching, so the average is from syslog*.

3. I replaced data disk1 with physically different drive.

4. Started data disk rebuild - rebuild completed successfully in same 15+ hours with same average speed ~140 MB/s.

Doubts removed.

___________________

* -- Syslog lines like this:

Mar 30 10:47:04 Testt kernel: md: recovery thread woken up ...

Mar 30 10:47:04 Testt kernel: md: recovery thread syncing parity disk ...

Mar 30 10:47:04 Testt kernel: md: using 1536k window, over a total of 7814036428 blocks.

Mar 31 01:50:34 Testt kernel: md: sync done. time=54210sec

Mar 31 01:50:34 Testt kernel: md: recovery thread sync completion status: 0

pkn · April 10, 2015

Just to report the findings... discovered abnormally slow parity check (check, not sync) phenomena. The reason, it seems, is an ancient kernel bug which only affects AMD-based systems and is not noticeable on Intel-based.

heffe2001 · April 10, 2015

Running a preclear on my 2 4tb Red's in a Areca raid0 array, using BJP's modded preclear and getting right around 305MB/s... Soon as that process has finished, I'll assign it as my Parity drive and see how long it takes to generate parity for my currently (unprotected ) array of 9 disks (+parity drive).

With the Areca naming fixes applied, the array is identified by more than just a number (same for drives assigned to that controller, the ones I put on it with data actually are able to see and read the data as well, was semi-worried that those drives would need to be reformatted coming from a SM MV8 card, just assign them as pass-through in the Areca setup, assign them and they worked without issue.

Name assigned by unraid to the Areca volume: ARC-1231-VOL#00_0000002794297501

Now if Limetech could fix the temp display of drives attached to the Areca controller in Dynamix, I'd be a happy camper..

heffe2001 · April 13, 2015

Can't seem to get Unraid to use the raid0 array as parity. It will let me select it, and doesn't give a size-error, but when I click the checkbox to accept the configuration, and hit start, it never gets to the part where it mounts the file systems.

Looks like to me the relevent bits of the attached syslog are:

Apr 13 15:32:10 media01 emhttp: shcmd (195): sgdisk -Z /dev/sdb &> /dev/null
Apr 13 15:32:11 media01 kernel: sd 6:0:0:0: [sdb] 1953509120 4096-byte logical blocks: (8.00 TB/7.27 TiB)
Apr 13 15:32:11 media01 kernel: sdb: sdb1
Apr 13 15:32:11 media01 kernel: sdb: p1 size 34359738360 extends beyond EOD, truncated
Apr 13 15:32:11 media01 emhttp: shcmd (196): sgdisk -o -a 64 -n 1:64:0 /dev/sdb |& logger
Apr 13 15:32:12 media01 logger: Creating new GPT entries.
Apr 13 15:32:12 media01 logger: The operation has completed successfully.
Apr 13 15:32:12 media01 emhttp: shcmd (197): udevadm settle
Apr 13 15:32:12 media01 kernel: sd 6:0:0:0: [sdb] 1953509120 4096-byte logical blocks: (8.00 TB/7.27 TiB)
Apr 13 15:32:12 media01 kernel: sdb: sdb1
Apr 13 15:32:12 media01 emhttp: invalid partition(s)
Apr 13 15:32:12 media01 emhttp: shcmd (198): :>/etc/samba/smb-shares.conf
Apr 13 15:32:12 media01 avahi-daemon[20909]: Files changed, reloading.
Apr 13 15:32:12 media01 emhttp: Restart SMB...

Wonder if it's because I used 4k block size instead of 64b LBA? Going to try changing it and see how it goes..

**EDIT** That was it, changed to 64b LBA and it allowed me to add as parity drive. Generating parity at just shy of 100MB/s..

syslog.txt

4+4... 2x4... and other 8TB algebra

Recommended Posts

Link to comment

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation