Asus M4A785TD-V EVO/U3S6 - passed the Test Level I

[aht961]

I have bought this 105€ AM3 ATX board yesterday (785G/SB710 chipset). unRAID booted without problems, found all my drives. Ran the memory test without glitches and now building up the parity. I will be testing it for a while, but the start was very promising. 2 x PCIe (x16 and x4 mode) + 1 x PCIe x1 + 3 x PCI 2.2 buses make it rather attractive. The board has onboard 5 x 3GB/s SATA + 2 6GB/s SATA + 1 eSATA ports. The 6GB/s ports are on the additional card which is PCIe x4 and comes along the board with the same price (this very same card called U3S6 has 2 USB3 ports as well). The unRAID recognised the SATA ports on the card as well without any problem. Highly recommended since all components seem to be 100% unRAID compatible (so far!).

 

http://www.asus.com/product.aspx?P_ID=Ic2Z48E0qdz0KwIU

 

I will let you know with performance details as soon as I get so far

[terrastrife]

the right thing now, would be to do level 1 testing

 

do it with a hdd or two on the 6g sata too, not just onboard

[aht961]

Yes, I will do it. I already have the cache drive on the 6GB SATA port, I will add another HDD to the other 6GB port which will be in the array. The array will have 6 HDDs then (5 on mobo + 1 on 6GB port). Obviously with standard HDD the 6GB/s doesn not help - with SSD it would be awesome. With this I intend to have a 8 HDD system: 6-HDD array  (max for PRO license) + 1 cache drive (+ 1 additional IDE HDD off the array). The purpose is to have a dedicated bandwidth with a little bit overhead for each SATA II HDD.

 

I went for 240e 45W CPU (since I read in a test that it is more energy efficient than single core sempron 140, which was my first choice). If the system goes through the level 1 testing, this will be a very cost-efficient system of high "performance" for an entry PRO level of unRAID.

 

My rig for those who are interested:

 

Mobo: Asus M4A785TD-V EVO/U3S6

http://www.asus.com/product.aspx?P_ID=Ic2Z48E0qdz0KwIU

- LAN:  RTL8112L

- Chipset: 785G/SB710

- SATA Controller on U3S6 card (included in mobo box): The card has three primary components:  an NEC D720200F1 USB 3.0 controller, a Marvell 88SE9123 SATA 6.0 controller, and a PLX PEX8613 http://www.maximumpc.com/article/news/exclusive_first_usb_30_and_sata_60_expansion_card_will_sell_30. This will be more than enough for 2 SATA HDD drives. If the SATA ports are saturated, however, USB should not be used at the same time (since this is a x4 card: 2 SATA + 2 USB3 => 6 + 6 + 5 + 5 = 22Gb/s. X4 will allow only 1000MB/s.)

 

CPU: Athlon II X2 240e/AM3 2.80G 2MB-L2 45W

 

Memory: CORSAIR 2GB DDR3 1066 (2x1GB), 7-7-7-20 (TW3X2048-1066C7 G)

 

PSU: Corsair cx400w -400W ATX http://www.corsair.com/products/cx/default.aspx

 

Case: Fractal Design, Define R2, Black Pearl (8 removable HDD trays)

http://www.fractal-design.com/?view=product&category=2∏=32

 

System cost: 358€ (with 240e) or 313€ (with sempron 140) + unRAID Pro key

 

I am expecting a write speed of 60-70MB/s with cache drive or 30-35MB/s without a cache drive using NFS in a GIG LAN with MAC Os X client.

 

PS: If you intend to add additional PCIe x1 or x4 or x8 cards, you will be able to attach plenty amount of SATA II HDDs without compromising the bandwidth needed for conventional HDDs.

x16 = 4000MB/s = 8 - 16 HDDs

x4 = 1000MB/s = 8 HDDs

x1 = 250MB/s = 2 HDDs

PCI x 3 = 133MB/s for each, ie. 1 HDD

on board SATA x 5 = 300MB/s each => 5 HDDs

That makes at least 24 HDDs! Please correct me if I have interpreted the bus speeds wrong.

 

[agw]

I don't mean to hi-jack your thread aht, but I clicked on the Fractal Design link in your message because I was not familiar with that brand I am very impressed with your case choice.  From the pics, looks like very high quality.  Looking forward to your comments after your build.

 

I also noticed this guy: http://www.fractal-design.com/?view=product&category=2∏=39

Wow - looks like the perfect case for those eyeballing the Supermicro X7SPA mini-itx board.  Very interesting.

 

Of course, doesn't appear to be readily available here in the States . . .

[aht961]

Hi agw. That is produced by a new Swedish company which was founded very recently. It has a minimalistic design and obviously designed by guys who have themselves built computers (not by the engineers). The case is very recently introduced to Finnish market and mine might be one of the first ones sold last week. The bigger brother does not have PSU neither a removable tray. Nothing much included. However, everything which is (screws, fans) included is of good quality. This case has had surprisingly quite a few good reviews, and I hope we will hear about these guys more in future.

 

The smaller cute "cube" which you have liked is not yet available here and costs around 150€ and has a small PSU (300W/80PLUS).

 

PS: I am having very nice write/read figures with this setup. At the moment I am doing a parity check and it is around 110MB/s. I will check from syslog when it will finish. Earlier with Intel mobo I had around 55MB/s with 4 drives and it took about 9 hours. Now seems to be much faster with 6 drives! The timing buffered disk reads vary between 105 MB/s (WD greens 2TB) - 122 MB/s (seagate 1,5TB). Earlier I had a write speed to cache drive with 50-60% of the buffered read speed, therefore, I might have 60-70MB/s with the current setup. Let's see.

[Rajahal]

Are there 6.0 GB/s SSDs available yet?  If so, I'm curious as to what speeds you could achieve using one as a cache drive (on one of your SATAIII ports, of course).  My tests with a 3.0 GB/s SSD cache drive on a SATAII port resulted in a max write speed of 73.2 mb/s.  I'm fairly sure you can best that.

[aht961]

Rajahal please have a look at this: http://www.techspot.com/news/37192-micron-intros-industrys-fastest-ssd-uses-sata-6gb-s.html

It says "sequential read speeds of up to 355MB/s and sequential write speeds of up to 215MB/s"  Remember though the 1000mbps limit for gig LAN (ie. 125MB/s).

[Rajahal]

Nice.  The SSDs I tested were all MLC, so I'm sure even a 3.0 GB/s SLC SSD would outperform them.

[terrastrife]

except you need capacity, which means the only real choise is the ocz z drive which is 2tb and 1400MB/sec

[aht961]

Slightly going off-topic, however, read this interesting article concerning the solid state RAID speeds: http://www.nextlevelhardware.com/storage/battleship/. Taking into account that we are dealing with unRAID and its known down sides concerning the "speed", I do not see any good reason or rationale why one should consider SSDs here as long as their "value per gigabyte" stay unreasonably high in particular for home usage. The best one could get under these circumstances seem to be already achievable with a 10000rpm HDD if 7200rpm is considered not good enough. My particular interest in 6 GB SATA ports was due to the fact that I knew that the bus speed of these ports would be at least as good as the ports on the motherboard making the number of available fast SATA ports 7 + potential of the additional PCIe  x16 and x1 buses and all this luxury with 100€    

[aht961]

This mobo has passed memory test (12 hours) with no errors. The parity check (6 Drives totalling 5.75TB of which 15% used. Parity drive: WD 2TB Green) finished in 24316secs with 80.3MB/s. No errors occured. The syslog is attached.

 

syslog_080410.txt

[aht961]

I have done some benchmarking to see how this build performs under real life conditions. I did not use any standard methods described elsewhere since they do not seem to simulate the real life conditions in user's own environment. All I wanted to know was "how long it takes in daily life for me to copy a file to unRAID and from unRAID to my client".

 

The testing environment:

1000Mbps LAN with CAT5e cabling in a two-floor house, where the longest cable distance from the switch (D-Link DGS-1008D 8 ports with jumbo frames) varies between 10-30m.

 

The client: iMAC 24" 3GHz, 64bit, desktop running Os X 10.6.3 snow leopard, memory 4Gt 1067 DDR3.

Ethernet card 1000Mbps (surprisingly turned out to be not PCIe-based).

 

The unRAID setting is given earlier in this thread. In addition the HDD data:

1. 00S_WD-WCAVY1215619 WD Green C. 2TB (Parity drive)

2. 00S_WD-WCAVY1232928 WD Green C. 2TB

3. SAMSUNG_HD753LJ_S13UJ90QB02761 SPINPOINT F1 750GB

4. ST31500341AS_9VS3HW5X SEAGATE Barracuda 7200.11 1.5TB

5. SAMSUNG_HD753LJ_S13UJ90QB02760 SPINPOINT F1 750GB

6. SAMSUNG_HD753LJ_S13UJ90QB02756 SPINPOINT F1 750GB

+1 CACHE DRIVE WDC_WD20EADS-00_WD-WCAVY1215260 WD Green C. 2TB

 

The client is connected to the unRAID server via the switch the distance in between being 10m. I have used 4 different types of files: 1) 512MB single disk image 2) 1GB single disk image 3) 4GB single disk image 4) A folder with various subfolders containing 1151 files with 337MG size. These include images, txt files, mps and all possible available stuff. The timings are taken using an electronic stop watch.

 

I have tested various scenarios and checked the write / read speeds to 1) directly CACHE drive mounted using NFS 2) directly to CACHE drive using Samba 3) Samba user share 4) NFS user share with cache disk and 5) User shared mounted using NFS.

 

I am very surprised to achieve an average write speed (WITHOUT cache drive), using the samba user shares, of 46.5MB/sec using a 512MB single file and 40MB/s with 1GB or 4GB files. With multiple files the figure drops down to 26.4MB/s. These are the averaged achieved goodput figures (ie. file size in MB divided by secs required for transfer) not the maximum achieved figures (which are higher). I could write directly to cache drive with a speed of 102.4MB/s.

 

Conclusions: Before I had a 4-drive setting based on D945GCLF2 Atom 330 1,6GHz (Intel i945GC/ICH7 chipset). I ran out of SATA ports (2 on board + 4 on 2.2 PCI). By swapping to the current setting, I have achieved a 53% increase (from around 30MB/s to 46MB/s) in writing speed without using cache drive to a user share as well as a 67% increase (from around 60MB/s to 100MB/s) in writing speed directly to a cache drive. This is probably due to:

- the new chipset

- all HDDs are now on dedicated lanes

- all IDE PATA disks removed from the array

- a slower SATA I drive replaced with a newer/faster SATA drive

Even though we know that this process does not need much CPU power, I am tempted to underclock the present CPU from 2,8GHz to <2GHz to see if this has any impact on the write/read speeds (just to make sure and besides to lower the power consumption). All in all, I believe with this very reasonably priced setting excluding the license and the cost of the HDDs (if the case excluded or changed for a cheap one and the CPU downgraded to sempron 140, the expenses would be around 250€), I hit the upper practically achievable write/read limits, which are indeed almost outstanding.

 

In light of my test results, I ended up with the following strategy:

 

- I will use cache drive (NFS) or any other off-array HDD only if I need some speedy temporary transfer

- Otherwise, I will use user shares without cache (and either NFS or samba, since the difference between them is very marginal, NFS share being better with multiple small files - this is suprising since Mac Os X is known not to like smb shares, probably snow leopard has changed the old story)  

- My additional or cache drive will be a potential HDD waiting to rescue a disk failure situation

 

Here I attach the figures where you will see different performance levels in various scenarios. Next step would be to test the same parameters and files using a Win 7 client - I will let you know as soon as I get time to accomplish that.

 

I highly recommend this beast to any one who will be pleased with a 6+1 (or +4) HDD setting!

____________

 

Please note that in the lower graph (read transfer speeds), the NFS user share+cache combination (violet line), lags behind samba user share and cache (smb). It should be as good as the normal NFS user share (light blue line). I was wondering this - and now I realised that during the copying process the test file was indeed still on the "cache" drive, since the mover moves that after midnight.

[aht961]

Theoretical background for the achieved average write speed to a parity protected drive (I use the excellent argument presented earlier by bubbaQ here:http://lime-technology.com/forum/index.php?topic=5831.msg55117#msg55117 to support the figures below):

 

The disks used have a read performance of around 100MB/s and write performance of 85-86MB/s.

Let's try to write a 512MB file: 512MB/100MB + 512MB/86MB secs => 5,12s + 5,95s = 11,07s required to achieve parity protected write process giving a speed of 46,25MB/s which is almost the same as the figure I got here indicating that I have hit the upper possible limit with the current HDDs used! The same holds true for 1GB and 4GB single files.

 

E.g.:

1GB: 21,6s required for write process = 46,2MB/s (in real life test I got 40MB/s)

4GB: 87s required for write process = 45,9MB/s (in real life test I got 40MB/s)

 

What happened if I used the fastest available HDD limited to the upper transfer limit of 1000mbps LAN (about 120MB/s)?

 

So let's assume "better" HDD's read and write speeds both are equal 100MB/s (e.g. WD Velociraptor).

 

For 512MB, 1GB and 4GB single files this should in theory lead to = max. 50MB/s, in real life with more occupied disks less than that?

 

I put this argument to support the assumption that even with a very modest low-end sytem like this one, I am easily taking the unRAID to its upper limit. Therefore, additional investment should effect the expandability of the unRAID systems not the performance.

 

I am just assuming - am I mistaken?

 

 

 

 

[Joe L.]

I put this argument to support the assumption that even with a very modest low-end sytem like this one, I am easily taking the unRAID to its upper limit. Therefore, additional investment should effect the expandability of the unRAID systems not the performance.

 

I am just assuming - am I mistaken?

I think you are looking at it realistically.  Yes, you can wring a bit more performance by spending a lot more money. (Velociraptor 10,000 RPM drives as an example) Or, you can get most of the performance with a modest system.

 

In the commercial world, writing nightly batch processes, we would work to improve performance to where all the tasks needed were performed before the systems were needed the following business day.  It did not matter too much if nightly processing took 3.5 hours, or 3 Hours.  It did matter if they took 27 hours and needed to run once a day, but that usually only happened when a database index got removed accidentally. 

 

For me at home, it does not matter if it takes 10 minutes to rip a DVD, or 15.  I click on the button and go about other tasks.  I'm still using a PCI bus based server, with parity calc speeds 1/5th of those routinely posted these days.  My primary use is to play movies.  The speeds are plenty enough to support multiple ISO images playing at the same time from a simulated disk image being reconstructed onto a replacement disk. (I think I got up to 4 or 5 movies playing at one time... ) The re-construction speed dripped down 4 or 5 MB/s for each additional Movie I played, but it was continuing while I tested.  At that point (4+ years ago) I knew performance was fast enough for me.

 

There are always people who need the absolute highest speed possible, and those who want the absolute highest speed possible, even if they don't really need it...  they help everybody learn the limits of the hardware.

 

Joe L.

[Rajahal]

Wow, great work aht961!  Thanks for the comprehensive benchmarks!  I appreciate your 'realistic' approach to benchmarking.

 

I am astounded by your write speeds, especially to the cache drive.  Simply astounded.

 

Also, from our discussion above, I would only consider using an SSD as a cache drive at the current time.  It does not make sense to use one as a parity or data drive, since they are just far too small.

[ysss]

Wow, nice job.

 

I wonder if this board would work with 2x AOC-SASLP-MV8 ?

Does anyone know how to tell if a motherboard could work with 2x AOC-SASLP-MV8?

[aht961]

I have calculated now the data transfer rates using a WIN 7 client and the same files (512MB, 1GB, 4GB and multiple files). Please find attached the revised figures where you will be able to compare the unRAID performance between Mac Os X and Win 7 clients. To my surprise, the worst performer is the Win 7 client using samba user shares during WRITING to the unRAID, and the Mac OS X while READING from unRAID. Mac Os X is faster despite the samba file system. When copying/writing multiple files/folders Win 7 is the worst performer. The best performance is achieved using Cache drive with NFS and Mac Os X client.

 

PS: The top figure shows the WRITE and the bottom one READ speed (MB/s).

 

[aht961]

With regard to the power consumption (measured), this rig draws 108W with all 6 HDDs spinning and 70W after spin-down. I also tried to check the results with undervolting/underclocking the 45W CPU (from 2,8GHz to 2GHz and from 1,29V stock to 1,025V), the result is a decrease of ONLY 6W both idle and HDDs spinned up (ie. 101W and 64 idle). However, the system runs much cooler, before HDDs peaked up to 39-40C during read/write process, now I get up to 34-35C and 19-26C during idle. No change in performance.

 

When the system is on 24/7, the power consumption will be around 35€ per year. 

[ysss]

Wow, nice job.

 

I wonder if this board would work with 2x AOC-SASLP-MV8 ?

Does anyone know how to tell if a motherboard could work with 2x AOC-SASLP-MV8?

 

Yep, it works just fine with a pair of 2x AOC-SASLP-MV8 on it. I'm getting parity sync speed of 92MB/sec with four test drives (WD15EADS, 1.5TB), connected 2 on each of the SAS controllers. I tried putting another WD15EADS to the onboard SATA, but parity check somehow dropped to 10MB/sec with that connected. I will have to tinker with the BIOS later.

 

Once I this thing tests out okay, I will move my primary unraid server (13 drives or so, 16TB effective space) to it and expand the space with these WD15EADS..)

 

sw: Unraid 4.5.3 pro

[Blofeld]

@ aht961,

thanks for your input..I'm a total newbie to this, and I pretty much copied your setup, as it seems to be in quite a sweet spot of price/performance/efficiency.

I'm on iMac 24' / OS X 10.6 as well, and I'm happy to see your transfer speeds.

 

I've just placed my order, see below for those interested (I live in Germany).

 

Case ATX Midi Fractal Design Define R2-SI silver (it's pretty, see here: http://www.fractal-design.com/?view=product∏=34)

Mobo ASUS M4A785TD-M EVO 785G AM3

CPU AMD Athlon II X2 245 2.90GHz AM3 2MB 65W BOX

HDD Samsung EcoGreen F3 2TB 2x

PSU Enermax EPR425AWT II 425W Pro 82+

RAM 2x2048MB Kingston Value DDR3-1333 CL9 Kit

SanDisk Cruzer Micro 4GB

Total was roughly €600

 

I've got another drive or two lying around that I'll add later, and I'm also planning on adding a cache drive, because I need fast access (I'm a photographer, and plan on keeping my ever-growing photo/video library there).

Based on my research, this combination should work fine (fingers crossed) - ok maybe I'll post it in a different thread as well to get some advance feedback

 

Thanks for now - I'll keep you posted! 

 

[Joe L.]

I'm also planning on adding a cache drive, because I need fast access (I'm a photographer, and plan on keeping my ever-growing photo/video library there).

The cache drive is ONLY used when writing to the array.  It has no effect on the speed you see when accessing files already on the server.

 

Your library of pictures will be protected, but just be certain to make a backup copy of critical files and store them off-site.  It only takes one fire/flood/tornado/lightning strike and your server and the data on it lost...

 

See here: http://lime-technology.com/forum/index.php?topic=2601.msg21033#msg21033

 

[Blofeld]

...yeah, I know - if you have something once, you don't have it at all...which is why I do have an off-site backup thing going - but thanks for the heads-up!

[Blofeld]

With regard to the power consumption (measured), this rig draws 108W with all 6 HDDs spinning and 70W after spin-down. I also tried to check the results with undervolting/underclocking the 45W CPU (from 2,8GHz to 2GHz and from 1,29V stock to 1,025V), the result is a decrease of ONLY 6W both idle and HDDs spinned up (ie. 101W and 64 idle). However, the system runs much cooler, before HDDs peaked up to 39-40C during read/write process, now I get up to 34-35C and 19-26C during idle. No change in performance.

 

When the system is on 24/7, the power consumption will be around 35€ per year. 

@aht961,

that's an interesting find, regarding the underclocking of the CPU. What are your longer-term experiences with that, are you keeping it that way? Still no performance decrease, and still running cool? Is it hard/complicated to underclock the CPU? My mobo/case/cpu/... should be in the mail any day now...