Okay,

 

I finally got all my hardware setup last night, and got to play with my first unRAID server.  It built my parity drive over night and I only got to play for about 30 minutes this morning before I had to go to work.  At this point, I'm a little unimpressed with the performance and I was wondering if there is anything I can do.

 

• PC Case w/ 4 x 5.25" ext bays, 2 x 3.5" ext bays, 4 x 3.5" int bays
  
• Icy Dock 3 in 2 Hot Swap Bay
  
• Antec 80 PLUS PSU w 4 20A Rails (one rail per cable, which is nice)
  
• GIGABYTE GA-G41M-ES2H Motherboard (on-board video)
  
• Pentium E6300 Wolfdale 2.8GHz Dual Core
  
• 4 GB (2x2) Corsair XMS2 RAM
  
• WD Caviar Black WD7501AALS 750GB SATA 7200 rpm
  
• WD Caviar Green WD10EAVS 1TB SATA 5400 rpm
  
• WD Caviar Blue WD10EALS 1TB SATA 7200 rpm
  
• 4GB SanDisk Cruizer Micro
  

 

Most of these parts were pulled from a previous working system, so I know they are good and monitored/tested and check out.  The only new addition was the WD10EALS, but it checked out okay with HD Tune.

 

This machine still had Windows 7 on it, so before switching to unRAID, I formatted the three drives and ran HDtune on each of them.  SMART came back clean and the two 7200 rpm drives had sequential read/write speeds of 100+ MB/s.  The WD10EAVS drive also had a clean SMART and posted read/write speeds of 85+ MB/s.  All the HDD's were mounted in their dock and connected by SATA cable to the ports on the motherboard.

 

So with the drives checking out okay, I proceeded to remove the extra add-in cards, change the necessary BIOS, and flashed my Cruizer with 4.5.6.  Setup was a breeze.  The WD10EALS was set as the parity drive, the WD10EAVS as disk1 and the WD7501AALS as disk2.

 

Then I built my parity drive.  I was getting speeds of 48-50 MB/s when building the parity drive.  Those numbers weren't far from what I expected.  I thought my overall throughput for reading from one disc (data) and the writing to another (parity) would be about 50% of the slowest drive, or about 42.5 MB/s in this case.  I was doing better than that, so no alarm bells.

 

I awoke this morning to find the parity drive complete.  I setup some user shares to start things off and then started copying some data.  And this is where I became a little disasspointed...

 

From a Windows 7 machine, I was copying from explorer to the user share (not the disk share) and I was constantly getting a speed of 18 MB/s.  I thought it would be much higher (at least double) based on the numerous posts I see of people claiming to get 30-40 MB/s using all 5400 rpm green drives.  The first thing I checked was my ethernet settings and I can confirm that both the PC and unRAID are set for 1000 Mbs connections.  Ping times between the two are 1-2 ms.

 

I'm new to linux, unRAID, etc. so I'm not sure what my next step should be.  I had read that writing to disk and not share was faster, and that using a Samba share was faster.  But would the combination of these two things double my speed?

 

Just looking for some advice on how to proceed...  or is everyone claiming 30+ MB/s writes just blowing smoke...

 

EDIT:  I guess I should also add that I ran HD Tune on the PC drive that contains the data (a WD5000AADS 500GB green drive) and it was a read/write of 88 MB/s.  The main C drive of that PC was doing 125 MB/s.  Both were clean on the SMART check and didn't show any alarming values.

There are tunings you can do.

 

Keep in mind, that the slowest disk can hold back the speed of a write.

I would surmise you can not expect a write to be more then 50% of a single drive's read speed.

So for the 85Mb/s drive, /2 = 42Mb/s that is the best you can ever possibly do, yet the parity drive comes into play thus dropping that to probably around 20-24Mb/s.

 

Now with some expansions and tunings you can get higher with up to 30Mb/s.

 

See this thread regarded to tunables.

http://lime-technology.com/forum/index.php?topic=7309.0

There's a script I used to play around with the tunings.

 

Set the number after *=3 to what you want to test with. This is where I stopped.

There are a number of other parms you can test with.

 

Another option is to play with the kernel tuning for caching.

I changed

sysctl vm.dirty_ratio=80

vs the system default of.

sysctl vm.dirty_ratio=10

 

I adjusted

sysctl vm.vfs_cache_pressure=100

because I was having out of memory issues. This means that your director entries are not cached as long.

 

Also if you went all out and put a Seagate 1.5TB 7200RPM or a Hitachi 2TB 7200 RPM drive in the parity slot you would see a slight improvement.

 

 

So you don't have any false expectations.

With Some tunings and with my parity as a raid0 setup I best at 35-40Mb/s.

After I changed the vm.dirty_ratio I would burst at 40-60, but as transmission took longer and longer it would drop down to lower 30s.

It all depends on how busy the cache is and how full memory and the drive are.

There have been times when I moved a 1GB file @60MB/s via teracopy.

 

All my benchmarks and times are with Teracopy on windows. or a dd directly to the drive.

I was wondering if you could do that, using RAID on your discs in an unRAID.  I had kicked around the idea of a RAID0 parity drive or possibly using RAID1 data drives to improve streaming performance for multiple devices.

 

So far, all my shares have written to disk1.  When I get home, I want to try to writes to the faster 750 GB Caviar Black to see how that goes.

 

I guess 18 MB/s isn't so bad, but 24-30 would be better.   I Googled and read some more and found tons of people (from 2008-2009 though) that were getting 10-12 MB/s write performance from their unRAID, so I guess I'm starting from a better position.

 

I'll have to study your tweaks some.  I like to know what a change does before I actually make one, and I know zip about linux.  I'm still having trouble getting smartctl command lines to do what I want, so it may be a few days before I dip my toes into scripting.  I sort of feel like I'm back in college learning Fortran or PASCAL again and searching the "web" using gopher on Unix.

Weird.... all night long I've been getting 28 MB/s transfers writing to my slowest drive (only 30 MB/s writing to the faster WD Caviar Black, so I don't know if it is worth the extra money buying the Blacks, unless maybe as parity first).  I've seen spikes up to 35-40 MB/s, but 99% of the time it is sitting at 28 MB/s.  I wonder why it was only 18 MB/s this morning.  I'll have to run my net monitor next time it happens to see if  there is local traffic or something slowing down the network.  Bad pun, but the difference is night and day, and I haven't done a thing (not even a reboot).

I assume you transferred different files. The issue is likely to be on the source side. Perhaps the files you were transferring from that went slowly were obscenely fragmented or had read issues, while the more typical transfer speed files were not.

A lot depends on your source (fast local drive, USB drive, firewire drive, optical media, flash memory, etc.) the type of files you're moving, and their condition.

 

If you're moving a lot of small, fragmented files from a USB drive, performance will suffer.  If you're reading from your system drive while it's churning away on some other process, performance will suffer.  If you're moving large, contiguous files from a SATA/eSATA drive that's not being used for any other I/O, you'll see maximum transfer speeds.

 

Using a 7200 RPM 2tb Hitachi for parity and a mix of 5400-7200 RPM 1tb to 2tb drives for data, I get sustained transfers in the 27-32MB/s range when reading from an optimal source.  I get low to mid 20s when reading large, fragmented files from a USB-attached hard drive.  5-15MB/s from optical media depending on the type and the drive used.

 

I never bothered testing a parity drive since most of my data was being copied from sources that couldn't max out a direct write to the array.  I was contemplating getting a fast SSD for a cache drive but I'm glad I didn't.  I've done the majority of my data shuffling and it wouldn't have sped things up at all and future writes won't be nearly as massive as the 7tb I've dumped on there. 

All the files were coming off the same 500 GB HDD internal SATA HDD on my desktop PC, but I have no idea how fragmented it is.  The initial 18 MB/s transfer was 8 DVD .isos @ around 4 GB each.  Everything transmitted last night was again .iso files and they varied in size from 1 GB to 7 GB, transferred over in batches.  I was getting 28+ MB/s with them.  I noticed that when I transferred my MP3's and photo archives that I was seeing speeds 35-40 MB/s, but expected that with smaller files that can burst.

 

Today I converted a Ts folder on my local PC to an iso stored on my unRAID, and it transferred at 38 MB/s according to that program.  Which there was a 100 MB buffer involved on my local PC side, so that might have made the transfer rate look a little better.

 

So I'm not sure what was going on that first morning, but its all okay now.  I streamed my first Blu-ray .iso last night and had zero playback problems and was utilizing under 25% of my network, so multiple streams of two or three Blu-ray .iso's look possible, depending on the read speeds I get from unRAID.  I assume reads should be close to the regular drive performance.  Streaming some of the DVD cartoon .iso's barely even registered on the graph.  I think it was 2% of the network.  I was streaming some AVC .mkv converts and you couldn't even see the activity on the chart.

 

I currently use a HDD in my son's room, because there is no wired ethernet connection there and I didn't think wifi would be sufficient.  But as long as he isn't streaming HD content, looks like a good n connection would more than handle DVD streams.  I may have to get another wireless bridge for his room.

 

My next step is to upgrade to the pro version I guess.  Looks like when using disk cache, peoples write speeds tend to increase 10-20%.

 

All is looking good and I'm pretty happy so far, but I still have a ton to learn.  I haven't been on a Unix box since 1996, so I'm finding I need to relearn some things before I get too heavy into it.  I did get my iTunes setup on my PC and my laptop so they are both sharing the same library on the unRAID  That went rather smoothly.

 

The only annoyance I have is that my network to TOWER keeps disappearing in Windows.  If their is no activity to unRAID after about 15 minutes, TOWER just disappears from my networking.  If I refresh windows explorer it comes back.  Not sure why it comes and goes, and I haven't checked to see if items that link out to TOWER still work even though it isn't showing.

Weird.... all night long I've been getting 28 MB/s transfers writing to my slowest drive (only 30 MB/s writing to the faster WD Caviar Black, so I don't know if it is worth the extra money buying the Blacks, unless maybe as parity first).  I've seen spikes up to 35-40 MB/s, but 99% of the time it is sitting at 28 MB/s.  I wonder why it was only 18 MB/s this morning.  I'll have to run my net monitor next time it happens to see if  there is local traffic or something slowing down the network.  Bad pun, but the difference is night and day, and I haven't done a thing (not even a reboot).

 

At the current speed, you will probably not see much of a performance boost for the Caviar Black unless it is a 2TB 7200RPM drive with a large cache.

 

I was wondering if you could do that, using RAID on your discs in an unRAID.  I had kicked around the idea of a RAID0 parity drive or possibly using RAID1 data drives to improve streaming performance for multiple devices.

 

There is no performance improvement of using RAID1 for data. The only place RAID1 provides anything useful is on the cache drive if you have one. In that location writes will be diminsihed slightly but they will be protected until the data is moved to the array.

RAID0 on parity works well at a cost.

Consider the cost of 1 Hardware raid controller, 2 Seagate 1.5TB drives and see if that is worth an extra 5-10MB/s.

 

I found the performance tweaks and kernel adjustment to provide a larger boost.

I went with my hybriid safe20% raid0/raid1 environment for a couple reasons.

2TB drives were costly, I scored the RAID controller for $99.

It gave me RAID0 on parity and RAID1 on cache with only 2 drives.

 

With that I was able to move my /home folders from vmware to an NFS mount on the cache.

Since the cache and parity are now always spinning. Any other write access does not have to wait for these drives to spin up.

 

In your case, I would recommend trying the software and kernel tweaks.

You can adjust the tunables in the settings page near the bottom.

 

Tunable (md_num_stripes):

Tunable (md_write_limit):

Tunable (md_sync_window):

 

When I adjusted

sysctl vm.dirty_ratio=80

sysctl vm.vfs_cache_pressure=100

 

The buffer cache did more work, but I also saw streaming writes hover around 50MB/s.

This weekend I plan to upgrade 4GB to 8GB and see what happens.

Weird.... all night long I've been getting 28 MB/s transfers writing to my slowest drive (only 30 MB/s writing to the faster WD Caviar Black, so I don't know if it is worth the extra money buying the Blacks, unless maybe as parity first).  I've seen spikes up to 35-40 MB/s, but 99% of the time it is sitting at 28 MB/s.  I wonder why it was only 18 MB/s this morning.  I'll have to run my net monitor next time it happens to see if  there is local traffic or something slowing down the network.  Bad pun, but the difference is night and day, and I haven't done a thing (not even a reboot).

 

At the current speed, you will probably not see much of a performance boost for the Caviar Black unless it is a 2TB 7200RPM drive with a large cache.

 

I was wondering if you could do that, using RAID on your discs in an unRAID.  I had kicked around the idea of a RAID0 parity drive or possibly using RAID1 data drives to improve streaming performance for multiple devices.

 

There is no performance improvement of using RAID1 for data. The only place RAID1 provides anything useful is on the cache drive if you have one. In that location writes will be diminsihed slightly but they will be protected until the data is moved to the array.

RAID0 on parity works well at a cost.

Consider the cost of 1 Hardware raid controller, 2 Seagate 1.5TB drives and see if that is worth an extra 5-10MB/s.

 

I found the performance tweaks and kernel adjustment to provide a larger boost.

I went with my hybriid safe20% raid0/raid1 environment for a couple reasons.

2TB drives were costly, I scored the RAID controller for $99.

It gave me RAID0 on parity and RAID1 on cache with only 2 drives.

 

With that I was able to move my /home folders from vmware to an NFS mount on the cache.

Since the cache and parity are now always spinning. Any other write access does not have to wait for these drives to spin up.

 

In your case, I would recommend trying the software and kernel tweaks.

You can adjust the tunables in the settings page near the bottom.

 

Tunable (md_num_stripes):

Tunable (md_write_limit):

Tunable (md_sync_window):

 

When I adjusted

sysctl vm.dirty_ratio=80

sysctl vm.vfs_cache_pressure=100

 

The buffer cache did more work, but I also saw streaming writes hover around 50MB/s.

This weekend I plan to upgrade 4GB to 8GB and see what happens.

 

Okay, I think I understand the vm.dirty_ratio after reading about it.  Basically, it is the percentage of system RAM that is allotted as a cache file for IO operations.  So when I do a write, it goes to the RAM cache and then is written to the HDD when the cache gets full.

 

So on a large sequential write operation, the setting may have minimal effect.  But if you are sending a bunch of smaller pieces of data (like 4k or 512K random data) they accumulate in RAM and essentially become a sequential write when the cahce fills (or 30 seconds passes).  HDD's are fastest at sequential writes, so this speeds up your process.  I guess their could be some potential for data loss.  Say you write 1 GB of small random data in 20 seconds (50 MB/s) and then the power goes out.  That data dies in RAM then because it hadn't been written to disk yet, correct?  Small risk, but still some risk correct?

 

And it is defaulted at 10% because many people run Atoms with 1GB of RAM and can't spare the RAM?

 

And since I have 4GB of RAM and I'm not running any add-ons that might need it, I should be able to crank that setting up?  And if install add-ons or increase my number of drives, I may need to dial it back down in the future, or install more RAM.

 

And by leaving yourself only 800 MB of system RAM, you were over-committing, and causing the unRAID system to choke (kill processes).  So you set the cache_pressure to favor application cache over file system cache.  So setting at 100 means no applications will ever be killed to free up RAM for your file system cache. 

 

http://www.westnet.com/~gsmith/content/linux-pdflush.htm

http://rudd-o.com/en/linux-and-free-software/tales-from-responsivenessland-why-linux-feels-slow-and-how-to-fix-that

 

I feel like I have a grasp on those settings now and may play with them some.  Maybe set my dirty to a conservative 50.  Maybe even up my cache_pressure to guard against a massive amount of I/O ever killing a needed app.