Right. I have a suggestion that I'd like to make to Lime Technology. I purpose having a Dual Parity Drive setup, configured as a RAID 0, to improve in write speeds in unraid. I might get shot down in flames over such a suggestion, but want to hear peoples thoughts on this suggestion:

 

The Benefits:

 

- Instead of have a single parity drive constantly having parity data being written onto one data bus/one parity disk, you could lighten the load and have two parity disks doing the job, resulting in using two buses, two disks instead of the one, thus improving parity writes by double and a lesser impact on unraids write speeds, whilst still having parity in same time as writing your data.

 

- Using this dual parity disk scenario, if one of the parity disks where to die, you simply stop the array, replace the failed parity disk, start the array and rebuild the the RAID 0 parity disks again from scratch and your back on track again.

 

The Disadvantages:

 

- Obviously, a RAID 0 setup is in no way the safest method of writing data, leading to double the chance of losing you parity data.

 

- If you're in this scenario of one failed parity drive, being in a RAID 0 setup, both the existing working and the replaced parity disks would need to be rebuilt from the current data disks on the array.

 

Note: We all know of the flaw in a RAID 0 setup that is you have double the risk of your data being lost, but then again, any data disk or even a cache drive could fail at any given time too, it would leave your array interrupted, incomplete and insecure. In the event that your cache drive were to fail, you're on the path to never recovering the cached data that wasn't written to the array! (You better hope that you still have another copy of it).

 

I ask Tom (or the other advanced unraid guru's) to answer these question below:

 

1). It is viable to introduce such a feature?

2). Is it worth the effort? (pending answering yes to the first question) & most importantly

3). Would it improve write performance by double giving that the role of one would be split into two buses/disks?

 

I've thought about this idea for quite a while and I would see a great benefit in it. But again, I'll put it to the people and the brains behind unraid.

 

Your thoughts, comments, ideas? Thanks.

 

Short Answer: NO BENEFIT.

 

You're bound by the rotational latency of the physical medium. This still exists on the data drives. You still need to do a read old sector, write new sector I/O pair on the data disk. That will be your bottleneck.

 

You would need to do a RAID-0 array PER EACH AND EVERY disk in the array to experience ANY benefit.

 

ie:

 

raid-0 parity (drive p1 + p2)

raid-0 data 1 (drive a + b)

raid-0 data 2 (drive c + d)

raid-0 data 3 (drive e + f)

i thought parity does a write read write thoughor something supidly complex, ie double the writes of a data disk. the problem is raid will INCREASE latency making it even slower.

Thanks for you honesty Barzija , I guess recreating the wheel (ie: to improve unraid's performance already), would prove to be difficult, giving the procedures on how it works. I thought I had it with this idea, but obviously not.

 

I might get shot down in flames over such a suggestion

you can bet your ass you will.   

do some search so you don't think you invented hot water.

 

No. Nothing that crazy. It's quite simple. The operations involved in unRAID are:

 

A1) Read existing sector from Data Drive

A2) Read existing sector from Parity Drive

 

B) Calculate new parity checksum value to write

 

C1) Write new sector to Data Drive

C2) Write new sector to Parity Drive

 

 

Steps A1/A2 are done in parallel. Steps C1/C2 are done in parallel.

 

What was proposed was to speed up steps A2 and C2. You still have to wait for step A1 to finish before you can perform step B. Then you also have to wait for step C1 to finish before you can move on. Step B is not a factor in what's the bottle neck.

Hi terrastrife. I can see what you mean, though I'll explain my idea in a little bit more detail. I was thing about this my suggestion by which having your desktop running in a RAID 0 array would be way faster then a single traditional disk or RAID 1, 5, etc...

The idea I'm purposing works the same way, so the two parity disks would be a separate array, formatted as a RAID 0 set (this itself would be fooling unraid to think it has one single disk to write parity too, but in fact it has two data buses and two separate disks to split the parity read/write data too, thus improving the performance).

So behinds the scenes, you have two disks acting as parity, but unraid only seems it as one parity disk (much like a Windows desktop formatted in a RAID 0 array, you have the performance and the space of two drives seen as one drive). In theory, if it were to be designed this way, it wouldn't be slower at all, in fact, it would improve unraids performance (Again, in theory!).

 

i thought parity does a write read write thoughor something supidly complex, ie double the writes of a data disk. the problem is raid will INCREASE latency making it even slower.

BRiT left off a few critical steps that prevent making the parity drive a "raid 0 set" from having any effect.

 

The operations involved in unRAID are:

 

A1) Read existing sector X from Data Drive

A2) Read existing sector X from Parity Drive

 

B) Calculate new parity checksum value to write

 

Rotate data drive disk platter a full revolution to position read/write head over sector X again for writing

Rotate parity drive disk platter a full revolution to position read/write head over sector X again for writing

 

C1) Write new sector to Data Drive

C2) Write new sector to Parity Drive

 

Rotate data drive disk platter a full revolution to position read/write head over sector X+1 for reading

Rotate parity drive disk platter a full revolution to position read/write head over sector X+1 for reading

 

Steps A1/A2 are done in parallel. Steps C1/C2 are done in parallel.

 

What was proposed was to speed up steps A2 and C2. You still have to wait for step A1 to finish before you can perform step B. Then you also have to wait for step C1 to finish before you can move on. Step B is not a factor in what's the bottle neck.

 

Since the disks must rotate a full revolution between each read and subsequent write of a sector when writing to the array, the "bottleneck" is the time it takes to physically rotate the disk. 7200 RPM drives rotate 120 times per second.  5400 RPM drives rotate 90 times per second, they will therefore introduce a rotational delay of between 1/60th and 1/45th of a second per stripe involved. (two rotations to read/write a sector and position to read the next).  The slowest rotation speed disk involved dictates the overall speed you can write to the array.  

 

Finally, unRAID operates on contiguous sets of sectors, referred to as a "stripe"  A full "stripe" is read/written at a time.  

The stripe size is configured to be less than the size of an entire cylinder on a disk.  This is for efficiency, A stripe will contain many sectors.

Therefore, a "stripe" is read from both data and parity drives, parity calculated, the platters rotate to position the disk head to write the entire stripe, then the platters rotate to read the next stripe, etc...

 

Joe L.

 

No performance improvement and you increase the chance of a failure of the parity volume. This is not what we're looking for

No performance improvement... here's why.

 

If you write 10GB to a data drive, you have to Read 10GB and write 10GB on that data dfive

 

Ignore the parity drive.  Assume your data drive will read at 110MB/sec and write and 85MB/sec.

 

You need 90.9 seconds to read and 117.6 seconds to write for a total of 208.5 seconds.

 

That is theoretical max unRAID throughput (TMURT) of 47.9 MB/sec.  I don't care if you have a rocket sled of SSD arrays for parity, NOTHING the parity drive can do will affect the fact that you simply can't do anything with the data faster than 47.9MB/sec.

 

Now if your parity disk is dogsqueeze slower than your data drive, then changing to a faster parity will help... but you are only going to go as fast (theoretically) as your slowest drive.

Thanks bubbaQ, Joe L and to others that explained the technical side of unraid with regards to this idea. In my mind, I kind of figured if their was going to be any improvement to unraid's architecture, it would of been done by now, but you can't blame a guy for trying . bubbaQ mentioned SSD's, to use this in unraid would be a little way ahead of of our time at present, I see it as premature technology and albeit you'll need to be loaded to even build such a server, but the bottleneck still remains regardless of whether you had traditional or SSD disks. Thanks.

Since the disks must rotate a full revolution between each read and subsequent write of a sector when writing to the array, the "bottleneck" is the time it takes to physically rotate the disk.

 

In practice, I d not believe this is true.  I ran some back-of-the-napkin calcs and came up with a theoretical max throughput with this rotational delay idea, and the actual throughput I was getting was significant HIGHER than he theoretical max.

 

I believe what happens is more like this (with larger files):

 

read a sector, and the drive pre-fetches the whole track, or portion thereof. (cost: 1 revolution)

many writes take place, and are buffered (no cost)

drive reorders writes into sequential order and flushes buffers (cost: partial rev to start, then one revolution to write the track)

 

Since the disks must rotate a full revolution between each read and subsequent write of a sector when writing to the array, the "bottleneck" is the time it takes to physically rotate the disk.

 

In practice, I d not believe this is true.  I ran some back-of-the-napkin calcs and came up with a theoretical max throughput with this rotational delay idea, and the actual throughput I was getting was significant HIGHER than he theoretical max.

 

I believe what happens is more like this (with larger files):

 

read a sector, and the drive pre-fetches the whole track, or portion thereof. (cost: 1 revolution)

many writes take place, and are buffered (no cost)

drive reorders writes into sequential order and flushes buffers (cost: partial rev to start, then one revolution to write the track)

 

As I mentioned earlier, unRAID works on a set of sectors, known as a stripe.  Tom once mentioned a "stripe" is 128k.  Thus, the rotation of the disk between read and write only occurs every 250 sectors and the bulk of a track (hopefully) is read at once into the disk buffer for the start of the subsequent read.  With luck, the read-ahead buffer in the disk will get the remainder of the track.    In the same way, writes can be buffered to a point, until you run out of buffer on the disk.

 

Joe L.

No performance improvement... here's why.

 

If you write 10GB to a data drive, you have to Read 10GB and write 10GB on that data dfive

 

Ignore the parity drive.  Assume your data drive will read at 110MB/sec and write and 85MB/sec.

 

You need 90.9 seconds to read and 117.6 seconds to write for a total of 208.5 seconds.

 

That is theoretical max unRAID throughput (TMURT) of 47.9 MB/sec.  I don't care if you have a rocket sled of SSD arrays for parity, NOTHING the parity drive can do will affect the fact that you simply can't do anything with the data faster than 47.9MB/sec.

 

Now if your parity disk is dogsqueeze slower than your data drive, then changing to a faster parity will help... but you are only going to go as fast (theoretically) as your slowest drive.

 

I agree with this and my findings of doing hardware RAID0 on parity proved there was a little throughput enhancement. I think it was on the order of 3MB/s.

 

I also tried with an areca PCIe x8 hardware raid controller. unRAID recognized the drives and RAID0 array, but could not assign the device. *sigh* Since this is a caching controller, I bet it would have had another "small' incremental improvement.

 

Where the extra RAID0 throughput comes in hand is when doing multiple writes to multiple drives.

Here is where you need the parity drive to operate as fast as possible.

 

My tests with 2 1tb samsung drives in raid0 write mode of 10Gb was 149MB/s. Read was 220MB/s.

I have not tried with 2 Seagate 1.5tb 7200rpm drives yet, but plan to.

 

What I really want to do is a SAFE33 setup where the outer tracks are raid0 and inner tracks are raid1 (cache) and see how this performs). The areca allows this sort of setup, but until unRAID supports the RAID portion of the areca interface, we cannot move forward.

 

Currently unRAID (or emhttp) only supports the areca drives in pass through mode.

Just a thought in regard to building larger and faster arrays. Having several Areca's and configuring 3+1 hardware based raid5 volumes and having 20 of them in unRAID (if the above Areca problem would be solved and the Areca supported). Array will grow immediately to 20*3 data drives, the Areca would do alot of caching, disadvantage would be that each 3+1 Drive Group will spin up/down as a whole. Thoughts? 

 

Edit: would be pretty cool expanding a Raid5 group online and then growing reiserfs on it (perhaps stopping and restarting emhttp and recalculating parity). That way max total (logical) drives in unRAID remains 20 but whole arrays is able to grow.

The Areca problem might have been user error during my testing.

I may have had the filesystem mounted during a test when I tried to put it into the unRAID array.

 

My tests this weekend revealed quite a few new exciting findings for unRAID.

 

1. It is possible to use RAID0 on the parity drive and improve performance using a hardware raid controller.

2. Both the Areca and 3ware raid cards are supported (I only noticed the 3ware card was supported after inspecting the /lib folder and figured I would test it.

3. with the Areca we can have SAFE33 mode (With spindown support).  I.E. carve up a 2 drive array into 2 raid volumes. 70% of the volume RAID0 for parity, 30% of the volume for RAID1 cache.  This is really exciting for me. It's exactly what unraid needs.

 

During my tests.

1. Using the 3ware on PCI-X with 2 Samsuing spinpoint F3 1TB drives in RAID0. I was able to get 43MB/s dd writes to a parity protected drive.

2. Using the Areca on PCIe x8 with 2 Samsuing spinpoint F3 1TB drives in RAID0. I was able to get 40MB/s dd writes to a parity protected drive.

3. Using the Areca on PCIe x8 with 2 Segate 1.5TB 7200 RPM 32MB Cache drives in a raidset with 1 Volume as RAID0 and 1 Volume as RAID1,  I was able to get 40MB/s dd writes to a parity protected drive. 80MB/s on RAID1

 

FWIW, the spinpoint F3's 1TB are really solid performers. I was getting very fast numbers on 3ware in RAID0 mode. in the order of 149MB/s writes and 270MB/s reads.

 

By using the SSD as a cache parity disk, eliminating rotational delays and seeking, I was able to get darn close to the theoretical maximum.  Consider this:

 

If you are writing 1GB to unRAID, you have to read 1GB and write 1GB.  There has been a lot of discussion of the optimization from reading a bunch at once, then queuing the writes and writing them back in a batch.

 

Consider a drive (such as mine) that reads at 100MB/sec and writes at 75MB/sec.  So theoretically it takes 10 seconds of reading and 13.3 seconds of writing for a total of 23.3 seconds to write 1GB to unRAID.  That means 43MB/sec of maximum theoretical throughput.  Using this formula, and I/O specs for your drives, you can easily calculate the maximum theoretical throughput for your system. 

 

I was able to do consistently over 40MB/sec (i.e. 93% of theoretical max) with the SSD as cache parity, and the WD Blue as data.  I'll try to scrounge up some other drives to do some more testing.

 

 

I wanted to quote the above because I was able to do close to the same speed using two quality drives in RAID0 on a hardware caching controller.

When I did the SAFE33 with the two seagates I was able to carve a 2TB Parity drive and a 400GB cache drive out of the two drives on an areca controller.

 

FWIW, the 2 port PCIe x1 Areca goes for around $175 new, and $125 or so used.

 

Kind of like a matrix raid array, one being the traditional unraid array and multipling it by 3, thus being the raid 5 array too, right? Would it work, though you would loose one lot of an entire unraid array alone just for parity (a lot of disks gone just for parity in this matrix raid array) and the remaining two unraid raid arrays would be used for data. I would suspect it would be impossible to configure unraid be compiled to work this way, right? If this is what you mean, I see no benefit at all as you could build three unraid servers and have much more utilization of disks being used for data storage, not an entire unraid array lot just used for parity.

 

Just a thought in regard to building larger and faster arrays. Having several Areca's and configuring 3+1 hardware based raid5 volumes and having 20 of them in unRAID (if the above Areca problem would be solved and the Areca supported). Array will grow immediately to 20*3 data drives, the Areca would do alot of caching, disadvantage would be that each 3+1 Drive Group will spin up/down as a whole. Thoughts? 

 

Edit: would be pretty cool expanding a Raid5 group online and then growing reiserfs on it (perhaps stopping and restarting emhttp and recalculating parity). That way max total (logical) drives in unRAID remains 20 but whole arrays is able to grow.

FWIW, the 2 port PCIe x1 Areca goes for around $175 new, and $125 or so used.

 

WeeboTech, which model is this exactly? Isn't a multiple port Areca better in terms of not wasting slots in the server, just wondering!?

FWIW, the 2 port PCIe x1 Areca goes for around $175 new, and $125 or so used.

 

WeeboTech, which model is this exactly? Isn't a multiple port Areca better in terms of not wasting slots in the server, just wondering!?

 

4.8.12.16.24 port areca's use x8 slots.

2 port areca uses an x8 slot.

 

I'm using the arc-1220, I got an open box special a couple years back for around $300.  8 port caching raid with MAID support and multi volume carving in a raid array.  slick card.

 

I mention the 2 port card as it could be used by the smaller slots and provide a large benefit to a system in speed.

For multiport controllers, you could use those smaller drives, chain them together to make a very large parity drive and gain in speed and usability of the small drives. (I have not tried this with more then 2 drives yet).

I was also thinking about putting a SSD drive for cache. Propability to fails is perhaps less than a traditional disk and about 120GB are probably goind to do it as a cache drive. Part of it can also be used as swap for VMware and the like.

I would agree and believe it is a good choice these days.

For me one of the 60GB Vertex drives would do fine, I never move that much data to the raid array at one time.

 

FWIW, there is a way to boot from a hard drive using grub4dos, so booting unRAID from a hidden directory on the cache drive (just booting bzimage and bzroot) would be many times faster then reading them from the usb flash drive.  Probably not a big deal, but when you are booting multiple times to test something, this can be really handy!

 

Or for larger systems, shutting them off and using WOL to wake them up saves a few moments of USB flash booting.

Yeah, also putting the thousand small YAMJ files on the Vertex... I am shutting off sometimes and then WOL the server automatically when needed and booting from the USB drive is slow. I will report sooner than later because I am hardly thinking about going with one 120GB Vertex, at lest 60GB.

 

Btw, what happens when unRAID boots from an SSD and can't find the registered serial number on it? Is it automatically referencing the USB stick?

3. Using the Areca on PCIe x8 with 2 Segate 1.5TB 7200 RPM 32MB Cache drives in a raidset with 1 Volume as RAID0 and 1 Volume as RAID1,

 

You can also use this technique when you want to install a new data disk that is larger then your parity disk.  It also obviates any penalty from lower-performance (i.e. green) drives as parity.

Yeah, also putting the thousand small YAMJ files on the Vertex... I am shutting off sometimes and then WOL the server automatically when needed and booting from the USB drive is slow. I will report sooner than later because I am hardly thinking about going with one 120GB Vertex, at lest 60GB.

 

Btw, what happens when unRAID boots from an SSD and can't find the registered serial number on it? Is it automatically referencing the USB stick?

It looks to match the GUID of the device with the UNRAID label with that in the .key file., so keep the flash drive labeled as UNRAID, and the .key file on it.

For comparison purposes, any numbers in this situation?

 

0a) Using a non-caching controller with a single parity drive. I was able to get __ MB/s dd writes to a parity protected drive.

0b) Using a caching controller with a single parity drive. I was able to get __ MB/s dd writes to a parity protected drive.

 

The writes/reads from the raid-0 portion are very nice. 

For comparison purposes, any numbers in this situation?

 

0a) Using a non-caching controller with a single parity drive. I was able to get __ MB/s dd writes to a parity protected drive.

0b) Using a caching controller with a single parity drive. I was able to get __ MB/s dd writes to a parity protected drive.

 

The writes/reads from the raid-0 portion are very nice.  

 

I don't have the numbers at hand. I was only really interested in RAID0 at maximum speed which was 43MB/s.

Nothing I could do hardware wise got it higher.  

I did not test scheduler or adjusting the md driver parameters. It was really a test to see how close it would come to Bubba's theoretical max which was 40Mb/s.  On the outer tracks and new filesystems it hit that max.

 

Inside and full filesystems will differ, this provides a possible point of reference.