We are wondering what others have found as bottle necks or performance improvements:

 

What would be consider "Overkill" in an unRAID build?

CPU has an influence on transfer speed over SMB, AFP, etc.

 

Parity disk is the IO bottleneck.

 

Ideally the parity drive needs to be SSDs to get the high speed IO.

Parity disk is the IO bottleneck.

 

Ideally the parity drive needs to be SSDs to get the high speed IO.

 

Parity speed is aligned with the slowest [involved] drive in your array. SSD's, as fast as they are, will still just sit there waiting on data from your slow spinners.

 

-edited for clarity-

Parity disk is the IO bottleneck.

 

Ideally the parity drive needs to be SSDs to get the high speed IO.

 

Parity speed is aligned with the slowest drive in your array. SSD's, as fast as they are, will still just sit there waiting on data from your slow spinners.

 

Not Quite True.  When doing a normal write to the array, only one data disk and the parity disk are involved.  (i.e., they are the only ones that are spun up.)  However, the parity drive has to be read to determine the current parity state (odd or even) and the new parity state calculated and written back to the parity drive.  The data drive only requires a write cycle to write the actual data.  Thus, the parity drive will have the greatest effect on write speed because both a read and write operation are required. 

 

However, I don't believe there are any SSD's with large enough capacities to make a reasonable sized unRAID setup possible at this time. 

 

 

The data drive only requires a write cycle to write the actual data.  Thus, the parity drive will have the greatest effect on write speed because both a read and write operation are required. 

Completely WRONG in an unRAID array!!!!

 

Both the data drive AND the parity drive must be read and then written.  Writing speed to the unRAID array is limited mostly by the SLOWER rotational speed of the two drives involved. 

 

Joe L.

Buy an SSD to use as a cache drive. Anything else is a waste of time and money IMO.

Ignore me...

 

Although my write speeds DID improve by getting a 2TB black as my Parity disk instead of a 2TB Green, especially in multi-user writes.

unRAID was never designed with high IO performance in mind.

It is mass storage at low cost will little overhead on desktop hardware.

It has a side benefit of low energy costs and good recover-ability in case of total disaster.

It is really a high volume storage appliance for long term data storage for things like media libraries and backups.

(most people use unUNRAID as a W.O.R.M. drive (write once read many))

if you need a high IO production file server, look elsewhere.. (PS, unraid is great for backing up that high IO production file server)

 

---

However....

You can tweak performance a bit by using 7200 rpm drives (reducing the power savings).

You can further tweak write speeds by using the built in cache drive feature (losing the parity benefit temporarily).

You can use an SSD for the cache drive for even greater performance.

Some people ~cough~ have gone so far as to use a raid for the cache drive (and even parity drives).

 

if price is not an object, you can tweak unRAID into a bit of a beast....  but.. it is not needed.

(for example, an ESXi server running both ZFS and unRAID as a hybrid server)

unRAID was never designed with high IO performance in mind.

It is mass storage at low cost will little overhead on desktop hardware.

It has a side benefit of low energy costs and good recover-ability in case of total disaster.

It is really a high volume storage appliance for long term data storage for things like media libraries and backups.

(most people use unUNRAID as a W.O.R.M. drive (write once read many))

if you need a high IO production file server, look elsewhere.. (PS, unraid is great for backing up that high IO production file server)

 

---

However....

You can tweak performance a bit by using 7200 rpm drives (reducing the power savings).

You can further tweak write speeds by using the built in cache drive feature (losing the parity benefit temporarily).

You can use an SSD for the cache drive for even greater performance.

Some people ~cough~ have gone so far as to use a raid for the cache drive (and even parity drives).

 

if price is not an object, you can tweak unRAID into a bit of a beast....  but.. it is not needed.

(for example, an ESXi server running both ZFS and unRAID as a hybrid server)

 

 

With the ever increasing data and disk sizes, a write speed of 30-40MB/sec is not going to be acceptable for long.

 

Especially given time machine doesn't work well with a cache drive and that is usually around the 10MB/sec mark for an average speed backup.

 

Performance enhancements need to be addressed IMO.

 

Perhaps there can be a performance mode where-by all writes are cached in memory and then (more) sequentially written thereby minimising read-write operations.

 

 

With the ever increasing data and disk sizes, a write speed of 30-40MB/sec is not going to be acceptable for long.

 

Especially given time machine doesn't work well with a cache drive and that is usually around the 10MB/sec mark for an average speed backup.

 

Performance enhancements need to be addressed IMO.

 

Perhaps there can be a performance mode where-by all writes are cached in memory and then (more) sequentially written thereby minimising read-write operations.

first off, i want to point out that I am not arguing with your concern, I am pointing out you can do it now, and it will only get better...

 

 

The bottleneck is the mechanical disk. that is the unfortunate nature of the beast.. and as they get larger, they are getting only marginally faster as they get more data per square inch.

the newest segate 3TB (1TB per platter) drives are much faster then any brands 2TB of just 3 years ago. they are about on par with the velociraptors of a few years back. They are now tweaking 1.25TB platters. by design they have to be faster (unless they slow down the platter).

 

also SSD's are getting larger and cheaper.. in a few years as we start to see 8TB+ mechanical drives, we will probably see 2TB SSD's at a consumer affordable price. you would be able to at least have an SSD cache drive or maybe a full SSD server for beastly performance.

 

 

a write speed of 30-40MB/sec :

I am way past that speed with my systems.. I am pretty sure I posted the speeds for my systems..

My Write speeds to Cache:

"Goliath" gets about full bandwidth of gigabit. 85-110Mb/s..

"Spartacus" gets the same unless I am pounding my ZFS array at the same time from another guest..

"unRAID Mini" is a bit slower. it hovers around 65-85MB/s due to the weak processor and laptop mechanical spinner Cache drive.

 

I'll admit TM backups can be a little slower due the fact that you cant use a cache drive. but mine are much faster then 10MB/s about 4 times that.

yes the first backup can take an hour. since it is a transparent process to the user. speed is not really that important user as long as it can reasonably keep up. that much data cant change each hour? (it is still about the same speed as it was to my time capsule)

 

Keep in mind that a major bottleneck is your source drive. your average I5 win7 dell desktop will have a bit of a hard time breaking 45MB/s in transfer speed. I deal with this issue all day work when moving TB's of data backing up desktops or restoring over copper to SANs arrays with speeds that would make you drool.

 

In my home network, all of my desktops are SSD and all of my Physical Windows servers are RAID5 or RAID6.

I can push data back and forth at full NIC speeds. My unraid reed speed is my bottleneck.

I'll admit i have an SSD cache in one server and a RAID 5 cache drive for my other server. with write speeds of over 450MB/s to both servers, this does eliminate my write issues when using cache writes. I know thats overkill to the average user, but it shows you can push the envelope.

 

I think you are worried about the future, but the future will be self healling...

the unRAId servers of a few years ago were writing straight to the array at 25MB/s ish transfer speeds. 2 years ago 40MB/s ish. I am getting 60ish with the new seagates (and yes i dislike seagate. but im running them due to low cost).

Your performance really depends on your server hardware.

 

PS. i am thinking about building a dedicated TM unraid with only 2 disks and see how that works. I really want to get my Timemachine backups off my unraid. especially since 2 of my MACs will be "wireless only" as soon as i move them into the basement studio.

 

 

EDIT:

I noticed your build is quite respectable. I am going to guess your WD greens and the slow cache drive are your bottlenecks. I bet you can squeeze quite a bit more from that build. add another black for TM only drive and upgrade to a 7200 RPM cache drive or even SSD.

 

 

 

 

 

 

Yes, the cache drive is rather disappointing. I'm going to get myself a 500GB Momentus XT (4GB Flash + 500GB disk), which allows for 105MB/sec.

 

With servers that have large amounts of memory, reordering writes and subsequent reads into a more efficient order should be possible.

 

I'd imagine UnRAID does do some sort of ordering but at a very minimal level, I would expect to use say, 4GB of my 8GB to be used as a write cache so writes can be done as and when appropriate...

That XT only uses the flash cache for reads afaik.

Storing data in memory is risking data for the sake of greater performance. IMO that's not something I'm sure I'd want. UnRAID works great for storing tons of movies and making system backups, things that don't change in a very dynamic manner. It's reliable and my data is pretty safe. Lots of packages offer more speed and I may set one of them up myself for that purpose. I'm not sure I'd want unRAID to move towards methodologies that could impact the safety of data as a large write cache might. It's also nice that my system only requires about 2gig of memory as it sits fit now...

 

 

Sent from my iPad using Tapatalk

That XT only uses the flash cache for reads afaik.

 

Yup, but it is a 500GB 7200RPM drive which gives the 100+MB/sec writes.

 

I run a few things off the cache drive which would benefit from the flash.

 

 

Storing data in memory is risking data for the sake of greater performance. IMO that's not something I'm sure I'd want. UnRAID works great for storing tons of movies and making system backups, things that don't change in a very dynamic manner. It's reliable and my data is pretty safe. Lots of packages offer more speed and I may set one of them up myself for that purpose. I'm not sure I'd want unRAID to move towards methodologies that could impact the safety of data as a large write cache might. It's also nice that my system only requires about 2gig of memory as it sits fit now...

 

 

Sent from my iPad using Tapatalk

 

If you currently have a power outage during writes, you still loose (a minimal) amount of data. I've even corrupted a Time Machine sparsebundle through knocking the server off by accident... (recovered it thankfully).

 

Besides, I said a 'Performance Mode', so users like me can decide to risk it because I have a UPS and the server only does backups, so if the transfer fails I can just initiate the copy again.