In the next release, can you please include SMP support? Since almost all CPU's out there are multicore nowadays, it seems prudent to use the extra cores, even if it doesn't make a huge performance difference (although it's great for vmware setups running on unraid).

Actually, is is pruden to *not* use SMP.

 

If you do a little searching, you will find SMP discussed, and also find that it is not just a matter of including SMP support.... unRAID is not thread safe, and thus can't safely support SMP.

 

unRAID is a high availability and fault tolerant application first and foremost.

Actually, is is pruden to *not* use SMP.

 

It is prudent to include it at sometime in the future.

Just not prudent to compile it in at this point in time without code review and beta testing.

I think once unRAID is thread safe, it would go a long way to smoother operation.

One of the downsides of the current environment is pauses during heavy buffer cache flushes.

So much so that it affects applications and/or network transfers.

 

>> In the next release, can you please include SMP support?

It doesn't hurt to ask. I'm sure it will come eventually.

It seems someone here has it already working... no?

 

 

Yes, Josetann had it working but partially by limiting what processes were on what cpu.

Someone else found a block where they put in semaphore locks to prevent kernel OPS when the array was stopped.

 

I don't know if more is needed to make it SMP safe.

I've been running my custom unRAID kernel with SMP support for the past 3 weeks weeks with no problems.  The reason I want SMP support is because I'm using Virtual Box to run VMs on my unRAID. 

 

I've looked at the unRAID kernel driver and compared it to the Linux driver that it's based on, and the only fix I had to make was the one I mentioned in the other thread:  http://lime-technology.com/forum/index.php?topic=1749.60.

 

I wouldn't really recommend anyone try what I'm doing unless you know what you're doing.

You know, even as a worst case, I'm ok if there is a spinlock in all the kernel-level unraid code -- what seems to be the big CPU hog is the user filesystem mappings.

 

Having said that, scott, can you post a patch file for your changes to unraid.c?

 

 

Sorry, just noticed that you posted it in the other thread.

 

What are you doing to check for corruption? It might be subtle ... I think copying a couple of large files at the same time and then doing md5 checks on the source and destination files would be a start...

I may try this patch in a day or so. I'll post my results.

Sorry, just noticed that you posted it in the other thread.

 

What are you doing to check for corruption? It might be subtle ... I think copying a couple of large files at the same time and then doing md5 checks on the source and destination files would be a start...

 

To be honest, I havn't really done any testing for corruption.  The only thing I've done is compared the code in unraid.c to the code in raid5.c in the Linux kernel (what unraid.c is based on) to determine the proper lock locations.  Also, I'm a programmer by profession if that gives me any more credibility. :-)

is it possible to keep unraid on the first cpu ?

 

Why not let parity calculation use multicores ?  (Xor-calculation as seperate posix threads)

 

could this maybe be the solution to speed up the write-speed ?

 

What if we could calculate 4 bits in one cycle ? Theoretical that would boost the speed up to 4 times.

 

There must be a way to use the second cpu without making it "unsafe" for unraid ?

 

 

 

I don't believe speeding up XOR calculations will result in a measurable improvement.  unRAID is I/O bound, not CPU bound.  I suspect the CPU is idling most of the time, waiting on drives.

unRAID is I/O bound most of the time. Whenever I do a top I see high values in %wa.

This is because of the nature of parity calculation on blocks.

I.E.  Read block from data drive, read block from parity, calculate to remove old data block from parity, calculate new data block for parity, write parity block, write data block.

 

Notice there are two reads and two writes for every write to the hard drive.

I suppose the only way to really speed it up is with allot of caching at the drive or controller level.

I've been even toying with the idea of an advanced controller with cache memory to see if it will speed up unRAID writes.

 

I suppose if SMP were enabled the system would be more responsive.

I.E. when heavy writes were occurring, other programs would be serviced sooner.

But if you have a small amount of memory, you will still eventually block until the buffer cache is flushed enough to service your program (if it is doing I/O).

 

The cache drive was a wise investment in time to speed up writes to the system over the network.

I don't use it, but I can see it's value if you are streaming allot of data to the system.

 

This is not to say SMP should not be visited, on the contrary, it should be. It will make our systems more responsive with the ability to run more network based applications in a smoother manner. I just don't see it improving I/O that drastically.

 

I have compiled unRAID and run it on an SMP system, as well as testing it on a non-SMP system and severely underclocked the CPU.... the result is essentially no real-world change in speed.  Some benchmarks that are cached improve, but that's all.

 

Also, I don't think caching will help much in the real world, as unRAID (should) be writing through cache.  You might be able to get your initial file write cached and then flushed after your application completed the write, but that would be risky from a data safety perspective if you did not have some fault tolerance or persistence on the cache.

Also, I don't think caching will help much in the real world, as unRAID (should) be writing through cache

 

If I use a battery backed caching controller it should not be an issue. I do use a UPS.

If I do write-back on the parity drive alone and when need be run periodic parity checks, I can get by.

My real issue is the pauses during operation as unRAID flushes writes.

This in turn causes my torrents to slow down and sometimes disconnect.

renice pdflush periodically.

I think where I'm going to fail is the arcea controller comes up as SCSI.

Perhaps we need s'more enhancements from Tom to add support for USB (arrays, drives) and SCSI drives.

my old intel providence had onboard adaptec 7880 and it won't work... that was a nice shot with 12x 18GB's but no go...

 

scsi support would be great, those drives are almost 12 years old and still no damage, i just love scsi.

i just love scsi.

As do I, then again, with today's technology and advanced controllers SATA is coming close.

I did build my heavy duty workstation with SAS.. but for general desktop and media storage, SATA is worth while.

 

I have a whole box of smaller scsi drives.. I'm trying to decide to off them or use them. 

I have a whole box of smaller scsi drives.. I'm trying to decide to off them or use them.

 

Great source for strong magnets 

 

I have a whole box of smaller scsi drives.. I'm trying to decide to off them or use them.

 

Great source for strong magnets 

 

I heard that, I think I'm going to take apart some 2GB SCSI drives I'm about to toss.

I have a whole box of smaller scsi drives.. I'm trying to decide to off them or use them.

 

Great source for strong magnets 

 

I heard that, I think I'm going to take apart some 2GB SCSI drives I'm about to toss.

I think I have a few that are 500Meg scsi... big in their day, but now, tiny...  their magnets might be their most valuable component. 

I still have my first harddrive... a Tandon TM-501 full-height 5 1/4".... a whopping 5 megabytes!

 

 

I still have my first harddrive... a Tandon TM-501 full-height 5 1/4".... a whopping 5 megabytes!

 

 

 

Ah, ancient computer hardware.

 

I was at the Computer History Museum in Mt. View, CA yesterday for an event - any other Sun Micro alumns in the house?  It was simply amazing to see so many computing devices that I have worked with (as a developer), sold (as a salesman), and owned.  Having them be in a museum makes me feel more ancient than I am.

 

I got to see the Babbage engine in use.  Oooooooooooh!

 

http://www.computerhistory.org/

 

 

Bill

I still have my first harddrive... a Tandon TM-501 full-height 5 1/4".... a whopping 5 megabytes!

 

 

I still have an 8" floppy drive on a shelf downstairs...  Talk about heavy-metal...