minmum configuration

[johnsonwc]

I've tried to get a response from Lime Technology with no reply, so I thought I'd try the forums.

 

I'm a noobie to unRAID so this question may be considered stupid. Luckily, I'm immune to being embarassed.

 

I have two possible applications in mind.

 

One is for low to moderate volume medium speed data. This is for a standard side scan sonar. Nothing special.

 

The other is high volume, high speen data. When I say high volume, I mean multiple terabytes of data in a day. High speed is about 90 megabits/s. This is for a bathymetric, synthetic aperture side scan sonar.

 

All of these will be housed in an underwater pressure vessel and powered by batteries.

 

Reffering to the first application, what is the minimum configuration for unRAID, hardware wise. Can I use just a single disk? I have very little space to work with.

 

Reffering to the second application, how fast can I store data. I'll be using a pc-104+ board with a gigabit interface, 2 gigs of RAM and a 1.6 gig Intel duo-core processor.

 

Thanks,

Walt

 

[bubbaQ]

As a rule of thumb, throughput writing to unRAID, with parity enabled, will average about 1 TB per day.

 

BTW, 90Mbps = about 1TB per day, so if that is your data rate, unRAID can probably handle it.  But I was confused when you said "multiple terabytes of data in a day."

 

For "multiple terabytes of data in a day" you will need striped RAID.

[johnsonwc]

Thanks,

  I was a bit imprecise with the "multiple terbytes" statement. In a single mission, which may cover several days, multiple terabytes would apply.

 

How about the single disk configuration? I can't find anything in the documentation saying what a minimum configuration is. I'd hate to have to use two different products for each NAS.

 

Walt

 

[Joe L.]

Thanks,

  I was a bit imprecise with the "multiple terbytes" statement. In a single mission, which may cover several days, multiple terabytes would apply.

 

How about the single disk configuration? I can't find anything in the documentation saying what a minimum configuration is. I'd hate to have to use two different products for each NAS.

 

Walt

 

You can use a single disk, but there will be no protection of data from a disk failure.  It will not be a "raid" array, but just a data disk shared on a LAN.  To protect the data from a disk failure, you need at least two disks. One data, and one parity.

[johnsonwc]

Thanks Joe,

 

I really appreciate your help. I can live with that I think.

 

As for the other application where I need high volume and high speed, I really like the fact that it isn't a striped "RAID", and that drives that aren't being used can be spun down automagically. That's a REAL BIG plus in battery powered systems. This is the first system that has that feature built-in that I have seen. I was hoping I wouldn't have to try and implement that on my own.

 

Thanks again Joe,

Walt

 

[SSD]

unRAID is available in a "free license" version that allows you to configure 3 disks (2 data + 1 parity).

 

I might suggest that you consider constructing a test array to ensure that unRAID provides the necessary write performance for your purpose.

 

[johnsonwc]

Thanks, I had planned to do that.

 

I have to say that this has been one of the most respinsive and helpful forums I've ever posted on.

 

Thanks to you all,

Walt

[lionelhutz]

 

90Mbps should be possible with parity (and therefore with single disk failure protection) enabled. With the right hardware unRAID can support this level of write speed. I can keep my 100Mbps network connection saturated during a write transfer if the writing computer is not accessing it's hard drive for something else.

 

90MBps is not possible with parity enabled. This is getting to the speed point where you'd need onboard or PCIe hardware card based RAID5. A typical 7200rpm drive can handle maybe 70MBps to 100MBps sequencial write speed.

 

Peter

 

[Joe L.]

The space needed for a second disk drive is not "that" much when compared to everything else involved, and... you can use a pair of 2.5 inch SATA drives perhaps to save even more space. (You would want to test their speeds, since sometimes they spin are slower than their 3.5 inch big-brothers)  Having parity protection might be possible, using 2 drives... if you have the room... and if you can get 2.5 inch drives with enough capacity.

 

Edit: the 2.5 inch notebook drives seem to be available at sizes up to 500Gig.

http://shop1.frys.com/product/5803123?site=sr:SEARCH:MAIN_RSLT_PG

 

You might even fit three, and get 1TB of protected space in the same physical space a single 3.5 inch drive would occupy.

 

Joe L.

[SSD]

 

90Mbps should be possible with parity (and therefore with single disk failure protection) enabled. With the right hardware unRAID can support this level of write speed. I can keep my 100Mbps network connection saturated during a write transfer if the writing computer is not accessing it's hard drive for something else.

 

90MBps is not possible with parity enabled. This is getting to the speed point where you'd need onboard or PCIe hardware card based RAID5. A typical 7200rpm drive can handle maybe 70MBps to 100MBps sequencial write speed.

 

Peter

 

I agree and definitely have concerns about unRAID satisfying a demanding need for fast write performance.  unRAIDs best technique for this is to use a cache disk and write to the array overnight.  It is unclear from the OPs message the duration over which his writes would occur.  But if you is going to copy 1T of data at 90 MB/sec, that would take a little over 3 hours.  To copy it to the array at 14 MB/sec would then take 20 hours.  Hey, that leaves 1 extra hour during the day to watch 1/2 a movie!

 

The fastest transfer I get over gigabit LAN is about 50 MB/sec.  I believe that is near the high end of what gigabit can practically support, even if the drives are faster.  Unless the application is running on the unRAID server itself, it seems unlikely that the OP is going to get near 90 MB/sec without using fiber or other higher bandwidth networking and maybe some SSD drives!

 

 

[bubbaQ]

The OP was pretty clear that his "high speed" rate was "90 megabits/s" ... we shouldn't confuse the issue talking about MBps

[SSD]

The OP was pretty clear that his "high speed" rate was "90 megabits/s" ... we shouldn't confuse the issue talking about MBps

 

You are 100% right.  I definitely missed that as I was posting this morning.

 

At 90 mb/sec (about 9 MB/sec), unRAID is more likely to be able to keep up.  But it really depends on the nature of how data is written. 

 

If there are lots of spurts and pauses, unRAID may not be able to quite deliver that average.  This is why I suggested doing the test.

[johnsonwc]

I really appreciate everybodies input. I've learned a lot. Thank You.

 

The data will be a continuous stream of data. The only pauses will be when the vehicle is repositioning itself or when it is surfacing to get a GPS fix to update its Inertial Guidance system.

 

I have an SSD that I want to use to store data on the slow system. That is where I was hoping to get by with a single disk since that vehicle is much smaller.

 

On the fast system, I have a number of laptop drives I'll be using in an array.

 

Thanks,

Walt

 

[TSM]

I think the question should be rephrased.  Is it going to make a few really big files, or is it going to make a lot of small files?  A continuous data stream could go either way.

 

I've noticed on my array that when I move large dvd images for example, I can get sustained rates well in excess of 90 megabits, with spurts fo well over 200, and my hardware is not that good.  But lets say I'm copying a few thousand files of about 100k a piece.  The speed goes down considerably.  Might be lucky to get 40 or 50 megabits per sec in this scenario. 

 

The other thing you should consider with very large files is.  Lets say you are using user shares, and are generating a file that might end up being 100 gigabytes in size.  You could have more than that available on the user share, but if you don't have an individual drive with that much space available, it will fail. 

[johnsonwc]

I have a lot of flexibility. I can control the file size. The software will write to the file until it approaches being full, then opens a new one and, if I choose, write to both until the first one is full and closes it, continuing to write to the new file.