Jump to content
We're Hiring! Full Stack Developer ×

The Quest for the Holy Grail of Storage … RAM Cloud


Rajahal

Recommended Posts

The Quest for the Holy Grail of Storage … RAM Cloud

 

Interesting article...discuss.

 

I like the idea and it actually seems fairly feasible to me.  After all, we all run low level (unRAID) servers based on RAM disks, though our data is of course still stored on HDDs and SSDs.  My main question is what happens if the power fails?  Sure you would be using a UPS and maybe even a backup generator, but if there were an extended outage (a week or more) then petabytes of data could be lost.

Link to comment

Interesting article. I just sent it over to our Cloud Product Manager and told him I have a customer who wants RAM Cloud... hahaha! He responds back to me 'why are you selling something we don't have' and 'it's scary having a sales guy who's more obsessed with storage than I am'.... ROFL.

 

I've been focusing a lot on my companies Cloud Storage and our Cisco VBlock to prospects and it's amazing how many mid-size companies just haven't put thought into disaster recovery and I kind of freak them out because I seem to like storage more than them...

 

Ahhh it's fun working for a CLEC who offers so many different products outside of just network

 

 

Link to comment

the funny thing is, most IPS's web farms, already run in ram.

 

our web server farm run entirely in ram..

we still have 100's of old "Rackables" servers (about 1/4 of the farm) that are not much more then a desktop board, a p4, a 36GB ultraSCSI drive and 32 gigs of ram.

the entire webserver loads into ram for speed.

we'll keep running those till they die (or someone does a cost of power usage vs new hardware cost analysis).

 

once they die,we replace it with a guest on a G7 blade as they fail

(nice hardware btw. 4 or 6 core 5600 Xeon and 384 gigs of ram... drool)

 

I also have some databases that run on 256gig ramdrives...

 

but nothing to the extent of what these guys are talking about..  they are talking mass storage..

 

 

Link to comment

I'm not sure how running from RAM only would solve any of the problems they described when opening the article. I'm not sure how they expect to simplify the way data is stored just because they have a faster storage media. I expect this type of thing just allows more data to be stored and more people to access it without the speed getting too slow. I suppose they can eliminate the schemes which decide which data is cached in RAM and how it is written back to the HDD's.

 

The big downside not addressed is that a RAM system would still need a close to real-time backup in case of some form of sudden catastropic power failure.  And yes, server facilities with their fancy redundant power schemes have gone down before.

 

Peter

 

Link to comment

I'm not sure how running from RAM only would solve any of the problems they described when opening the article. I'm not sure how they expect to simplify the way data is stored just because they have a faster storage media. I expect this type of thing just allows more data to be stored and more people to access it without the speed getting too slow. I suppose they can eliminate the schemes which decide which data is cached in RAM and how it is written back to the HDD's.

 

I may be out of my league here (and someone please correct me if I'm wrong), but I believe that with RAMdisks it is possible to avoid the standard reliance on the concept of blocks like HDDs and SSDs do.  That means that the data doesn't need to be broken down into little chunks (or at least not as many chunks), which results in a simpler storage system.  Imagine if an HD movie were stored massive 1GB blocks in a RAMdisk - scrubbing through the movie should be much faster.  Now imagine if there were no blocks at all - scrubbing through the movie should be nearly instantaneous.

 

The big downside not addressed is that a RAM system would still need a close to real-time backup in case of some form of sudden catastropic power failure.  And yes, server facilities with their fancy redundant power schemes have gone down before.

 

Indeed, and that is my primary concern as well.  I can imagine this type of system working like this: RAMdisks are the primary storage level, and SSDs/HDDs provide the backup (redundant) level.  All writes are initially stored in the RAMdisk and then as soon as possible re-written to the SSD/HDD level.  There would clearly be a lag as the SSD/HDD couldn't keep up with the speed of the RAMdisk, so a power failure would still result in some data loss, just not very much.  Data would likely need to be written to the backup level in a round-robin fashion so that the slower SSDs/HDDs would have a chance to keep up (that, or maybe the drives would need massive write caches).  A system like this would work much better for an application that could tolerate a bit of data loss during a power failure, such as a non-critical web server or maybe a file/image host (power failure = "an error occurred, please upload your file again").  It certainly would not work for a critical application, such as financial data.

 

In a sense, unRAID already runs on this RAMdisk scheme, just at a much smaller scale.  Let's say you have an unRAID server with 8 GB of RAM, a 60 GB SSD (cache drive), a 2 TB HDD (data disk1), and another 2 TB HDD (parity).  If you write a 4 GB movie to the server, the movie is first written to the server's RAMdisk - this allows the server to max out the GigE network connection.  Next, the movie is moved from the RAMdisk to the SSD cache.  Finally, the movie is moved from the SSD cache drive to the 2 TB data HDD while simultaneously the 2 TB parity drive is updated with new parity checksums.  If this current scheme were scaled up so that the RAMdisk, the cache drive, and the data/parity drives were all the same size, then the scenario I described above would be realized.  I do realize that the filesystem overheads and other factors come into play and slow down the whole process, but for the sake of this thought experiment I think they can be safely ignored.

Link to comment

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...