Now that the beta has a "cachedrive" feature i would like to suggest that this cache also be allowed as a RAMdrive.

 

For me at least 1GB of RAMdrive would be more than adequate for an intermediate write cache.

 

unRAID doesn't use anything like the amount of RAM in my box and even if it did adding more RAM is cheap.

 

Not a bad idea, but you might want to add a large UPS to your shopping cart, since unlike a cache drive on a hard disk, the files on any ram used for that same purpose are gone when the power fails.  You would need to move any files to physical disks if a power failure is detected, and the UPS sized to make that possible.

 

Joe L.

See the Gigiabyte Ramdisk

 

http://www.giga-byte.com/Products/Storage/Products_Overview.aspx?ProductID=2180&ProductName=GC-RAMDISK

http://www.giga-byte.com/Products/Storage/Products_Overview.aspx?ProductID=2678&ProductName=i-RAM%20BOX

 

I'm considering one of these. Although not for a cache disk...

For programs, var space and a guess nfs root directory for mounting by other machines (less drives to spin up)

 

In the mean time what I did was added more ram to my machine and recompiled the kernel with PAE.

So now I have 8GB of ram in the machine.

 

By doing

 

mount -t tmpfs -o size=1m,mode=0775 tmpfs /somemountpoint

 

I was able to create a 1MB ram disk for remounting stuff via NFS (my own test).

 

 

The beauty of all the ram is that when the kernel is retuned, all of the directory entries are in ram so drives do not spin up as often.

I can run an slocate every night without the drives spinning up.

I know PAE is supposed to have a huge performance hit, however I;m using a 2.6ghz cpu in a very underutilized environment.

 

So the feasibility of lots of ram and a ram disk does exist.

 

I do think the hardware product would be a better buy.

Yet, I've heard this device has a hard time in linux with certain controllers.

 

I was thinking of using one of the PATA ports and a PATA to SATA converter.

I know the speed would be slightly less, but it will save an SATA port, Use a port that is unused and would still provide enough performance for my needs.

 

Using my ram to buffer writes is a little scarey, especially if it is going to be for HOURS.  When I copy my latest photos to the array, I need to know they are very secure and can be deleted from my workstation.

 

Using my ram to buffer writes is a little scarey, especially if it is going to be for HOURS.  When I copy my latest photos to the array, I need to they are very secure and can be deleted from my workstation.

 

the solution would not be for everyone and you would likely not even think about it without a UPS. But for "some" its all the advantages of an always on massively fast cache drive without the disadvantage of physically installing a drive.

 

Once the mechanism for adding RAMdrives is there as part of unRAID "rather than just another we know linux hack" WeeboTech ideas become very nice indeed. Adding a few GB of RAM is comparatively cheap and there is definite scope for it making unRAID much slicker in many ways.

 

I like the gigabyte ramdrive cards as they allow for battery backup but its alot of expense IMHO. For sure if someone has the cash then terrific but a good old fashioned ramdrive will be essentially free to alot of users with way more ram in theor boxes than they need (im sure there are loads out there)

 

 

Quite frankly, I don't see a ramdrive of too much use to the normal unRAID crowd.

It really seems to be something usefull more so to the we know linux people in a way to utilize idle hardware more efficiently.

 

Now if gigabyte created a ram drive capable of 8-16GB then I could see it being a cache drive.

I would think the cache drive would have to be enough to hold a DVD or two to be of any use to the normal crowd.

 

If unRAID were 64Bit then all of ram slots on modern motherboards would be useful

(unless an alternate PAE compiled kernel were made available)

 

Right now I'm in PAE mode and may use the RAMDRIVE as a direct drive for data storage or for swap (have not fully decided, I bought one last night).

I'm even considering two and rigging an internal hardware raid port multiplier to give me RAID0 so I can have the 8GB.

Either that or a new supermicro Low Voltage xeon capable of 32GB of ram. These babies pull 30W of power, which is less then my core 2 duo now.

 

For my use, NFS mount on a ram drive will be very fast and make it highly usable.

(of course you have to be studious about backups.)

 

For a cache drive on ram (internal or external) there would need to be a MOVE NOW button to trigger a move manually when you are ready.

 

I bet solid state drives will be cheap enough by the time ram drive on unRAID is really feasible/worthwhile to the normal end user.

 

 

wouldnt it be better to wait for SSD's to become cheaper?, that would be much easier and will be good enough for a cache drive

at the moment you can get a 16 gig SSD for about 280 euro, but 8 gig of ram + a ramdrive isnt cheap ither

i suspect that the 8 and 16 gig SSD drives will become cheap enough this year

 

so i bought a 320gig 1 platter samsung for now (50 euro) and planned to switch to SSD later

 

I still think its a good idea. I am prepared to bet that many users have over 1GB of ram thats never used already in their boxes and likely even more have loads kicking about they have never added as it didn't need it.

 

The whole idea of a cache drive is to get stuff onto the box quick and then unraid moves stuff over for you. There is no reason a ram drive couldn't be perfect for this.

 

The caveat is that there is a finite risk of data loss between the point the data is in ram and it is copied onto unraid physical drives. This risk is higher than using a real physical cache drive that is for sure but is not unacceptable.

 

This is a user choice but what we are talking here is alot of bang for zero bucks. SSD et al are alot of bang for alot of bucks.

 

Also im not talking about 8GB of ram or eve 16GB ramdrives here. Im talking 0.5GB to 1GB... realistic amounts people would have spare. Once you start going up in size this solution would not be viable.

 

But... none of this doesnt mean its not a good idea for many users.

 

 

There is no advantage to a software RAM drive.  If you have 4GB of RAM, linux will use all that RAM as buffer cache anyway.  There is no advantage to partitioning the RAM and creating, e.g, a 3GB RAM drive - all that will happen is that file data will get remapped from buffer cache to RAM drive.

 

If you're talking about something like a SSD, then sure that's a different story.  But remember, the system bottleneck is the network - it doesn't do you much good to have a SSD that r/w at 300MB/sec if you can only get data into/out of the system via GigE.

 

The main use of the 'cache drive' is to offer increased write-performance, especially to decrease the time it takes to load up a new unRAID server with existing data from other systems.  Another future use is to implement a "smart" allocator: when it comes time to move data off the cache drive to the array, then we can make some decisions about how to "pack" the data efficiently on the array.

 

In a future release we will enable PAE which will let you install up to 64GB of RAM - but this will require testing and/or bug reports from WeeboTech who apparently has already enabled it 

NASuser, I don't agree with you on this one,  For the average user a small ram drive as a cache does not afford much.

Where it does afford allot is if you have a remote mount from other systems with "frequent" writes/appends.

 

or "frequent" random reads of small files and such.

 

This is my case exactly. 

Let me explain (this will be a lil long winded).

 

NFS reads chunks of files via UDP and it does pretty well. It also writes chunks of files via UDP, and it's the writes that have a performance penalty. Normal NFS clients wait until the block is acknowledged by the server as being written to the disk.

I.E. being stored in the buffer cache is usually not good enough. This slows down writes. (sort of like flash where writes are verified to be successful).  For the reads, having the root file system of multiple machnes (and home directories) in ram disk can prove to save a few spindles.

 

In addition, with frequent writes/appends, in order to minimize head movement, writing to a ram drive saves wear and tear on the drive mechanics. Granted I have never had too much of an issue, but in my case, the to goal is to minimize spindles.

In my network I have a centralized log host. All syslog messages go to one server. This server receives the messages via UDP, writes them to the log, and also updates a mysql database.  So every few seconds a write is occurring.

 

My goal with the ram drive is to have the data written to a ramdisk (in memory or gigabyte ramdrive) so all the frequent writes to mechanics are minimized.

 

So for the average joe, the ram drive does lilttle to assist.

Linux already has the buffer cache mechanism which caches all writes, then flushes them out to disk periodically.

In a future release we will enable PAE which will let you install up to 64GB of RAM - but this will require testing and/or bug reports from WeeboTech who apparently has already enabled it 

 

so far PAE has not given me any trouble, or a noticeable performance penalty.

I'll have to rip some dvd's to see.

 

In the meantime I did a top, made a 5GB file, then did a find down the tree 500,000 files without a drive spin up.

in only a few seconds too.

Then re indexed the locate cache in a few seconds also.

Then deleted the large file.

 

I just have to get another matching 4gb to see how well everything works out.

 

 

root@Media:~# sysctl -p
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_no_metrics_save = 1
net.core.netdev_max_backlog = 2500
vm.dirty_expire_centisecs = 100
vm.dirty_writeback_centisecs = 50
vm.dirty_ratio = 10
vm.dirty_background_ratio = 5
vm.vfs_cache_pressure = 10

top - 18:16:15 up 3 days, 21:48,  2 users,  load average: 0.00, 0.00, 0.00
Tasks:  74 total,   1 running,  73 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   6228380k total,   656860k used,  5571520k free,   123088k buffers
Swap:        0k total,        0k used,        0k free,    96848k cached

root@Media:~# dd if=/dev/zero of=/junk count=10240000 
10240000+0 records in
10240000+0 records out
5242880000 bytes (5.2 GB) copied, 17.4145 s, 301 MB/s
root@Media:~# ls -l --si /junk
-rw-r--r-- 1 root root 5.3G Apr 28 18:17 /junk

top - 18:17:29 up 3 days, 21:49,  2 users,  load average: 0.20, 0.06, 0.02
Tasks:  74 total,   1 running,  73 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.8%sy,  0.0%ni, 99.1%id,  0.1%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   6228380k total,  5782648k used,   445732k free,   123088k buffers
Swap:        0k total,        0k used,        0k free,  5216848k cached

root@Media:~# time find /mnt -type f | wc -l 
502,538

real    0m3.138s
user    0m0.630s
sys     0m0.610s

top - 18:18:21 up 3 days, 21:50,  2 users,  load average: 0.21, 0.08, 0.02
Tasks:  74 total,   1 running,  73 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.8%sy,  0.0%ni, 99.1%id,  0.1%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   6228380k total,  5783268k used,   445112k free,   123088k buffers
Swap:        0k total,        0k used,        0k free,  5216848k cached

root@Media:~# time /etc/cron.daily/slocate 

real    0m2.647s
user    0m0.490s
sys     0m0.390s

root@Media:~# rm /junk 

top - 18:19:13 up 3 days, 21:51,  2 users,  load average: 0.16, 0.08, 0.02
Tasks:  74 total,   1 running,  73 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.8%sy,  0.0%ni, 99.1%id,  0.1%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   6228380k total,   657356k used,  5571024k free,   123088k buffers
Swap:        0k total,        0k used,        0k free,    96848k cached

 

Gents excellent posts.

 

It seems I am fundamentally mistaken to how the cache drive gives an advantage perhaps its worth hammering this out.

 

As i understood it the increased write speed from the cache drive was due to the fact that the data write does not instigate an immediate parity write allowing you to get the data onto the box quickly and then unRAID moves it to the array unrelated at a later time. But Tom said above that the bottle neck is the network. If this is the case how on earth does a cache drive speed up the network (obviously it doesnt so what am i missing)?

 

Gents excellent posts.

 

It seems I am fundamentally mistaken to how the cache drive gives an advantage perhaps its worth hammering this out.

 

As i understood it the increased write speed from the cache drive was due to the fact that the data write does not instigate an immediate parity write allowing you to get the data onto the box quickly and then unRAID moves it to the array unrelated at a later time. But Tom said above that the bottle neck is the network. If this is the case how on earth does a cache drive speed up the network (obviously it doesnt so what am i missing)?

 

 

Because when writing to the array the network is not the bottleneck, read/modify/write required to update parity is.  This rate peaks at around 15 MB/sec sustained (for typical drives).

That makes sense. So I simply dont get why a RAMdrive wouldnt offer any advantages then?

That makes sense. So I simply dont get why a RAMdrive wouldnt offer any advantages then?

 

A ram drive would not offer any advantage of speed because writes to a regular "non parity protected" hard drive and/or a ram drive would occur at approx. the same speed because the network is the real bottleneck.

 

In addition, before the write actually occurs to the hard drive, the data is actually written to ram in the buffer cache. So it is actually written internally to ram, just not a ram drive.. after a predetermined age the data is flsuhed out to disk.

 

Pretty much what the cache drive does for unRAID physically, but at a higher and faster level in ram.

 

Therefore, unless you had gobs of ram to throw at a ram drive, it really would not provide much of a benefit vs a temporary unprotected single spindle multi gigabyte cache drive.  In fact the single unprotected spindle cache drive provides way more bang for the buck then a perceived benefit from using a ramdrive as a cache drive.

 

Consider cost of a single multi gigabyte spindle, vs the cost of 4GB (or 8GB) of ram.

Then consider the cost of a motherboard that can possibly support more ram, the few I've looked at are in the $300 range.

 

Then consider how large some DVD rips will be, you'll see how quickly you could run out of space on that ram drive.