May 3, 200917 yr Cool... does this mean I can tell my wife that I am spending this money for a MAID instead of a server?
May 16, 200917 yr Came across this interesting writeup on MAID. http://www.veridictusassociates.com/images/MAID_Jan_WP.pdf By this definition, unRAID is not really MAID. They are talking about specialized (data center) devices that hold hundreds or thousands of drives. Interesting that most will NOT allow all drives to spin at once, to reduce cooling and power requirements.
June 25, 200917 yr This is an interesting concept. I wonder if certain aspects of it could be implemented into unRAID in the future - specifically limiting the number of drives that can be spun up at any given time, as well as fully powering off drives as opposed to spinning them down. I doubt there is much need for this, since all of us are currently limited to 21 drives anyway, and the power savings will likely be negligible at that level. Even if these ideas were implemented into unRAID, it would still not be a MAID system, just somewhat more power-efficient. I just thought of this, tell me what you think. Its inspired by MAID, though it itself is not MAID proper. Introduce into unRAID a user-definable limit to the number of drives that can be active (powered on and spun up) at any given time. Say I have 15 data drives, one parity drive, and one cache drive and I set the server to only allow 3 drives to be active at a time. Perhaps I've been reading and writing a bunch of files over the past hour and 3 data drives are active (though none are currently being read/written to). If at this point I start another transfer to a 4th drive, the system could force one of the 3 active drives to spin down and power off immediately, thereby keeping the power consumption relatively constant. I imagine performance could be increased if the parity and cache drives were exempt from this 'active drive limit', so the total active drives could be 5 for a write operation (parity, cache, 3 data) or 3 for a read operation (3 data). Through this system there is obvious potential for power savings, though that would have to be factored against the potential for reducing the life of drives due to the extra spin ups and spin downs. If it causes your drives to fail faster, it would probably be more expensive in the long run. I had another idea while reading the MAID article in the post above - again, feedback would be appreciated. I wonder if it is possible to design a storage system that is a bit more intelligent than a simple data store, and could actually 'learn' which data is more important, higher priority, and/or more frequently accessed by the system's user(s) without specific input from said user(s). If this were possible, then the storage system could dynamically re-organize itself to increase read performance and possibly the safety and integrity of this important or often-used data. The system could move the most often accessed data to the most quickly read sectors on the fastest disks, and replicated the most important data across several disks to add another layer of protection against drive failure. All this would be done behind the scenes. In a sense, the server would 'learn' its user's preferences, and adapt itself to serve that user better.
June 25, 200917 yr Author Introduce into unRAID a user-definable limit to the number of drives that can be active (powered on and spun up) at any given time. I'm not keen on this idea. How would you handle the fail condition for the X'th process that wants to read from a disk which is now specified as staying powered off. I would much rather invest the time into a smarter user filesystem that caches the directories internally and only spins up a drive when it really needs it. I know we have the cache_dirs option, yet if you have an active system. the direntry cache can be flushed.. If the user file system were intelligent enough to keep information about directories read, then you would not spin up unless you really needed to. I wonder if it is possible to design a storage system that is a bit more intelligent than a simple data store, and could actually 'learn' which data is more important, higher priority, and/or more frequently accessed by the system's user(s) without specific input from said user(s). If this were possible, then the storage system could dynamically re-organize itself to increase read performance and possibly the safety and integrity of this important or often-used data. The system could move the most often accessed data to the most quickly read sectors on the fastest disks, and replicated the most important data across several disks to add another layer of protection against drive failure. All this would be done behind the scenes. In a sense, the server would 'learn' its user's preferences, and adapt itself to serve that user better. Although cool, I don''t see this as a possibility directly in unRAID. Perhaps if the user share filesystem kept statistics, then a program could move the files to the fastest disks and/or save them in a duplicate space. This is quite feasible and even by the community. (if we had statistics) As far as moving files to the outermost tracks.. Now we're talking about an intelligent program that knows the filesystem internals intimately. This is where it gets tricky. Now unRAID has to monitor very specific details on any filesystem changes.
June 25, 200917 yr Introduce into unRAID a user-definable limit to the number of drives that can be active (powered on and spun up) at any given time. I'm not keen on this idea. How would you handle the fail condition for the X'th process that wants to read from a disk which is now specified as staying powered off. Of course you are right, while replacing a failed drive the entire array must be on and accessible. So certain operations would need to be able to override the active drive limit. It is intended for everyday use. I would much rather invest the time into a smarter user filesystem that caches the directories internally and only spins up a drive when it really needs it. I know we have the cache_dirs option, yet if you have an active system. the direntry cache can be flushed.. If the user file system were intelligent enough to keep information about directories read, then you would not spin up unless you really needed to. That sounds good too. Perhaps if the user share filesystem kept statistics, then a program could move the files to the fastest disks and/or save them in a duplicate space. This is quite feasible and even by the community. (if we had statistics) So the limiting factor is currently the file system and not unRAID? As far as moving files to the outermost tracks.. Now we're talking about an intelligent program that knows the filesystem internals intimately. This is where it gets tricky. Now unRAID has to monitor very specific details on any filesystem changes. So it may be possible, but not feasible as it would have to keep track of too much and possibly be too slow?
June 26, 200917 yr ... as well as fully powering off drives as opposed to spinning them down Just a quick comment, if the drive is fully powered off, you can't talk to it, so it can't be spun up or used again, until the whole system is powered up. There is no current capability, in desktop systems, to control the power lines by the controller card, in order to re-start a drive. If you want a drive to spin back up, you have to keep it on standby with its little processor powered, so it needs to keep burning a watt or 2. As to an arbitrary number of active disks allowed to spin up, I can't see how that is better than the normal power saving unRAID policy of: if it is not active, spin it down. Trying to limit the current number of drives does not save you anything, because if the drives are needed, you will pay the same cost now, or later, when they are allowed to run. Plus as WeeboTech said, you have to deal with how you fail the extra drive request, such as, how you tell the 4th kid he can't watch a movie until one of the other 3 finish! That makes the kid sad, and unRAID look bad.
June 26, 200917 yr Author Perhaps if the user share filesystem kept statistics, then a program could move the files to the fastest disks and/or save them in a duplicate space. This is quite feasible and even by the community. (if we had statistics) So the limiting factor is currently the file system and not unRAID? There is no limiting factor here. unRAID itself or the third party programs do not have the intelligence yet to monitor traffic statistics on a regular basis and move files to disks accordingly. If the user share filesystem were enhanced to keep statistics, something could possibly be done on a third party basis. This would require statistics on the most used files (from user share) and the fastest drive (test program or kernel???). I don't see a kernel monitor to determine fastest drive so a test program will need to benchmark the drives and create a lookup table. Perhaps something could be written with the inotify tools to monitor all user share activity. I think the intelligence needs to be at the admin level. I knew my torrents and day to day files required top speed. 1.5GB parity for writes, Seagate 1TB 32MB cache for torrent and day to day, 5400 RPM WD for movie stores. As far as moving files to the outermost tracks.. Now we're talking about an intelligent program that knows the filesystem internals intimately. This is where it gets tricky. Now unRAID has to monitor very specific details on any filesystem changes. So it may be possible, but not feasible as it would have to keep track of too much and possibly be too slow? This requires a program with intelligence of a filesystem defragger. Then it requires intelligence on file usage. Frankly there is a benefit to larger amounts of ram if you have a busy system. Is there a bottle neck in lay out or design that cannot be handled by more intelligent file layout?
Archived
This topic is now archived and is closed to further replies.