May 19, 201313 yr ... by the way, I'm running a "correcting" parity check (always do) => i.e. the box that says "Correct any Parity-Check errors ..." is checked. I don't think this makes any difference in the timing of the check, but just to be sure, you should run yours the same way so it's an exact "apples-apples" comparison
May 19, 201313 yr A little curiosity I noticed about power supplies... The efficiency also varies depending on the supply voltage used with the power supply, at least on the ones I have looked into in detail. It seems most, if not all multi-voltage input switching power supplies are also more efficient at 240 VAC input voltage than they are at the US standard of 120 VAC! Since in the USA we also have 240 VAC available for appliances in most homes and buildings, I was thinking of running separate 240 VAC branch circuits to my computers next... (after all I already have done that for my imported game consoles...) Very interesting. I had noticed the same, but it never occurred to me to go that route. What about the other components, like monitors/TV's? Since I make a living with computers and monitors (and lot's of them), and I work from my home office, this might be worth pursuing for me. I wouldn't retrofit my current home, but I'm planning to have a new home built in a couple years, and I could easily have 240V outlets installed for cheap in a few key areas. Would I (or should I) have European 240V outlets installed? Or would I have US 240V outlets? Which is easier to plug into using the components we're talking about? Would it even pass inspection if I had European outlets installed? Hmmm... I have not looked into monitors or TVs, but I would 'guess' it may be the same situation, as long as you have a multi-voltage capable switching power supply in them. I just a bit of research reading various electrical journal articles and power supply data sheets, and looking at efficiency charts.. It seems that typically you should be able to expect between 2-3.5% efficiency gains in using 240 VAC instead of 120 VAC with a power supply that is capable of operating on both voltages. There are no guarantees here though, and it really depends on the design of the power supply. Look VERY closely at the labels, and verify anything you wish to use on 240 VAC will properly support and operate on the higher voltage. Cautions however... If you are using a switching power supply, with a removable power cord, precautions need to be taken so your new cool 240 VOLT power cord does NOT get plugged into a device that is only able to operate on 120 VAC, or that has a switch that is set for 120 VAC operation! If double the voltage is applied to a device either not rated for it, or not properly configured for it BAD things could of occur. I have painted the end of my 240 power cords RED on the end that connects to my imported video game consoles. Another thing that should probably be done, is also tie the cord somehow to the unit so if it is unplugged, it does not become available to plug into something else, when reaching behind things and plugging in "BLIND" fashion by feel only. In the USA, I would just stick to the normal 15 or 20 amp 220 volt outlets and plugs that are standard here. Then there would be no problems, and only a quick answer would be needed if asked about by an inspector. The parts are also easily sourced that way. I have thought a few times about sourcing European and Asian power outlets, but they are a bit pricey, after shipping, and certainly would raise questions during an inspection, but might be approved, depending more on the inspector than actual code... The connectors to look for in the USA, would be probably most commonly used with larger 240 VAC window AC units. Look for NEMA 6-15 for 15 AMP, and 6-20 for 20 AMP 240 VAC use. Power cords are available with the proper ends for such applications that will plug into the standard IEC 14C (I think that is correct...) power connection on standard computer power supplies, and other items. I found it easier however to just buy plugs and cut the old 120 VAC plug off and replace them with the 240 VAC NEMA 6-15 plug.
May 19, 201313 yr While the slight efficiency benefits of using 220v equipment is intriguing, I don't think it's worth the cost of conversion. For one thing, all it takes is ONE mistake (i.e. plugging a 120v device into a 220v outlet) and you've likely done more damage in 1 second than you'll save in a couple years. I once told a tech rep who was delivering a system in Brussels (from the US) that he could NOT plug the monitor into the outlet on the back of the PC's tower due to the voltage difference => but he was insistent that since he'd switched the PC's PSU to 220v it would be fine. I tried to make him understand that the power outlet on the tower was just wired to the source -- that the 110/220 switch (this was 20 years ago, before virtually all PSUs were auto-volt) just switched the PC's internal PSU -- but he wouldn't listen. He plugged the monitor in; and needless to say, when he turned it on there was a loud "pop" as it destroyed itself If you DO decide to use 220v wiring and equipment, be SURE that you use DIFFERENT plugs so there's no chance of your maid plugging your vacuum cleaner into a 220v outlet !!
May 19, 201313 yr Parity check completed. I noted the speed at various points throughout -- in particular at the 67% mark, right after a 2TB drive would have been no longer part of the check -- so from this point on your system should have very similar results. At the 67% point the speed was 106.41MB/sec Here's the complete list of points I noted the speed: 48% 120.61MB/sec 58% 113.81MB/sec 63% 110.03MB/sec 67% 106.41MB/sec (5:08) 74% 98.92MB/sec 80% 94.14MB 86% 86.97MB/sec 88% 83.6MB/sec 92% 80.27MB/sec 95% 74.68MB/sec 99% 69.67MB/sec The slowdown as it gets to the inner cylinders is very apparent. I didn't note the speed at first, but it starts in the 130MB/s range ... and drops down to about 55% of that by the time it's on the innermost cylinders. This is typical of ALL modern drives -- and the fact your system has to do this for 3 different size drives certainly makes a BIG difference in the total parity check time.
May 19, 201313 yr Author Parity check completed. I noted the speed at various points throughout -- in particular at the 67% mark, right after a 2TB drive would have been no longer part of the check -- so from this point on your system should have very similar results. At the 67% point the speed was 106.41MB/sec Here's the complete list of points I noted the speed: 48% 120.61MB/sec 58% 113.81MB/sec 63% 110.03MB/sec 67% 106.41MB/sec (5:08) 74% 98.92MB/sec 80% 94.14MB 85% 84MB/s 86% 86.97MB/sec 82MB/s 88% 83.6MB/sec 89% 79MB/s 92% 80.27MB/sec 95% 74.68MB/sec 97% 71MB/s 99% 69.67MB/sec 69MB/s 99.1% 66MB/s 99.9% 65MB/s The slowdown as it gets to the inner cylinders is very apparent. I didn't note the speed at first, but it starts in the 130MB/s range ... and drops down to about 55% of that by the time it's on the innermost cylinders. This is typical of ALL modern drives -- and the fact your system has to do this for 3 different size drives certainly makes a BIG difference in the total parity check time. Great info, thank you garycase! I started my parity check at 10:19am (after movie night ) before going to bed, and tried to wake up early enough to catch it around 67%. I missed it, and my first observation was at 85%. It just finished a few minutes ago at 9:36am. 11 hours 17.5 minutes (40,651 seconds). 72,083.5 KB/s. I didn't see your email before I started, and I ran a NOCORRECT parity check. But I had 0 sync errors, so in all likelihood the timing was the same. I put my check points in blue with your data. At the end, it looks like we finished with the same speed. At the 85% mark, it looks like you are about 5% faster, and I close the gap as we approach the end. I'll have to get some more data points to verify. It's certainly possible that I have a bottleneck above 75MB/s, but without seeing my 67% score it's too early to say for sure. Going from memory, I never broke 100MB/s at the 67% mark, but I may be wrong about that. I've been upgrading my drives, and I just removed my last 1TB drive. This was my first parity run without the 1TB, and it does feel like it finished a little quicker. I'm working on replacing my six 1.5TB drives now, and hope to have this done in a couple weeks. This will give me an excellent before and after snapshot of the impact of the multiple drive size slowdown. I'm upgrading a 1.5TB today, so I won't have another parity check until probably Tuesday.
May 19, 201313 yr Author I have not looked into monitors or TVs, but I would 'guess' it may be the same situation, as long as you have a multi-voltage capable switching power supply in them. I just a bit of research reading various electrical journal articles and power supply data sheets, and looking at efficiency charts.. It seems that typically you should be able to expect between 2-3.5% efficiency gains in using 240 VAC instead of 120 VAC with a power supply that is capable of operating on both voltages. There are no guarantees here though, and it really depends on the design of the power supply. Look VERY closely at the labels, and verify anything you wish to use on 240 VAC will properly support and operate on the higher voltage. Cautions however... If you are using a switching power supply, with a removable power cord, precautions need to be taken so your new cool 240 VOLT power cord does NOT get plugged into a device that is only able to operate on 120 VAC, or that has a switch that is set for 120 VAC operation! If double the voltage is applied to a device either not rated for it, or not properly configured for it BAD things could of occur. I have painted the end of my 240 power cords RED on the end that connects to my imported video game consoles. Another thing that should probably be done, is also tie the cord somehow to the unit so if it is unplugged, it does not become available to plug into something else, when reaching behind things and plugging in "BLIND" fashion by feel only. In the USA, I would just stick to the normal 15 or 20 amp 220 volt outlets and plugs that are standard here. Then there would be no problems, and only a quick answer would be needed if asked about by an inspector. The parts are also easily sourced that way. I have thought a few times about sourcing European and Asian power outlets, but they are a bit pricey, after shipping, and certainly would raise questions during an inspection, but might be approved, depending more on the inspector than actual code... The connectors to look for in the USA, would be probably most commonly used with larger 240 VAC window AC units. Look for NEMA 6-15 for 15 AMP, and 6-20 for 20 AMP 240 VAC use. Power cords are available with the proper ends for such applications that will plug into the standard IEC 14C (I think that is correct...) power connection on standard computer power supplies, and other items. I found it easier however to just buy plugs and cut the old 120 VAC plug off and replace them with the 240 VAC NEMA 6-15 plug. Fantastic info! Thank you for sharing electron286! Great tips on the plugs and safety precautions. I prefer to run all of my computers off of pure-sinewave UPS (everything runs better and lasts longer; a UPS even solved a recurring server crash I was experiencing with unRAID), and these devices typically have a hard-wired plug and do not accept multiple voltages. I would either need to source a US-spec 240V UPS (I assume these exist for datacenters, but are probably mucho $$$), or import a European-spec 240V UPS, which would have all European 240V outlets. UPS's are heavy, and overseas shipping would probably be prohibitively expensive. I'll have to do some more research to see what options are available. And as garycase accurately pointed out, the risks alone may outweigh the rewards.
May 19, 201313 yr The % numbers will vary a bit, so I'd say your numbers are very comparable. But I'd definitely run another check at a time when you'll be able to catch the 67-68% number -- in fact you'll want to see both the "just before" and "right at" numbers (i.e. 66% and right after it passes the 2TB point). r.e. 220v UPS units => these are readily available in the US from the major UPS manufacturers, but as you've guessed they're notably more expensive ==> they don't make any inexpensive "consumer-oriented" versions of them, since consumers aren't buying 220v equipment
May 19, 201313 yr I was also thinking as I wrote, that 2 - 3.5% efficiency by itself, is probably not worth a move from 120 VAC to 240 VAC operation. In a large data center there are many other advantages too. The most obvious is the size of scale and multiplied savings. If you only saved 1% of a $10,000 electric bill per month, it would be very nice! BUT, you may get some additional advantages also like are seen in data centers, depending on how much equipment you are running... Other factors to consider... IN a data center environment, you do not have random people, unfamiliar with, or unaware of the 'commercial' standard wiring of 240 VAC to the server equipment. So the chances of someone accidentally hooking up the wrong voltage to a 120 VAC piece of equipment is less likely. There are a number of server class power supplies, from multiple vendors, that actually are rated at HIGHER wattage output when used with 240 VAC rather than 120 VAC power. (This is due to the higher DC bus voltages attained in those designs, and the way it is frequency/time duration converted to the lower output DC voltages.) Running a UPS with 240 VAC, yes very common in data centers, also introduces another level savings! Many, UPSes also see about a 1.5-2.5% increase in overall efficiency compared to a similar 120 VAC speced model, (or same with lower 120 VAC voltage order specification.) Twice as much equipment can be run on the SAME size of wire! With a standard 120 VAC 20 AMP circuit, provided by a single run of 12 AWG sized wire, you can provide 1440 WATTS of power. If you have a 240 VAC 20 AMP circuit, it would also use the same single run of 12 AWG sized wire, and would be able to provide 2880 WATTS of power! In a residential service, this would be a change in the breakers from a single pole, to a little more costly double pole breaker. If sourcing higher quality spec grade outlets, the price is about the same for NEMA 5-20 (120 VAC) and 6-20 (240 VAC) Depending on the power design in a data center, they could also save a little more by also using a single pole breaker for the 240 VAC power, not a lot of savings, but on a large scale project, it would add up to initial installation savings. So, if you have enough computer equipment running, to make it more close to a data center wiring model, your savings may very well be worth it, both in initial installation savings to provide power to a large number of computer resources, (copper wire savings - reduced circuit count - not to mention reduced heat loss in wiring if running near ratings...) Potential efficiency savings on a properly specified 240 VAC UPS, plus additional efficiency savings on the computer power supplies. For me, I run a individual branch circuit to a single dedicated outlet, in EACH room, where I expect i might EVER want to have a computer, I have just run 120 VAC circuits however so far, since I want it to be east for anyone that may use it in the future. I also run all of the computer circuits from a dedicated power sub-panel. That way I can easily connect to a large centralized UPS and/or generator set-up should I ever decide to... (after all, who needs lights in a power outage, when you can have a computer!) In an area I expect to be data-centric, I run multiple circuits, and try to plan for future expansion possibilities too, (such as with a finished basement planned, run over-sized conduit to such expected areas, even though not usually used in residential, to allow for additional added circuits in the future...) This usually means empty conduit for now, since I already ran Romex and thought about future needs later... It is also nice to have smaller 1/2" or 3/4" conduits (PVC is fine) run to home run termination points for network wiring, etc. This way you can upgrade easily from old RG-62 (ARCNET), to RG-58 (ETHERNET 10Base-2), to CAT3, ETHERNET 10Base-T, to CAT5e/CAT6 - ETHERNET 100Base-T/1000Base-T, to fibre?... (no not yet...) While still related to power efficiencies this should probably move to a new topic if more comments are needed... ;-)
May 19, 201313 yr Pauven and garycase... When periodically checking your parity check speed, where are you looking? On the main unRAID console web page, or a plug-in like unMENU? Are you just manually re-freshing the page when you want to look at it? The reason I am asking is that most of my unRAID configurations are running on MUCH less capable hardware, so it is noticable that when I refresh a web page numerous times, that the parity check slows down and expected time to completion is extended... to be expected with slower, older hardware, that is maxed out with a parity check... So with a nice new, not pushed to the limits, system running parity checks, do those numbers even change with multiple web page refreshes? I was thinking while watching your test results... it might be really nice to write a script to capture the data at fixed intervals, such as 15 minutes, or 30 minutes, and place them in a CSV file to later make charts, and be available for further analysis steps... perhaps overkill, but it would make it very nice for a repeatable test between hardware tests and changes... :-) Maybe a new thread there too! :-)
May 19, 201313 yr Author Pauven and garycase... When periodically checking your parity check speed, where are you looking? On the main unRAID console web page, or a plug-in like unMENU? Are you just manually re-freshing the page when you want to look at it? The reason I am asking is that most of my unRAID configurations are running on MUCH less capable hardware, so it is noticable that when I refresh a web page numerous times, that the parity check slows down and expected time to completion is extended... to be expected with slower, older hardware, that is maxed out with a parity check... So with a nice new, not pushed to the limits, system running parity checks, do those numbers even change with multiple web page refreshes? I was thinking while watching your test results... it might be really nice to write a script to capture the data at fixed intervals, such as 15 minutes, or 30 minutes, and place them in a CSV file to later make charts, and be available for further analysis steps... perhaps overkill, but it would make it very nice for a repeatable test between hardware tests and changes... :-) Maybe a new thread there too! :-) I check using unMenu on a page that does NOT show temperature or any other SMART data. Polling the SMART data causes a temporary interruption in data processing, which leads to the parity checks slowing down. I like the unMenu > myMain > Detail view. It autorefreshes. It shows current position % in tenths (i.e. 18.1%). And it doesn't have any SMART data. You'll also notice that there is only one manual 'Refresh' button - on myMain pages that poll SMART data, there are two buttons, one that polls SMART data and one that does not. Use that as a clue to avoid those pages, as the autorefresh feature will always poll the SMART data. I only visit those pages once in a while to get a temp read to make sure nothing's amiss. The 'Speed' value naturally fluctuates from update to update, even if you don't poll the SMART data. Right now I'm running a rebuild, and it swings from a low of about 79 MB/s to a high of about 85MB/s, normally hovering somewhere in the middle. I think the reason it swings so much is that the calculation only measures a very short timeframe (perhaps even instantaneous). It would be nice if they had an 'Average Speed' that is the current average from the beginning. Additionally, a rolling '10 Minute Average' would give you a nice accurate number for your current position, without all the wild fluctuations. I like your idea to log the speed data. I would be happy with 100 samples, spaced at every 1%. Not an overwhelming amount of data, but with enough accuracy to see the impact of smaller drives completing. Perhaps Joe L., or one of the other UNIX/LINUX genius', can figure out a way to do that. By the way, something odd I've been noticing during my rebuild (upgrading a 1.5TB to a 3TB Red): The rebuild started off averaging about 82MB/s, and now at the 20% mark it is still averaging 82MB/s. No slow down at all! Obviously the rebuild would never progress faster than the 3TB Red drive can be written to, but if that was the limiter I would have still expected a gradual speed decrease from 0% to 20%. It does not appear to be CPU constrained. I'm fairly certain this is evidence of a bandwidth bottleneck. Possibilities include the Adaptec 1430 cards, the PCIe bus, and system memory.
May 19, 201313 yr I've also noticed the slowdown when you do frequent polls during parity checks -- but I typically only check a few times during the check, so it doesn't have much overall impact. When I polled to get the numbers I posed, I waited for the initial reply; then waited ~ 15 seconds and refreshed the display. The 2nd response is virtually instant, and is the result I posted. I didn't realize, however, the UnMenu - MyMain - Details would provide this data ... so I'll use that from now on
May 19, 201313 yr Out of interested do you need that speed of the 2760A? When buying this card it has RAID functionality is not better to use that then Unraid? That's a good question, and still one I'm trying to answer. I'm not going for ultra high performance (of which the 2760A is certainly capable, no doubt). If I was using this for non-linear video editing or a high demand database, yes, I would want ultra high performance, and I would use the on-board RAID capability. Doing so would require all drives to spin up for every access, though, so this would also be a high power consumption server. Typical RAID redundancy only allows for the loss of 1 or 2 drives, any additional drive losses and you lose your entire array, so you better have some good backups! With unRAID, only the drive that contains the data you're reading spins up, plus parity if you're writing, so it is very energy efficient (except for parity checks/rebuilds). unRAID can handle a single drive failure with no data loss, but if you have additional drive failures you only lose the data on those drives. Unlike RAID, which stripes your data across all drives, unRAID keeps your data whole and intact on a single drive. So if I lost 2 drives on a 24 drive server, I've only lost 2 drives of data, and not all 24. This approach also minimizes capacity lost to parity. All of these features match my requirements perfectly. I would happily pay less for a 2760A without the RAID features. So, what is my performance goal, then? Well, I want an unRAID server that performs unthrottled. One of the most crucial tasks with an unRAID server is regular Parity Checks. With less than optimal throughput, Parity Check time increases. Ditto for data rebuilds in case of a failure or drive upgrade. An unRAID server will never, never be faster than a single drive or the network connection. But a less than optimal hardware build can lead to an unRAID server that performs much slower than a single drive. These problems only get more common the larger you scale up the unRAID build. At 24 drives, I'm trying to be careful not to throttle the performance with unwise hardware choices. In my own unscientific tests (more anecdotal than anything), I've noticed parity check processing time increase when I've provided less than 250MB/s per drive. I can't provide more than 250MB/s with my current build, so I don't know if I'm at peak performance or not, but my hunch is that I'm not. That's why I picked the 2760A, as it will give me 333MB/s per drive of bandwidth. Is it really needed? Is it enough? I have no idea... A good learning example is that garycase's parity check takes ~8 hours on a 3TB parity - he has all drives connected directly to the MB, so his configuration is optimal. I too have a 3TB parity (same Red 3TB drives), and my parity checks are closer to 12 hours. My drives are connected through 4 Adaptec 1430SA PCIe 1.0 x4 adapters, so each drive is bandwidth limited to 250MB/s, plus 5 more drives on the MB which are not bandwidth throttled. While I can't say for sure exactly where my performance is hindered (CPU, memory, bus bandwidth, too many components, a high drive count, slow HD's, inefficient drivers, etc.), I'm certainly giving up some performance somewhere. Parity Checks and rebuilds also take longer the larger your parity drive. I've watched my parity checks increase from a few hours with 1TB drives to 12+ hours with 3TB drives as my system has grown. What happens with 6TB drives? 24 hour parity checks? Yikes! Where does it end... Another consideration is heat. Drives are going to heat up when spun up - but my drives are spun up 50% longer than garycase's on a parity check. Assuming similar cooling, who's drives do you think will fail first? Drives are also continuing to get faster as platter densities increase. What may not be a bottleneck with today's 3TB drives may be a bottleneck with tomorrow's higher performing 6TB drives (oh, did I say tomorrow... I meant 2016's). So my performance goal is simply to have a great performing unRAID server, both today and 5 years from now, while minimizing drive wear and tear and energy consumption. Sorry for my long winded-ness. I probably didn't even answer your question... Thank you for your extended explanation. much, much appreciated. I never thought about the bandwidth of cards and the time of a parity check. Very interesting stuff! I'm at the moment not a unRAID user but this change in couple of days / weeks when i rebuild my server setup. My mail,web,plex,etc. will be located on a Mac Mini. I'll mount some shares from unRAID via NFS to the Mac Mini. I will test this for awhile and if i like this i build a new unRAID server.
May 19, 201313 yr I've also noticed the slowdown when you do frequent polls during parity checks -- but I typically only check a few times during the check, so it doesn't have much overall impact. When I polled to get the numbers I posed, I waited for the initial reply; then waited ~ 15 seconds and refreshed the display. The 2nd response is virtually instant, and is the result I posted. I didn't realize, however, the UnMenu - MyMain - Details would provide this data ... so I'll use that from now on How often do parity checks happen?
May 19, 201313 yr How often do parity checks happen? They only happen when you run one ... although there's a convenient script in UnMenu that will automatically run one once-a-month.
May 19, 201313 yr When I fisrt started using unRAID, I did not even think about (nor had read about...) doing periodic parity checks. 'out of the box' unRAID, as garycase said, does not initiate parity checks, they only occur when you ask one to run. There are some VERY good reasons to run them on a periodic basis, and monthly seems to be a good period. unMENU does make it very easy to set the monthly parity check. This will force a full read of all the surface of your drives! This gives the drives internal SMART monitoring circuits to check and re-allocate if needed, and areas on the disc that might be deteriorating. This process is invisible to the user, if done often enough to catch it before loss of data on a drive occurs. I had one netbook drive, as an example, that did not have very good media, and would not retain data at some locations for more than a few months or so. Since I never did a full surface read, by the time I read a failing area of the disk, data was lost and files corrupted. Even the restore partition had areas that had failed!@@@ "luckily" I am a bit paranoid, and had made multiple disc images of the hard drive when I bought the netbook, even before I loaded any software on it and started to use it... So after a few re-stores, I finally gave up and replaced the hard drive, with a larger one... :-) all is good now. I just need to find an easy way to check my drives on my windows computers, that works as easy as the monthly parity check I let my unRAID systems do on their own...
May 19, 201313 yr How often do parity checks happen? They only happen when you run one ... although there's a convenient script in UnMenu that will automatically run one once-a-month. Both unMENU and SimpleFeatures have the ability to set automatic parity checks on a schedule of your choosing. I do monthly parity checks on my server.
May 19, 201313 yr Author How often do parity checks happen? They only happen when you run one ... although there's a convenient script in UnMenu that will automatically run one once-a-month. Both unMENU and SimpleFeatures have the ability to set automatic parity checks on a schedule of your choosing. I do monthly parity checks on my server. I too observe the monthly schedule, and this is a good baseline. But there are events for which I always run additional parity checks: Just prior to upgrading a drive (I like to do a Correct parity check here) Just after upgrading a drive (I like to do a NoCorrect parity check here) After writing large amounts of data to the server in a short timeframe (might indicate a bad copy) After an 'Event', like a power failure, improper shutdown, loose cable, or hung server (extremely rare) After a parity check in which sync errors were found and corrected (resolve any issues first!) When I'm suspicious of a component... I typically don't see any sync errors, but if I do they are most often (in my experience) related to a drive failure. The thing to keep in mind is that a drive failure could either be the parity drive itself, or a data drive. If a parity drive is failing, simply replacing the drive and rebuilding parity is a quick fix. But if a data drive is failing, further attempts to sync parity only sync bad data, so that when you finally replace your bad data drive, it gets rebuilt from parity data that includes bad bits that were synced from your failing drive. As soon as you notice sync errors, determine where they are originating from and resolve asap. Since you're new to unRAID, you'll also want to look up PreClear (a fantastic script!). Before adding a new drive to unRAID, it is recommended to PreClear the drive, at least once, before adding it to the array. That way if a new drive bites the dust (all too common) you didn't expose your array to it. Some people run multiple PreClear passes on a drive, to really work it hard and try to make it fail. Survival of the fittest. Myself, I just run one pass. Also, if you have any critical data that you can't afford to lose, make a backup. One nice feature of unRAID is that you can make two copies of your critical data (manual effort), keeping it on two different drives. Assuming a 2-drive failure occurs (preventing recovery from parity data), the hope is that at least one copy of your critical data survived on a drive that didn't fail. Again, unRAID doesn't eliminate the need for backups, but with some smart practices you can minimize the occurrences of needing to restore from a backup. Hmmm, we seem to be getting off-topic again (and I'm not helping, but good conversation nonetheless). As an attempt to get back on-topic, I read that many of the ultra low power (i.e. 10W) computers people were building were running Windows because it implements better power saving features. Linux was called out for not really maximizing CPU/system power saving modes, even though Linux was noted for reaching an idle state more frequently (Windows always seems to be doing something...). Is this still the case? Are there certain parameters I can set, or modules I can install, to allow Linux to access the lowest power states? Are there certain power saving features that just aren't implemented in Linux?
May 19, 201313 yr I don't know the answer to Linux's power saving features, but I'm fairly certain it does. A quick "Google" of "Linux CPU power saving modes" results in quite a few discussions r.e. how to "tune" the power saving modes, etc. Whether or not UnRAID actually implements any of these I don't know -- perhaps one of the Linux gurus can chime in on this. But I do know my CPU temps go up a bit during parity checks; so that's at least anecdotal evidence that the CPU is throttling down when it doesn't need a lot of power.
May 19, 201313 yr I have wondered that for a long time myself... From what I remember, AMD power saving features were possible in the linux kernals before Intel features. But from what I have learned.. long past and possibly way outdated now... There was no automatic implementation of power saving modes in Linux, and it needed to be done at the application level. :-( I could be wrong, but that was my understanding then, and lots may be different now.
May 19, 201313 yr Author So I did a bit of poking around. I know on my current AMD based unRAID server, the PowerNow cpu scaling features are not enabled by default. I load the CPU Frequency Scaling Driving by running: modprobe powernow-k8 in my 'go' file which saves a lot of power. From what I can tell, the Intel SpeedStep functionality is enabled by: modprobe acpi-cpufreq Newer version of Linux have more power saving features built in, especially with kernel version 3.4 (which unRAID current RC is on). More info here: https://wiki.archlinux.org/index.php/CPU_Frequency_Scaling From what I've read, I think the default scaling governor settings are appropriate for an unRAID server. It looks like Intel has incorporated some driver rewrites into kernel 3.6, including Ivy Bridge support: http://www.phoronix.com/scan.php?page=news_item&px=MTE0NzY So it seems really easy to get the CPU power saving features enabled and optimized, but I might need the latest drivers from 3.6 for my new Ivy Bridge based Celeron. From additional research, Linux is reported to fall behind Windows on 3rd party driver power saving functionality. For example, in my new build the following devices all have drivers that may or may not enable power saving functionality in Linux: HD2500 graphics (though since this is integrated into the CPU, maybe it works with above) HighPoint RocketRAID 2760A (I really hope it works out of the box, otherwise it will consume 45W all the time) Realtek 8112B Gigabit LAN Not sure if I missed any other biggies. I plan to disable in the BIOS all features that won't be used for unRAID (i.e. Audio, SATA ports, COM ports, etc). I need to do some more research to determine if I need to load special drivers for any of the above devices to enable power saving features, and if there are any special settings I have to set for the drivers to actually enable power saving.
May 20, 201313 yr VERY COOL!!!! It does look like multiple CPU power down options DO work now in 5RC, and they ARE NOT in 4.5.3, which is what my main unRAID builds are on... also I just noticed the lsmod command, which lists which modules are loaded... Looks like another good reason to make the switch to 5
May 20, 201313 yr Author To help investigate whether parity checks/rebuilds can be bandwidth limited, I'm posting additional sample points from my Disk Rebuild. This is relevant to the original topic of the 24-bay server build, as the hypothesis is that controller cards with less bandwidth available per drive may throttle overall performance. The HighPoint 2760A drive controller has been selected for my new build because it has higher calculated bandwidth per drive, but test results and unRAID compatibility results are not yet available. The 'Speed Delta' column is how much faster garycase's parity check speed is compared to mine under similar conditions (WD Red 3TB drives only). garycase has a constant 6%-7% faster processing rate, which can probably be attributed to system architecture (different processors, memory speed, system chipsets, drive controllers, etc). My drives that are connected via Adaptec 1430SA controller cards are bandwidth limited to 250MB/s per drive (based upon the PCIe 1.0 spec), while none of garycase's drive are limited in this fashion. If my Parity Check performance was being throttled by the 250MB/s bandwidth of the PCIe bus, then the speed delta between garycase's and my systems should have shrunk as HD speeds slowed towards the end of the parity check. Looking at the data, I don't think the HD speeds are throttled at all at 250MB/s per drive bandwidth (which is a great result). This probably means that the higher 333MB/s bandwidth afforded by the HighPoint 2760A will not have a measurable increase in Parity Check speed (but at least it shouldn't be throttled). Conversely, per drive bandwidth at some point below 250MB/s is expected to slow down the Parity Check speed, but how far below 250MB/s throttling kicks in is yet to be determined. Outside of the consistent speed delta between our two systems, nothing is notable from 67% onward. % garycase Parity Pauven Parity Pauven Rebuild Speed Delta Notes 82MB/s Bandwidth Limited? 25% 82MB/s Bandwidth Limited? 33% 77MB/s 1.5TB HD Speed Limited 48% 120.6MB/s 50% 50MB/s 1.5TB HD Speed Limited 51% 70MB/s 1.5TB HD Done 58% 113.8MB/s 60% 60MB/s 63% 110.0MB/s 66% 47MB/s 2.0TB HD Speed Limited 67% 106.4MB/s 100MB/s 6% 2.0TB HD Done 74% 98.9MB/s 75% 93MB/s 6% 80% 94.1MB/s 88MB/s 7% 85% 84MB/s 85MB/s 86% 87.0MB/s 82MB/s 82MB/s 6% 88% 83.6MB/s 79MB/s 6% 89% 79MB/s 78MB/s 92% 80.3MB/s 75MB/s 7% 95% 74.7MB/s 70MB/s 7% 97% 71MB/s 67MB/s 99% 69.7MB/s 69MB/s 65MB/s 7% 99.9% 65MB/s 64MB/s 0% I'm using my Disk Rebuild numbers as they seem virtually identical to my Parity Check numbers: Pauven's Parity Check: 11 hours 17.5 minutes (40,651 seconds). 72,083.5 KB/s. Pauven's Disk Rebuild: 11 hours 16.5 minutes (40,602 seconds). 72,170.5 KB/s. One pattern that did emerge is that my rebuild speed was constrained to 82MB/s from 0% to approximately 29%-30%, after which point the processing speed gradually dropped as is expected behavior as the hard drive heads move across the platter from the outside edge toward the inside edge. CPU utilization was checked below 20%, and was found to be below 100% usage, so this does not appear to be a CPU speed constraint. This unexplained constraint did not re-emerge at the 67% mark, when processing speeds hit 100MB/s. Looking for patterns in the data: Below 50%, the server had 17 drives active: 3 1430SA's with 4 drives active each, and 5 MB SATA ports also active. Above 67%, the server had 8 drives active: 1 1430SA with 4 drives active, 1 1430SA with 2 drives active, 1 1430 with 1 drive active, and only 1 MB SATA port active (parity drive). All 3 1430SA were verified as connected via PCIe 1.0 x4. Perhaps there is a system bus bandwidth limit, or memory bandwidth limit, in my particular system that is causing processing speed to be constrained to 82MB/s when all drives are active. From 0% to ~30%, with 17 drives active at 82MB/s, the system was processing 1,392MB/s. System Constraint? At 51%, with 12 drives active at 70MB/s, the system was processing 840MB/s. At 67%, with only 8 Red 3TB drives active at 100MB/s, the system was processing 800MB/s. From what I can determine, the AMD CPU's HyperTransport link in my particular system should be providing about 2000MB/s unidirectional to the PCIe bus, which is suspiciously close to the ~1400MB/s speed limit observed. The HyperTransport bandwidth in my particular system is probably the bottleneck.
May 20, 201313 yr I don't think the HD speeds are throttled at all at 250MB/s per drive bandwidth (which is a great result). I'm not surprised. No modern rotating platter drive can provide data at that rate. If you were using SSDs, it'd be a different story Minor differences may be due to the occasional interface-rate transfers to/from the disks buffers; but I doubt that's even a factor in parity checks, where the data's being processed as fast as the disk can provide it -- thus the sustained transfer rate of the drives is the key limiting factor.
May 20, 201313 yr I got a not-particularly-usefull reply r.e. the PicoPSU. r.e. efficiency: "... Unfortunately we don't have the efficiency curve for the mentioned AC/DC adapter." They DID note that it has a no-load power consumption of less than 0.5W r.e. spinning up 6 drives: "... We haven't tested the picoPSU-160-XT with 6 hard drives. It should be able to hold the spin-up but we don't guarantee that. " So ... it sounds like an intriguing possibility ==> not sure I'll spring for one just to try it -- but if I'm bored some week I just may If anyone else happens to try it, please post the results !!
Archived
This topic is now archived and is closed to further replies.