unevent

A data point: Plex transcoding 3:10 Yuma full 1080 blu-ray to a single 3mb 720 stream and at 50% the transcode folder size is 1GB. I expect ~2GB final size for just this one stream at movie completion. Transcoder is set for speed over quality. edit: movie is at 1:33 out of 2:02: root@Tower:/tmp# du -ch 1.8G ./plex-transcode-386zepytk18-2c3445d1-ffcf-497e-9101-09b180aea3f8 0 ./mc-root 4.0K ./.vbox-root-ipc 32K ./hsperfdata_nobody 0 ./notifications/archive 0 ./notifications/unread 0 ./notifications 12K ./openvpn 4.0K ./web 0 ./.X11-unix 0 ./.ICE-unix 1.8G . 1.8G total root@Tower:/temp# free -l total used free shared buffers cached Mem: 8277284 8011088 266196 0 110356 7032472 Low: 836380 582744 253636 High: 7440904 7428344 12560 -/+ buffers/cache: 868260 7409024 Swap: 8008396 4 8008392

I saw this on YouTube but didn't see the page. I find it might be close enough. And yes, was just going to use the led output and use a solid state relay to pass off the signal / voltage needed to operate the eye. It would be off on. No in between. This LED version would be the best modern / power saving method to match but wondering about noise or a motor spinning the disk?? Also, the power contacts must wear out at some point? The eye will the the "piece de resistance" of this unRAIDIO. Just worried about meeting everyone's expectations of this thing. I would suggest removing the mechanical switches and contacts from the design. Depending on the final design power requirements, power can be transferred in simplest form by adding a coin cell to the rotating element, solar cell, a second hobby DC brush motor with a shaft on both ends to make a generator, slip ring, etc. Google POV displays or spinning clock for ideas.

Simulate it like what this guy did: http://www.hpfriedrichs.com/radioroom/magiceye/rr-magiceye.htm. What this guy does is a simple method, though can get more stable and more precise control with a microcontroller such as PIC, Arduino, etc. Curious as to how you want it to display disk access? Simply oscillate on access or variable based on drive access LED duty cycle?

Put it back together, have it restored and sell it to a collector.

I've always killed cache_dirs before startng a preclear because of the memory usage required during preclear.

Feature request. Would be nice if cache_dirs printed one-time file counts by directory into syslog on startup execution for all files cached by the script. Display the normal script startup information then give a file count by directory list that it is trying to cache based on startup parameters in syslog. No need to repeat the counts, just once at startup.

I understand the lowmem is static after boot, based upon the configuration you booted with. That's not what I was asking. What % of lowmem should I draw a line at for allocating memory to stripes? Obviously if I allocated 100%, we got problems. I could allocate up to 50%, and leave a nice buffer, or I could be more aggresive. I understand that there will be other processes competing for the same memory, so there's no hard and fast rule. I'm just looking for general guidance like "don't allocate more than 80% of total lowmem to stripes". This would primarily be to stop a FULLAUTO test at lower stripe values in case a particular server booted with significantly less lowmem (for whatever reason). Whatever you allocate leaves that much less for any addons. And since a swap partition or swap file is not default, being too aggressive will cause people problems once they re-enable their addons. One of those notes or disclaimers for the first or second post and let them decide.

That's an elegant solution to eliminate network performance as a factor. Having a very high performance cache drive, perhaps a modern fast ssd, would also help mitigate external influence. A RAM drive, with extremely low latency and high transfer speeds, would still be best. Any way to create a RAM drive and assign it as a cache drive (or simply mount it at all) in Linux? From http://lime-technology.com/forum/index.php?topic=25250.msg220405#msg220405: This method will provide some protection against the server running out of memory since you are defining a max size and it will never get larger.

Interesting, though do not believe that may be the best approach (for everyone). Thinking out loud, it would be logical for the driver to provide the 'quality of service' and not rely so much on 'thrashing the drive' where you start stacking IO waits because the interface is saturated. The driver works on a FIFO basis regardless of reads, writes, or parity, if I understand the driver correctly. At any point in time if a read request comes in it allocates a strip from the max stripe count, any writes, same, parity, same until the max stripe count = zero then it is listed (or queued) and sleeps until a stripe becomes available. No matter what you are doing there should be resources available (disk interface bandwidth) to perform the task. That I feel is the sweet spot where the driver is providing your 'quality of service' for read/writes/parity stripe throughput. Falls back to the testing methodology to find that sweet spot. Going through this effort to shave 5 minutes off a parity check does make one curious An idea for the script is to take 'hdparm -tT' readings from all drives in the array and taking inventory of the drive OEM and model numbers. Would also be helpful to have the script stop nfs/samba/afp and cache_dirs services during these tests to reduce the chance of the results being tainted and skewed. I feel it may be the cause of some of the 'weirdness' seen in reports.

I just looked in the md.c source code, and here are the commands I've found: set (md_trace, md_num_stripes, md_write_limit, md_sync_window, invalidateslot, resync_start, resync_end, rderror, wrerror, spinup_group, rdev_size) import start stop check (with CORRECT or NOCORRECT options) nocheck clear dump spindown spinup I don't see a parameter that looks like it kicks off a rebuild. I read through the source code, and I don't see a command line entry point for a rebuild. It seems you might only be able to do this through the GUI. It's possible that running a mdcmd check CORRECT, if exactly one disk is 'invalid', then it "might" start a rebuild, but that is purely a guess. From looking at the source code, I don't think it connects. It is initiated with '/root/mdcmd check CORRECT'. If the driver detects one invalid disk it begins rebuilding it, if all disks are valid it does a correcting check.

I thought it was rather clear I was commenting on leaving enough resources available for other services which require reads/writes to the array - a more balance approach. Regardless, no one fully understands these parameters yet so no point in debating testing methodology. One thing to note is the source is available on every unRAID installation as md.c and unraid.c which are heavily modified from Linux open source md.c and raid5.c. Since they both fall under GNU v2 they are required to be included in the distribution. The only post I found where Tom explains these parameters is http://lime-technology.com/forum/index.php?topic=4625.msg42091#msg42091.

Ran normal test: Tunables Report from unRAID Tunables Tester v2.0 by Pauven NOTE: Use the smallest set of values that produce good results. Larger values increase server memory use, and may cause stability issues with unRAID, especially if you have any add-ons or plug-ins installed. Test | num_stripes | write_limit | sync_window | Speed --------------------------------------------------------------------------- 1 | 1280 | 768 | 384 | 108.5 MB/s 2 | 1408 | 768 | 512 | 118.9 MB/s 3 | 1536 | 768 | 640 | 125.8 MB/s 4 | 1664 | 768 | 768 | 127.0 MB/s 5 | 1920 | 896 | 896 | 127.1 MB/s 6 | 2176 | 1024 | 1024 | 122.5 MB/s 7 | 2560 | 1152 | 1152 | 127.6 MB/s 8 | 2816 | 1280 | 1280 | 127.7 MB/s 9 | 3072 | 1408 | 1408 | 126.5 MB/s 10 | 3328 | 1536 | 1536 | 127.5 MB/s 11 | 3584 | 1664 | 1664 | 127.5 MB/s 12 | 3968 | 1792 | 1792 | 127.5 MB/s 13 | 4224 | 1920 | 1920 | 127.5 MB/s 14 | 4480 | 2048 | 2048 | 127.5 MB/s Completed: 0 Hrs 28 Min 48 Sec. Best Bang for the Buck: Test 4 with a speed of 127.0 MB/s Tunable (md_num_stripes): 1664 Tunable (md_write_limit): 768 Tunable (md_sync_window): 768 These settings will consume 149MB of RAM on your hardware. Unthrottled values for your server came from Test 8 with a speed of 127.7 MB/s Tunable (md_num_stripes): 2816 Tunable (md_write_limit): 1280 Tunable (md_sync_window): 1280 These settings will consume 253MB of RAM on your hardware. This is 23MB more than your current utilization of 230MB. NOTE: Adding additional drives will increase memory consumption. In unRAID, go to Settings > Disk Settings to set your chosen parameter values.

Interestingly enough, even at 50% bigger your read allocation is worse than stock unRAID. I don't see why you'd call it "worse", when it is in fact giving me better performance. See, sync_window and write_limit in this example are exactly what your tuning test would suggest. Only on top of that I have the num_stripes value a little larger. The result is having both fastest possible parity check speeds and better overall responsiveness when reading/writing to the server while a parity check is running. That's an interesting challenge. I think I'll modify my script to do a few iterations for num_stripes, say at 11%, 15%, and 20% at each pass, while at the same time a controlled and throttled read from the array is taking place. Six pages later we come full circle to post #4 and the link in post #7. You have an interesting and useful script, but it is only optimizing for maximum parity throughput with no regard for simultaneous array reads or writes (the basic media server functions). Thus, the script has the potential to cause problems for people wanting to use their media server while parity sync is active after using this script. That will lead those to run it in the off-hours which for most is already done that way. All said, it is a useful tool especially for those with large arrays where parity sync can take painful amounts of time. However, I would think to paint a clearer picture for people about the purpose of this script a note or disclaimer should be put in the first post to inform people that this script tries to tweak the the tune-ables while only quantifying one of the three (md_sync_window) and empirically assigning a value for the other two (md_num_stripes being some percent higher than md_sync_window and md_write_limit basically being carried). I can see some getting an increase in performance all around and not just with parity throughput because they have more RAM than processor. Testing with simultaneous writes and checking IO waits would definitely make this an all-encompassing tool to provide the best numbers for the tune-ables. With writes comes the issue of other people's data which is most likely the reason Tom went conservative in the first place (along with the 512MB RAM min requirement).

Yes. No. To all of it. That certainly clears things up, thank you. http://lime-technology.com/forum/index.php?topic=4625.msg42091#msg42091

Ok, I'm a bit confused. Granted I have not run the test and that I skimmed through the posts above, I gather it is only tuning for max throughput of a parity check with no regard to the effect on anything else happening with the disk subsystem. Is it monitoring the IO waits? How does it simulate external read/write requests while doing the parity check? The default config is obviously not set up for maximum speed of parity calculations and it is done that way on purpose so that the disk IO queue is not too deep to create excessive delays in responding to external read/write requests during the parity calc. By maximizing parity calculation speed you sacrifice usability during the parity process.

http://www.tomshardware.com/reviews/4tb-3tb-hdd,3183-4.html