Neo_x

Hi guys I am having a challenge with s3_sleep (set to shutdown rather than sleep) it seems like all the timers runs out, but then no shutdown happens. Part of the syslog is captured as per below : is there maybe something i am missing? eg do i maybe need to add a powerdown package still? thx Neo_x *Edit* managed to find the answer from earlier in this thread : The above settings act like an unconditional shutdown, in other words your system goes down after 30 minutes - if all works (better define at least one condition). s3_sleep makes use of 'powerdown' to do a proper shutdown of the system, however this is not standard in unRAID and the powerdown plugin needs to be installed. See this topic: http://lime-technology.com/forum/index.php?topic=31735.0 To install powerdown, use the install page under the plugin manager. Copy and paste this URL to install the plugin https://github.com/dlandon/unraid-snap/raw/master/powerdown-x86_64.plg

Awesomeness!! thank you for your efforts into this - experienced exactly the same. Will implement the two changes , and report back if anything is still going wrong.

thx - overlooked that - fixed in OP

Neo_X commendable work. As WeeboTech points out you need to look a bit deeper to get the info you actaully need. Look at this post for some useful relevant debug commands http://lime-technology.com/forum/index.php?topic=4500.msg286103#msg286103 Hi Nas/ guys as promised, some more data below -which hopefully can assist. Some observations - strangely not seeing major memory leakage (compared 4 hours and 10 hours), will keep monitoring. Big difference is that on the same hardware, Memory usage on V5 was 1116MB versus 2777MB on V6(no xen) and 2392MB on V6(xen)..... which is odd at best. Unless 64 bit has more overheads to store the same data? All testing was performed with stock unraid's , capturing before cache_dirs, and after running cache_dirs for about 4 hours. v6 cache_dirs was modified to comment out the ulimit line I know some data was repeated unnecessary(eg file counts and sizes), but rather repeated them to make sure nothing slips through Character limit - had to attach the captures rather (2 posts) EDIT -As recommended by NAS, pastebin was utilized v5_0_5 - no cache_dirs -> http://pastebin.com/cHRDuEy8 v5_0_5 - cache_dirs running(4 hours) -> http://pastebin.com/GPCB9tuB v6b4(no xen) - no cache_dirs -> http://pastebin.com/UP3TQ36w v6b4(no xen) - cache_dirs running(4 hours) -> http://pastebin.com/4RTYZrAW v6b4(xen) - no cache_dirs -> http://pastebin.com/6LXjv40P v6b4(xen) - cache_dirs running(10 hours) -> http://pastebin.com/Zy2esyEZ

i did run cache-dirs via go script, i found it consumed all resources on my server and basically killed dom0 and all domU's after approx 2 days, what issue were you seeing when starting it via go script, high memory consumption, or something else?. i have of course stayed away from running cache-dirs for the time being after that experience, can anybody confirm they have this running stable on v6 and if so what flags they are using. cheers. binhex. p.s nice to see ya on another board :-) Hi Binhex same here - glad to see you. I have one domain running , and upon starting cache_dirs via the go script, and checking top after about 30 mintues to an hour, i saw 100% cpu utilization (most of which was taken up by the cache_dirs script). Luckily on my end i have dedicated one core to Dom0, which stabilized the system drastically (thus nothing crashed) my syslinux : label Xen/unRAID OS kernel /syslinux/mboot.c32 append /xen dom0_max_vcpus=1 dom0_vcpus_pin --- /bzimage --- /bzroot strange part - upon trying "cache_dirs -q" it reported that it is not running... thus i dad to resort to manually killing the process id i tried adding a sleep 120 in the go script before the cache_dirs line, but it didn't help only option so far was to manually telnet into the box after a few minutes, and run the cache_dirs from the telnet prompt. busy running with stock tests , will check if the same occurs when Xen is not running. regards Neo_x

Neo_X commendable work. As WeeboTech points out you need to look a bit deeper to get the info you actaully need. Look at this post for some useful relevant debug commands http://lime-technology.com/forum/index.php?topic=4500.msg286103#msg286103 Hi Nas I can make the debug captures no problem (even a comparison after a few hours between V5 / V6 again) , but the very strange part is that i am seeing the reverse - eg V5 is managing the Cache with less memory than what V6 (64bit) is. hmmm just for fun, i think i will make a capture without XEN as well, just in case regards Neo_X

Some findings on Cache_dirs ( little bit troubling). on V5 , via Top, caching both my movies and series media ( yes there is a huge amount(11TB) of that..). On V5, top reports Full Top Capture (V5) -> http://imgbin.org/images/17191.JPG On V6 top reports (note about 2Gb from max is used by a domain/VM, but i think dom0 still remains separate) Full Top Capture ( V6) -> http://imgbin.org/images/17192.JPG Thus V5 caching uses about 1.58GB of Memory total on the V5 unraid box, versus the huge 3.62GB V6 unraid (and V5 has Mysql(+-300Mb) running as well) - that is over 1.5GB more to cache the same data .... I know Cache_dirs is only performing a simple find function, and thus the issue is most probably not with the script, but does anybody else maybe have an idea as to why this could be? I really don't like leaving the server running with only about 50MB of memory free root@Storage:~# free -m total used free shared buffers cached Mem: 3593 3539 53 0 767 1696 -/+ buffers/cache: 1076 2517 Swap: 0 0 0 Hopefully a linux guru can maybe assist? :) - *free beers is available* Ps V6 Cachedirs also gave an issue when started via go script - not sure if related. had to kill the process id , and restart manually.

I did consider cases like this when I added that "-d type" option. You are one of the first to report they were able to use it on their hardware. ... .. dev/sdb and /dev/sdg will never be the same disk. You can type: ls -l /dev/disk/by-id/* to see a listing of all your disks and disk partitions by model and serial number. The preclear script was written to not allow you to clear a disk that is assigned to the array, or mounted and in use. Have fun, Joe L. Thx for an informative post Joe yes the controller cost me a pretty penny (more in the region of penny wise pound foolish ). about 2/3 years ago i was a strong follower of RAID5/ RAID6 - and thus decided i needed a good controller that can give me the biggest RAID-6 possible relative to my case capacity at that stage - 16 drives max. problem is - as with any data - i outgrow the server recently, and thus needed to upgrade. Problem is - although its a very nice controller, it was following the same rules of RAID. all drives needs to be the same capacity -and it seems it had another limitation - 16 drives max for a cluster. so yes - i wasn't willing to shell out additional cash upgrading the 16 x 1.5TB drives to 16 x 3TB drives, as this would just be pointless(converting the RAID would have taken days if not weeks - in the region of 30 hours per drive), and then i would be stuck in the same position a few years down the line. so yes - now i am moving over to Unraid - and definitely seems to be the way to go. otherwise - nope are having a "normal" desktop pc in use ( Tri-SLI board with one of the first Quad cores(Q6600) running and i think in the region of 4GB RAM - nothing serious ( home user ). the controller have a duo core cpu and 512MB ram onboard- so i think i am safe performance wise. i am seemingly having some issues with spin-up / spin-down on the controller - will investigate a bit more once i finished migrating data over from backups onto the server. will keep you guys updated

Hi guys nvm - Solved - discovered after studying the usage script that the following command is possible:\ preclear_disk.sh -d sat /dev/sda this instructs preclear to utilize alternate commands when running Smartctl. hope someone can assist - i am running Unraid via an Adaptec controller (Model 52445). It is rather overkill for unraid since it is meant for high levels of RAID, but i didn't want to shell out additional $$$'s to get another controller - currently its performing admirably with roughly 64MB/s on post read clearing 12 drives at the same time everything seems fine - ie i set all the disks up as JBOD - which seems to simluate pass through(not sure on the correct terms) anyhow - so far so good. Unraid picks up the first set of 12 drives i connected (having power issues with connecting more ). Problem i am having - it seems that smartctl doesnt give correct stats on the drive. See sample output below. root@Storage:~# smartctl -a /dev/sdb smartctl 5.40 2010-10-16 r3189 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net Device: ST2000DL003-9VT1 Version: CC3C Serial number: 6YD1RLL6 Device type: disk Transport protocol: SAS Local Time is: Sat Sep 1 08:46:46 2012 SAST Device supports SMART and is Enabled Temperature Warning Disabled or Not Supported SMART Health Status: OK Error Counter logging not supported Device does not support Self Test logging root@Storage:~# as can be expected - this messes up Preclear a bit, since it is unable to read Smart results before ,during and after. (although otherwise - it doesn't crash or halt the preclear in any way - GREAT SCRIPT JOE L ) I managed to find a smartctl command that does give the output for the drive as required Thank you Google root@Storage:~# smartctl -d sat --all /dev/sg1 smartctl 5.40 2010-10-16 r3189 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Device Model: ST2000DL003-9VT166 Serial Number: 6YD1RLL6 Firmware Version: CC3C User Capacity: 2,000,398,934,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 8 ATA Standard is: ATA-8-ACS revision 4 Local Time is: Sat Sep 1 08:47:26 2012 SAST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED See vendor-specific Attribute list for marginal Attributes. General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 612) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x30b7) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 109 099 006 Pre-fail Always - 24419656 3 Spin_Up_Time 0x0003 090 090 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 286 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 073 060 030 Pre-fail Always - 4318984658 9 Power_On_Hours 0x0032 096 096 000 Old_age Always - 3906 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 286 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0 189 High_Fly_Writes 0x003a 099 099 000 Old_age Always - 1 190 Airflow_Temperature_Cel 0x0022 059 023 045 Old_age Always In_the_past 41 (75 200 42 25) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 285 193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 286 194 Temperature_Celsius 0x0022 041 077 000 Old_age Always - 41 (0 14 0 0) 195 Hardware_ECC_Recovered 0x001a 036 015 000 Old_age Always - 24419656 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 66043712114499 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 2825939409 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 3158544765 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. So i guess my question is then - how do i go about trusting a drive after a preclear? reading through some of the posts, it seems i need to look for the following : FAILING NOW attributes, 5 Reallocated_Sector_Ct (this should be preferably zero - or else stay a very low number.) 197 Current_Pending_Sector (this should be preferably zero - or else stay a very low number.) also - should i be worried about which "device" i am clearing? (since i gather that SDB and SDG is possibly the same thing...) clear should be completed in about 6 hours - will report on any results i don't understand Thank you Neo_x PS syslog attached just in case syslog.zip