arcane

Hi bonienl - version 1.6.9.

+1 This was immensely useful in v5. Would love to have the template for v6 if you could spare the time gfjardim. Thanks!

Hi All - posted this in the Dynamix plugins thread but thought I'd crosspost here since I'm having a problem with cache_dirs specifically. I recently upgraded to v6, and was using cache_dirs without a problem in v5. I have 4GB RAM, about 225K files (but not all are included in cache_dirs), and my folders are no deeper than five levels. Using the same arguments I did with v5, my disks keep spinning up with the plugin version in v6. I've tried messing with the min and max arguments (as discussed previously in the thread), the cache pressure, etc. but am not having any luck keeping my disks spun down. The min scan time argument had the largest effect when bumped up to 10, which kept the disks spun down for about 10-15 minutes before they started to spin up. If I disable folder caching, then the disks remain spun down. A far more minor problem is that I can't cache one of my user shares named "3D Movies" that I didn't have a problem with in v5. Other user shares with spaces in the name are cached successfully; it is only this particular share that doesn't remain selected in the GUI after being started. Any advice would be appreciated.

Hello, Yes, I am using PIA with your default settings. Is there an alternate port I should try? Thank you.

Hi BRiT, the directory is listed with read, write and execute permissions for all groups owned by nobody, group users. There are no actual files themselves in that user share, but folders with individual movies and metadata. This is the same structure as the other directories with spaces in the name. I have checked for extra spaces and case in the name, but same results (can't cache dirs).

Hmmm... I have a user share named "3D Movies" but it doesn't seem to be cached. I can choose it it the plugin, but after I enable the plugin, it is deselected. When I invoke cache_dirs manually from the command line, it says the directory does not exist. I have other user shares with spaces, and they are included without a problem. Any suggestions?

Hi binhex, just upgraded to v6 and installed your delugeVPN Docker, which works perfectly except for two issues. I was hoping I could get some of your input. 1) I have configured my PC settings to use Privoxy and to ignore it for local (intranet) addresses. However, when I try to access other systems on my local network by IP, I get a failed connection. I have to use the hostname to connect to the systems. 2) In addition, I have DHT enabled but it doesn't actually show any DHT connections as shown. Is this normal? Any advice appreciated. Thanks!

I'm not using 6, but the latest of 5 and had an Eaton UPS. APCUPSD would not work with the Eaton at all.

cirkator - Sure. Joe suggested playing with the cache pressure and running it in the foreground: "Best you can do is experiment with other cache_pressure values. (the tunable used by the kernel to try to free buffers as needed) cache_dirs -p 100 -w is as good as any to try. (and even it may not help) If not specified, it defaults to 10." "run cache_dirs -F -v in the foreground, to see where it is spending its time. you might have a file-system that needs attention, or can be excluded." Weebotech - Thanks for the suggestion. I'll remove the shares with the largest amount of files and see if that helps things any. Joe L. - Again, thanks for all your help and troubleshooting!

Thanks for the additional input Joe. At the moment, cache_dirs is running without any exclusions. I've tried some of the troubleshooting steps you mentioned in a private e-mail (e.g. running in foreground, changing cache pressure, etc.), but unfortunately haven't been getting any closer to solving the problem. I'm not convinced memory is a problem, as everything was running as expected under v4.7. Under rc12a, cache_dirs will run fine by itself, but as soon as spinup_when_accessed (presumably doesn't have a large memory footprint) is added to the mix, it stops working properly. Output of free -m: total used free shared buffers cached Mem: 4040 3934 105 0 214 3327 -/+ buffers/cache: 391 3648 Swap: 0 0 0 root@Tower:~#

Hi Joe - thought I'd combine the previous post with this one to keep things centralized. As you know, I recently upgraded to rc12a from 4.7 and was using cache_dirs previously without problem. However, it appears to be spinning up the disks as I'm opening up the root of the user share (on a PC or in a telnet session). It appears it's a problem when running in conjunction with the latest spinup_when_accessed script. If cache_dirs is running alone, then listing the contents does not spin up the disks. Running inotifywait which browsing to my Backups share (on disk12) shows: Setting up watches. Beware: since -r was given, this may take a while! Watches established. /mnt/user/ OPEN,ISDIR Backups /mnt/user/ OPEN,ISDIR Backups /mnt/user/ OPEN,ISDIR Backups /mnt/user/ CLOSE_NOWRITE,CLOSE,ISDIR Backups /mnt/user/ OPEN,ISDIR Backups /mnt/user/ CLOSE_NOWRITE,CLOSE,ISDIR Backups /mnt/user/ OPEN,ISDIR Backups /mnt/user/ CLOSE_NOWRITE,CLOSE,ISDIR Backups Input would be appreciated. Thank you.

As I recall, your collection is vastly larger than mine (with corresponding .jpg and .xml). How does your server manage it? Do you run across similar entries in your syslog?

Thanks Joe for the tips. I modified the shares to contain no brackets, but they do contain spaces. My current syntax, tested at the command line seemed to locate the files correctly: find "/mnt/user/Movies G-J" -type f \( -name *.jpg -o -name *.xml \) -exec cat "{}" >/dev/null \; However, after running the script, it seems to be exhibiting the same behavior of spinning up the disks as it traverses the shares searching for the files; eventually, almost all disks are spinning. I have attached an excerpt fro the syslog. Thanks! cache_excerpt.txt

Hi Joe. Adding this line as suggested seems to constantly keep my disks spinning as it searches for the files. As far as I can tell, almost all disks (since the movies are spread across almost all disks) have not been spun down for three days since the reboot after changes have been made. Is this unusual or should it take a while? Also, my movies are note under one user share but split into several alphabetical bins (e.g. E-J, etc.). Is this the correct syntax to have to search all the Movie shares - find /mnt/user/Movies*? Thanks Joe!

Will give that a shot... thanks tbone!

Was this problem ever resolved? I'm getting the same errors in syslog, made the cache_pressure change as suggested, but don't have any telnet sessions open. Running cache_dirs with just the -w switch. Thanks!

Whew... I was hoping I wouldn't have to redo all 22 drives. Here are the results: http://goo.gl/Mre5p I intend to do at least two more cycles on those drives which have reallocated sectors. Are there any other drives I should be concerned about? It seems like there are a lot of the Barracudas which have many of the same types of errors, but I'm not certain which are important. Perhaps I won't try preclearing that many drives again at once, if only because of the high temperatures. I can post the full logs if needed... Thanks for helping to interpret the results...

There's no way I need all that storage space for the immediate future so I'll kill power to a few of those backplanes before putting it into service. However, are the preclear results now invalid because of possible power problems? Hoping you'll say no...

Good point. I should probably look into upgrading... far cheaper to upgrade the PSU than buy more HDs! The 850W should be sufficient... OTOH, it's probably good that the system hasn't crashed yet especially considering I'm using a MB/CPu combo that hasn't been tested here before...

But the 2.0 TB Caviars finished about 7 hours less... but perhaps this is due to the RPM difference? I don't plan on running all 22 drives simultaneously when it's all up and running. Since unRAID spins down unused disks, the PSU should be good to go, right? You do bring up a point of concern though. If power was insufficient, would pre-clearing this many drives at once cause errors in the final report? On some of the drives, I note things near threshold like spin retries, high fly writes, etc.

It's what I had available I plan to transition to WD green drives eventually (I have two 2.0 TB EADS but those took over 30 hrs to clear... typical?). And I'm running it off a Seasonic X-650.

I noticed that. The Caviars finished at a higher temperature than the rest of the Seagate Barracudas I have been preclearing (which end up at an average of 35C). I also noticed that while temperatures were high ( >40C) during the pre-clear and post-read, they were average mid-30s in the phases between. I should also note that I'm pre-clearing 20 drives at once, and most of them are 7200 rpm. When I was doing the preclear with only 3 drives, temperatures were cooler (see a few posts back). Would this perhaps explain the temperature difference? I have replaced all fans in the 4224 running at max speed.

Just wanted some confirmation that I wasn't crazy. Didn't think it could reach those temps no matter how brief...

Just pre-cleared a new, unused WD Caviar Black drive. Although it looks fine, the temperature has me a bit concerned. Am I interpreting it correctly... that the temperature went over 100C at some point? The interim reports I received all averaged about 35C. Maybe it has something to do with me preclearing 20 disks at once? Will post results of other drives when finished. Thanks for looking. ========================================================================1.9 == invoked as: ./preclear_disk.sh -M 4 /dev/sdi == WDC WD2001FASS-00W2B0 WD-WMAY00412287 == Disk /dev/sdi has been successfully precleared == with a starting sector of 63 == Ran 1 cycle == == Using :Read block size = 8225280 Bytes == Last Cycle's Pre Read Time : 5:37:34 (98 MB/s) == Last Cycle's Zeroing time : 4:59:27 (111 MB/s) == Last Cycle's Post Read Time : 12:23:18 (44 MB/s) == Last Cycle's Total Time : 23:01:26 == == Total Elapsed Time 23:01:26 == == Disk Start Temperature: 35C == == Current Disk Temperature: -->46<--C, == ============================================================================ ** Changed attributes in files: /tmp/smart_start_sdi /tmp/smart_finish_sdi ATTRIBUTE NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS RAW_VALUE Temperature_Celsius = 106 117 0 ok 46 No SMART attributes are FAILING_NOW 0 sectors were pending re-allocation before the start of the preclear. 0 sectors were pending re-allocation after pre-read in cycle 1 of 1. 0 sectors were pending re-allocation after zero of disk in cycle 1 of 1. 0 sectors are pending re-allocation at the end of the preclear, the number of sectors pending re-allocation did not change. 0 sectors had been re-allocated before the start of the preclear. 0 sectors are re-allocated at the end of the preclear, the number of sectors re-allocated did not change. ============================================================================ ============================================================================ == == S.M.A.R.T Initial Report for /dev/sdi == Disk: /dev/sdi smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Device Model: WDC WD2001FASS-00W2B0 Serial Number: WD-WMAY00412287 Firmware Version: 05.01D05 User Capacity: 2,000,398,934,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 8 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Mon Apr 25 17:27:53 2011 PDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x80) Offline data collection activity was never started. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (29280) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x3037) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 100 253 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 100 253 021 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 6 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 100 253 000 Old_age Always - 0 9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 0 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 5 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 2 193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 13 194 Temperature_Celsius 0x0022 117 114 000 Old_age Always - 35 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 253 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. == ============================================================================ ============================================================================ == == S.M.A.R.T Final Report for /dev/sdi == Disk: /dev/sdi smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Device Model: WDC WD2001FASS-00W2B0 Serial Number: WD-WMAY00412287 Firmware Version: 05.01D05 User Capacity: 2,000,398,934,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 8 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Tue Apr 26 16:29:18 2011 PDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x80) Offline data collection activity was never started. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (29280) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x3037) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 199 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 100 253 021 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 6 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 100 253 000 Old_age Always - 0 9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 18 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 5 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 2 193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 15 194 Temperature_Celsius 0x0022 106 104 000 Old_age Always - 46 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. == ============================================================================

Just pre-cleared three drives and was wondering if everything is OK? Or should I run some additional cycles on the Seagate drives? Thanks! ========================================================================1.9 == invoked as: ./preclear_disk.sh -M 4 /dev/sdb == SAMSUNG HE103UJ S13VJ1NQ700004 == Disk /dev/sdb has been successfully precleared == with a starting sector of 63 == Ran 1 cycle == == Using :Read block size = 8225280 Bytes == Last Cycle's Pre Read Time : 3:26:40 (80 MB/s) == Last Cycle's Zeroing time : 3:05:45 (89 MB/s) == Last Cycle's Post Read Time : 6:43:34 (41 MB/s) == Last Cycle's Total Time : 13:17:02 == == Total Elapsed Time 13:17:02 == == Disk Start Temperature: 21C == == Current Disk Temperature: 26C, == ============================================================================ ** Changed attributes in files: /tmp/smart_start_sdb /tmp/smart_finish_sdb ATTRIBUTE NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS RAW_VALUE End-to-End_Error = 100 100 99 near_thresh 0 Airflow_Temperature_Cel = 74 79 0 ok 26 Temperature_Celsius = 75 73 0 ok 25 No SMART attributes are FAILING_NOW 0 sectors were pending re-allocation before the start of the preclear. 0 sectors were pending re-allocation after pre-read in cycle 1 of 1. 0 sectors were pending re-allocation after zero of disk in cycle 1 of 1. 0 sectors are pending re-allocation at the end of the preclear, the number of sectors pending re-allocation did not change. 0 sectors had been re-allocated before the start of the preclear. 0 sectors are re-allocated at the end of the preclear, the number of sectors re-allocated did not change. ============================================================================ ============================================================================ == == S.M.A.R.T Initial Report for /dev/sdb == Disk: /dev/sdb smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net ========================================================================1.9 == invoked as: ./preclear_disk.sh -M 4 /dev/sdc == ST31500341AS 9VS070GD == Disk /dev/sdc has been successfully precleared == with a starting sector of 63 == Ran 1 cycle == == Using :Read block size = 8225280 Bytes == Last Cycle's Pre Read Time : 4:28:47 (93 MB/s) == Last Cycle's Zeroing time : 4:10:28 (99 MB/s) == Last Cycle's Post Read Time : 9:32:32 (43 MB/s) == Last Cycle's Total Time : 18:12:55 == == Total Elapsed Time 18:12:55 == == Disk Start Temperature: 35C == == Current Disk Temperature: 30C, == ============================================================================ ** Changed attributes in files: /tmp/smart_start_sdc /tmp/smart_finish_sdc ATTRIBUTE NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS RAW_VALUE Raw_Read_Error_Rate = 110 114 6 ok 27804642 Spin_Retry_Count = 100 100 97 near_thresh 0 End-to-End_Error = 100 100 99 near_thresh 0 High_Fly_Writes = 59 86 0 ok 41 Airflow_Temperature_Cel = 70 65 45 near_thresh 30 Temperature_Celsius = 30 35 0 ok 30 Hardware_ECC_Recovered = 54 40 0 ok 27804642 No SMART attributes are FAILING_NOW 0 sectors were pending re-allocation before the start of the preclear. 0 sectors were pending re-allocation after pre-read in cycle 1 of 1. 0 sectors were pending re-allocation after zero of disk in cycle 1 of 1. 0 sectors are pending re-allocation at the end of the preclear, the number of sectors pending re-allocation did not change. 0 sectors had been re-allocated before the start of the preclear. 0 sectors are re-allocated at the end of the preclear, the number of sectors re-allocated did not change. ============================================================================ ============================================================================ == == S.M.A.R.T Initial Report for /dev/sdc == Disk: /dev/sdc smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net ========================================================================1.9 == invoked as: ./preclear_disk.sh -M 4 /dev/sdd == ST31500341AS 9VS0AK9C == Disk /dev/sdd has been successfully precleared == with a starting sector of 63 == Ran 1 cycle == == Using :Read block size = 8225280 Bytes == Last Cycle's Pre Read Time : 4:23:55 (94 MB/s) == Last Cycle's Zeroing time : 4:25:01 (94 MB/s) == Last Cycle's Post Read Time : 9:24:00 (44 MB/s) == Last Cycle's Total Time : 18:14:03 == == Total Elapsed Time 18:14:03 == == Disk Start Temperature: 36C == == Current Disk Temperature: 32C, == ============================================================================ ** Changed attributes in files: /tmp/smart_start_sdd /tmp/smart_finish_sdd ATTRIBUTE NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS RAW_VALUE Raw_Read_Error_Rate = 105 113 6 ok 8274288 Spin_Retry_Count = 100 100 97 near_thresh 0 End-to-End_Error = 100 100 99 near_thresh 0 High_Fly_Writes = 93 95 0 ok 7 Airflow_Temperature_Cel = 68 64 45 near_thresh 32 Temperature_Celsius = 32 36 0 ok 32 Hardware_ECC_Recovered = 54 33 0 ok 8274288 No SMART attributes are FAILING_NOW 0 sectors were pending re-allocation before the start of the preclear. 0 sectors were pending re-allocation after pre-read in cycle 1 of 1. 0 sectors were pending re-allocation after zero of disk in cycle 1 of 1. 0 sectors are pending re-allocation at the end of the preclear, the number of sectors pending re-allocation did not change. 3 sectors had been re-allocated before the start of the preclear. 3 sectors are re-allocated at the end of the preclear, the number of sectors re-allocated did not change. ============================================================================ ============================================================================ == == S.M.A.R.T Initial Report for /dev/sdd == Disk: /dev/sdd smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net