September 25, 200817 yr Author As to the log files, that's too bad. I'll boot up a clean stock and check the location of the logs. Display messages are in the kernel ring buffer, so they can be copied out to disk wth dmesg even if the display log is not maintained. As for a package manager, binary downloads work, except using getline they choke on eof characters.... my original testing was on some image files that apparently didn't have an imbeded eof. When I try the downloaded tar archive, it gets part way through and chokes on the "unexpected end of file." I was afraid of that... Too bad the stock unraid did not include at least an FTP client. This was why I made mention of wget or better yet curl. With curl you can use http, ftp, sftp... Actually there could be allot of automation of things with curl... Grabbing the latest version of unRAID for example. i.e. grab the download html, parse it, determine current version vs latest version. etc, etc.... At the very least if you design a package manager that does some of the functionality mentioned, then later on there may be a method to do the rest (perhaps porting to a CGI when one becomes available... one way or another).
September 25, 200817 yr Author RS = ".{1,128}" # read 128 bytes at a time, this should allow us to handle binary data How does the RS = work in awk? Are you actually defining a record separator there? From my understanding, you can't really have any record separator, because a .tgz file could contain all 255 characters of the character set.
September 25, 200817 yr RS = ".{1,128}" # read 128 bytes at a time, this should allow us to handle binary data How does the RS = work in awk? Are you actually defining a record separator there? From my understanding, you can't really have any record separator, because a .tgz file could contain all 255 characters of the character set. When awk is invoked normally, you can set it to a character or string of characters that represent a record. (line) Normally, it is "\n", or "\r\n" When you use the -W re-interval option to awk, RS can be a regular expression. ".{1,128}" is a "regular expression" indicating you want to match from 1 to 128 of any character (the ".") It should be able to handle anything. Joe L.
September 25, 200817 yr In the context of other add-ons is there any way we can see the mean R/W transfer rates for each drive? Great for tracking down bottlenecks (like did I forget to remove the throttling jumper from my Seagate SATA drive ) I like this idea. Rather than a benchmark, it would be nice if it could take a short sample and determine how much data is read and written to each disk during that interval and compute the real-time statistics of that disk. So if you were playing a movie off of a disk, you'd be able to tell how much bandwidth it needed. If you were copying files to your array, you'd be able to measure the write speed. You could use such a tool to determine the fastest ways to accomplish various tasks.
September 25, 200817 yr Author I think bwm-ng can do disk statistics... http://www.netmonitoring.org/bwm-ng-network-disk-monitoring/bwm-ng-network-bandwidth-disk-io-monitoring/ http://repository.slacky.eu/slackware-12.1/utilities/bwm-ng/0.6/ UPDATE - it does... bwm-ng -t 1000 -T avg -i disk bwm-ng v0.6 (probing every 1.000s), press 'h' for help input: disk IO type: rate - iface Rx Tx Total ============================================================================== sdb: 0.00 B/s 0.00 B/s 0.00 B/s sda: 0.00 B/s 0.00 B/s 0.00 B/s sdc: 4.04 MB/s 67.53 KB/s 4.11 MB/s sdd: 0.00 B/s 0.00 B/s 0.00 B/s sde: 0.00 B/s 0.00 B/s 0.00 B/s sdf: 0.00 B/s 0.00 B/s 0.00 B/s sdg: 0.00 B/s 0.00 B/s 0.00 B/s sdh: 63.56 KB/s 67.53 KB/s 131.08 KB/s hda: 0.00 B/s 0.00 B/s 0.00 B/s md1: 3.98 MB/s 67.53 KB/s 4.05 MB/s md2: 0.00 B/s 0.00 B/s 0.00 B/s md3: 0.00 B/s 0.00 B/s 0.00 B/s md4: 0.00 B/s 0.00 B/s 0.00 B/s ------------------------------------------------------------------------------ total: 8.08 MB/s 202.58 KB/s 8.28 MB/s
September 25, 200817 yr Author There is also vnstat if people are interested. http://humdi.net/vnstat/
September 25, 200817 yr Ahhhhh Thanks for the lesson. No problem... It's why I get paid the big bucks. If it were not for that "-W re-interval" feature, the file browser plug-in would not be able to serve up pictures to your browser. Since awk was originally written as a text based utility, it does not have any documented features on how to handle binary data. (It was only by doing a LOT of searches on google did I find the clue I needed.) We are definitely using "awk" in ways very few others have in the past. Joe L. ps. Did you know there are a LOT of pages on the web... it is getting really difficult to keep up with all the stuff going on... and they keep adding more pages...
September 25, 200817 yr I thought this forum was supposed to be in ENGLISH! Every once in a while, a bit of geek-speak sneaks in, doesn't it.
September 26, 200817 yr I've been trying to run a short SMART report and nothing seems to happen. Should the results be displayed on the same page like the hdparm results? I have tried it a couple times on 300GB drive and left it for almost 3 hours each time and nothing seems to happen. How long does it take for your drives?
September 26, 200817 yr Author The way the smart tests work is you submiit the test, then come back later and check the status via smart logs. short test is around 2 minutes. long test is around 2 hours. Activity on the drive slows down the tests also the long test slows down basic I/O to the drive. If doing a long test, choose an idle time.
September 26, 200817 yr I've been trying to run a short SMART report and nothing seems to happen. Should the results be displayed on the same page like the hdparm results? I have tried it a couple times on 300GB drive and left it for almost 3 hours each time and nothing seems to happen. How long does it take for your drives? The smart report buttons just schedule the report and as you said, return immediately. (basically, they start it running) Now that you have requested a Smart Test, and it has probably run to completion, use the "Smart Statistics" button to see the results. From what I understand, if the test is still running, it will say so. (I'm new at this SMART stuff too..., perhaps somebody with more experience will advise more) Joe L.
September 26, 200817 yr Author Also after submitting the test (scheduling it to run in the background via the drive's firmware) you can grab a smart log and see something happening. I.E. root@Atlas [1] /mnt/disk1/bittorrent>smartctl -d ata -a /dev/sdd Right around here... you will see if it is running. === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (27180) seconds. and right around here (near the end) is where you will see it running or it's completion status. SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 1812 - # 2 Short offline Completed without error 00% 1811 - # 3 Extended offline Completed without error 00% 1800 - # 4 Short captive Interrupted (host reset) 90% 130 - # 5 Short offline Completed without error 00% 129 - Note time life in hours will increment according to when the test was run. Also if the test is running or failed there will be a value in Remaining.
September 26, 200817 yr I get SMART data on any drive I try when I click on the "SMART Statistics" button even if I haven't clicked on the "Short SMART Test" button first. Is it possible that simply clicking on the "SMART Statistics button" performs the short SMART test?
September 27, 200817 yr Author Is it possible that simply clicking on the "SMART Statistics button" performs the short SMART test? I doubt that. Executing a long or short test is a specific task. If you look at the bottom of the smart statistics, you will see a history of the last tests. Compare Power_On_Hours in this section SMART Attributes Data Structure revision number: 10 ...skipping... 9 Power_On_Hours 0x0032 097 097 000 Old_age Always - 2691 with LIFETIME(HOURS) field in this section. SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error 1 Short offline Completed without error 00% 2454 - # 2 Extended offline Completed without error 00% 807 - # 3 Short offline Completed without error 00% 234 -
September 27, 200817 yr I noticed that that field was empty on all my drives, even after "running" several smart tests by hitting the buttons on the Disk Mgmnt page. Dropping to a command shell when I issue a smart test it fails every time on all of my array's Seagate 1tb drives: root@SERVER:~# smartctl -t short /dev/sda smartctl version 5.36 [i486-slackware-linux-gnu] Copyright (C) 2002-6 Bruce Allen Home page is http://smartmontools.sourceforge.net/ Short offline self test failed [unsupported field in scsi command] root@SERVER:~# any ideas?
September 27, 200817 yr Author I've not had these issues with smartctl 5.38. Perhaps updating to a later version will work with your drives and/or controller. root@Atlas /boot>smartctl -d ata -a /dev/sdd smartctl version 5.38 [i486-slackware-linux-gnu] Copyright (C) 2002-8 Bruce Allen Home page is http://smartmontools.sourceforge.net/ === START OF INFORMATION SECTION === Model Family: Seagate Barracuda 7200.11 Device Model: ST31000340AS Serial Number: 3QJ08RZW ..... 9 Power_On_Hours 0x0032 096 096 000 Old_age Always - 3549 ..... SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 2454 - # 2 Extended offline Completed without error 00% 807 - # 3 Short offline Completed without error 00% 234 - ..... root@Atlas ~>smartctl -d ata -t short /dev/sdd smartctl version 5.38 [i486-slackware-linux-gnu] Copyright (C) 2002-8 Bruce Allen Home page is http://smartmontools.sourceforge.net/ === START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION === Sending command: "Execute SMART Short self-test routine immediately in off-line mode". Drive command "Execute SMART Short self-test routine immediately in off-line mode" successful. Testing has begun. Please wait 1 minutes for test to complete. Test will complete after Sat Sep 27 16:11:08 2008 Use smartctl -X to abort test. ..... Self-test execution status: ( 246) Self-test routine in progress... 60% of test remaining. ..... SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Self-test routine in progress 60% 3550 - # 2 Short offline Completed without error 00% 2454 - # 3 Extended offline Completed without error 00% 807 - # 4 Short offline Completed without error 00% 234 - ....... SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 3550 - # 2 Short offline Completed without error 00% 2454 - # 3 Extended offline Completed without error 00% 807 - # 4 Short offline Completed without error 00% 234 -
September 28, 200817 yr I take it from that you guys aren't running a vanilla unRaid 4.3.3 from a stick.... 4.3.3 comes with 5.3.6 I found that in the "vanilla" 4.3.3, the buttons for mounting a disk not in the array don't work; from the syslog: "Sep 28 08:02:13 SERVER kernel: ReiserFS: sdd1: warning: unknown mount option "umask=111"" I think some more "vanilla" testing is needed....
September 28, 200817 yr I take it from that you guys aren't running a vanilla unRaid 4.3.3 from a stick.... 4.3.3 comes with 5.3.6 I found that in the "vanilla" 4.3.3, the buttons for mounting a disk not in the array don't work; from the syslog: "Sep 28 08:02:13 SERVER kernel: ReiserFS: sdd1: warning: unknown mount option "umask=111"" I think some more "vanilla" testing is needed.... I was using 5.3.6 version of smartctl up until last night when I installed 5.3.8 and the libraries needed to run it. The older version was unable to start short or long tests on one of my hard disks, but worked fine on all the others. I do intend the basic functions to be useable on a stock system. For some advanced ones, and some plug-ins, additional packages will need to be installed... towards that end, a package-manager/installer plug-in is in the works. As far as the mount option, I've only tried an NTFS partition, so thank you for the feedback... clearly, the available options are different with various file-system types and umask is not one allowed on reiserfs.. Clearly, I'll need to do something to deal with the mount options. It has nothing to do with the version of smartctl used. You were the first to report it. As I'm sure you know if you have been following this thread, it has been an iterative process to cover everyone's needs, as there is no way to test all possible combinations of software and hardware on all the unRAID user's servers. To cover any possibility, I'll make the mount option a set of items in the umenu.conf file, one per file-system type. That way, the button on the unmenu.awk disk-management screen will handle anything, and be locally configurable if a special option is needed by a specific user. Joe L.
September 28, 200817 yr Author I take it from that you guys aren't running a vanilla unRaid 4.3.3 from a stick.... 4.3.3 comes with 5.3.6 Yes, I upgraded my smartctl and added the support library for it too. I think it needs the cxx lib. root@Atlas /mnt/disk3/z.bittorrent>ldd /usr/sbin/smartctl linux-gate.so.1 => (0xb7f42000) libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7e5a000) libm.so.6 => /lib/libm.so.6 (0xb7e33000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0xb7e28000) libc.so.6 => /lib/libc.so.6 (0xb7ce6000) /lib/ld-linux.so.2 (0xb7f43000) root@Atlas /mnt/disk3/z.bittorrent>tar -tvzf /boot/custom/usr/share/packages/cxxlibs-6.0.8-i486-4.tgz | grep libstdc++ -rwxr-xr-x root/root 732224 2005-09-03 13:14 usr/i486-slackware-linux/lib/libstdc++.so.5.0.7 -r-xr-xr-x root/root 272136 2002-06-02 16:53 usr/i486-slackware-linux/lib/libstdc++-3-libc6.1-2-2.10.0.so -r-xr-xr-x root/root 274948 2001-03-18 23:13 usr/i486-slackware-linux/lib/libstdc++-3-libc6.2-2-2.10.0.so -rwxr-xr-x root/root 702796 2002-06-02 16:53 usr/i486-slackware-linux/lib/libstdc++.so.4.0.0 -rwxr-xr-x root/root 906580 2007-02-14 16:39 usr/lib/libstdc++.so.6.0.8 a vanilla unRaid 4.3.3 from a stick.... This is why I am building a base unpatched development environment.
September 29, 200817 yr I am install unRaid menu. I am add to "go" script for start awk....unmenu.awk cd /boot/unmenu_awk5 nohup awk -W re-interval -f unmenu.awk >/dev/null 2>&1 & cd / Great and nice work boys. Thanks for it
September 29, 200817 yr That essentially open a telnet session and kleeps it open at all times, correct? Is there any downside to that?
September 30, 200817 yr That essentially open a telnet session and kleeps it open at all times, correct? Is there any downside to that? I'm not sure what you mean by "that" If you invoke the unmenu.awk server on the command line by logging in via telnet and then typing while true; do awk -W re-interval -f unmenu.awk; done Then yes... you have an open telnet session. No downside I know of, other than the usual "security" ones. (Anybody can use the telnet session to do anything they want to on your server) I've had open telnet sessions, and often multiple sessions, open for days. The only time they are closed is if I reboot or lose power to the network switch. If you start the script with line like this in the "go" script, there is no telnet session involved, and none stay open. nohup sh -c "while true; do awk -W re-interval -f /boot/unmenu.awk; done" >/dev/null 2>&1 & Joe L.
September 30, 200817 yr That essentially open a telnet session and kleeps it open at all times, correct? Is there any downside to that? I'm not sure what you mean by "that" Joe L. Sorry, I thought I quoted Ropo when I replied. By "that" I meant Ropo's commands (specifically the second line) in his go file. If you start the script with line like this in the "go" script, there is no telnet session involved, and none stay open. nohup sh -c "while true; do awk -W re-interval -f /boot/unmenu.awk; done" >/dev/null 2>&1 & So then you can simply type //tower:8080/ in a browser and access unmenu whenever the server is running?
Archived
This topic is now archived and is closed to further replies.