smakovits

Ah, I see it now. The bad drive has been unassigned and the new precleared drive is rebuilding. This drive will be RMA-ed.

Can WDIDLE be run on drives that are already in the array without losing data? I have some very high counts on my EARS and EADS drives and hope to prevent it from getting too much higher.

funny thing. so before stopping my array to add the drive to replace the failing one, MyMain was showing the failing drive as needing sectors moved or what not, now after the reboot it does not. What gives? here is the latest report smartctl -a -d ata /dev/sdb (disk1) smartctl version 5.38 [i486-slackware-linux-gnu] Copyright © 2002-8 Bruce Allen Home page is http://smartmontools.sourceforge.net/ === START OF INFORMATION SECTION === Device Model: WDC WD20EADS-65R6B0 Serial Number: WD-WCAVY2136899 Firmware Version: 01.00A01 User Capacity: 2,000,398,934,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 8 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Tue Oct 25 23:31:37 2011 EDT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x84) Offline data collection activity was suspended by an interrupting command from host. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (42360) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x303f) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 155 149 021 Pre-fail Always - 9225 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 702 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 081 081 000 Old_age Always - 14147 10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 56 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 25 193 Load_Cycle_Count 0x0032 069 069 000 Old_age Always - 393110 194 Temperature_Celsius 0x0022 121 109 000 Old_age Always - 31 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 196 196 000 Old_age Always - 1461 198 Offline_Uncorrectable 0x0030 200 198 000 Old_age Offline - 7 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 196 000 Old_age Offline - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay.

Thanks Joe. I did read your post, but I wanted to be overly cautious... Once the preclear is complete, I want to confirm my steps. Stop array. Unassign failing disk, assign new, and then start array? There is a check box to rebuild parity or something like that, right? Also, is there a way to look at the AF drives in the current array to make sure that they are indeed formatted correctly with -A and NO jumper or jumper and no -A? I look at the hd link via MyMain and all my disks list sector 63. may have nothing to do with the sector 64, but thought it was worth the ask. Thanks again for all the help.

If I were to run this do I have to remove the drives from the Array? what is the overall affect on the disk and its content and the way it is seen by unRAID? I have a few drives showing 300k and even 400k + load cycles. If a typical lifetime is supposed to average 300-600k should I be worried?

OK, I have 4k alligned set for all disks, but the drive I am adding is an EADS RMA drive, so it is not an AF disk. Does this mean that the -a, -A and 4k-alligned stuff does not even come into play? If I ran the command with -a, does it matter at all?

OK really quick. Replacing a failing non AF drive (EADS). My server is set to default 4k aligned. Therefore, does this mean I need the -a switch? ./preclear_disk.sh -a -m [email protected] /dev/sdX Figured I would ask before I started the 24 hour process wrong...

excuse my ignorance but how does one do that?

Thanks, that worked. Now, while on topic of failing drives, I want to ensure that I only have the one bad drive and that some of my failed writes to another disk are simply a result of the server being unstable. Therefore, I got the smart report from the other drive that was being written at the time of the write failure, just to check its health. smart.txt

Drive shipped... Crazy thing, I can still access unraid via putty and as a share to write to and read/watch content. However, the web interface is now down completely and inaccessible. I was going to reboot, but thought it might only make issues. So I will wait on the drive to arrive first.

Is it OK to just leave the drive in the array for the time being? RMA submitted, so I assume advance turn around is next week +1 day for preclear.

OK, I want to post back and report this as SOLVED. Turns out that the culprit was my on-board NIC. I installed an Intel NIC and ever since, I have been running clean. I no longer see the random complete system lockups during transfers. It has been a few months now that I have been running like this, so I believe it is safe to say it was the NIC in the end. I couldn't be happier to be functioning and stable again. I see my transfers in the 25-30MB/s range. While not the much higher that others have reported in the past, this seems pretty good to me. The only other thing I have given thought to is jumbo frames. Any thoughts?

OK, I will look into a cmos battery. otherwise, here is the SMART output. smart.txt

Is there an easy way to tell which drive that is? I thought it was sdb, but the smart report said passed, same for sda, therefore, I either have the wrong drive or I am missing something.

Hey guys. Been running solid for a good bit ever since I replaced the on-board NIC with a add-in Intel card, until today. All of a sudden access became slow and sometimes unresponsive. I thought it was a switch that I added to the network in the office to work on another computer and removing that I thought the issue was resolved, but I was wrong and the trouble returned. As a result I thought it could only be one of 2 things, maybe a bad NIC cable in the office or trouble on the server. I checked the logs and didnt like what I saw, but it was from several days ago, so i am not sure if it is associated with today's issues or not. as for the cable, it is yet to be replaced. At any rate, I wanted to start by uploading the logs from the system to see if there is anything useful in them. This was on the smart report page: Disk 1: *ERROR* - Current_Pending_Sector has increased from 0 to 1461 since 2011-03-20 Disk 1: *ERROR* - Current_Pending_Sector it is now 1461 (error threshold is 5) Disk 1: OK - Power_On_Hours is 14003 Disk 1: *ERROR* - Offline_Uncorrectable has increased from 0 to 7 since 2011-03-20 Disk 1: *ERROR* - Offline_Uncorrectable it is now 7 (error threshold is 5) As always, thanks a ton. Hopefully it is nothing super terrible, but we will see. In the mean time I will also try replacing the cable. I had to compress the log due to size syslog-2011-10-19.zip

Well, time to bring this topic back into play. Soon after I last posted, my issue returned and so I plugged my unRAID server back into my D-link and things worked fine for a month. Thats great except I want the unRaid plugged into the switch in my rack vs the one outside it as that requires an extra cable. I thought what the heck, maybe it will work, but no. When my office machine is plugged to the dlink and the dlink to the trendnet to my unraid, things break. Unraid locking up is the worst part, but it is the only way I can test. ---------------------------- news flash ---------------------------- no joke, as I was writing this I began another test, office pc and unraid both in the trendnet. The transfer was running fine, way past the usual lockup point and suddenly I realized the transfer failed. unRAID is now unaccessible, so maybe the issue is the trendnet as suspected all along...because others report great success with this unit given its price, the unit is going to be sent tuesday for RMA and I guess we will see what happens when the new unit arrives. Man, if thats all it is I will be so relieved as my server has been a rock the last month when not connected to the trendnet and it sucks to break uptime over a network device... Sorry for the sudden 180 mid-thought, but when I saw the failure occurred when the trendnet was the only device in play, I thought there was no reason to continue. stay tuned

Oh hell, here is a good one. I tore everything apart tonight and started the network over. To test the Trendnet, I plugged everything into it and did some test copies and they all worked. Next, I added my Dlink green 8 port switch and transferred across between the 2, success. Added the wireless router and hard wired into it, across the dlink to the trendnet and success. Added my 10/100 to the trendnet, this is where the internet comes into the network and the dish network satellite box connects (no unraid traffic is really sent through it, but I wanted to note that it is part of the setup). And again everything good. Finally, I switched back the network cable to the original and things still worked. Each time I copied a 6 gig DVD structure folder and a 13 gig BD video file. I honestly have no clue. All test that were failing before, tonight succeeded and I have no idea how. My only guess would be that there is a faulty cable somewhere and once the data is traversing 2 switches to unraid it breaks. It must be way more sensitive than Windows because a transfer that failed and crashed the unraid box still completed to my home server. Because of this is why it makes no sense really, because I have changed nothing at this point. It is all back to the way it was and knock on wood, as of this moment it all worked. Now who is to say it wont fail tomorrow, but still. A job transfer that broke the unraid worked to WHS. This same transfer didnt break the unraid when both were plugged into the dlink switch. Then after un-hooking and re-hooking everything back, the same transfer that once broke the unraid server is now working, go figure. I will leave it a bit and see what happens after a few days.

has anything else recently changed in your setup?

What happens at your console, are you able to type anything there, or is that all locked up as well? Also, does it lock up on its own, or just during a file copy? I am having a similar issue, but mine occurs during file copies, http://lime-technology.com/forum/index.php?topic=11826.0 I have had no luck in tracking a solution and it is becoming a little aggravating, because there is very little information available as to a potential cause. My network is not that complex that I would be the only one to experience this so that is certainly why I am beginning to get nervous.

So at what point can this be considered an unknown error? I mean sure it can be the switch, but I can copy to other machines that are connected the same way or is it that unraid is that much more sensitive to potential network related issues?

I have attached my latest syslog after a clean reboot syslog-2011-04-07.txt

Found a different post talking about nic settings and such and it said to run some commands and post the results, so I figured it to be worth a shot to do it before anyone asked. Although not the same issue, his end solution was a new switch, which I hope not to need... Here are the outputs from the commands: when on Dlink where things seem to function, Tower login: root Linux 2.6.32.9-unRAID. root@Tower:~# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:24:1d:2c:a7:5f inet addr:192.168.0.101 Bcast:192.168.0.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2220 errors:0 dropped:0 overruns:0 frame:0 TX packets:1615 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:257772 (251.7 KiB) TX bytes:525824 (513.5 KiB) Interrupt:26 Base address:0x4000 root@Tower:~# ethtool eth0 Settings for eth0: Supported ports: [ TP MII ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Half 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Half 1000baseT/Full Advertised auto-negotiation: Yes Speed: 1000Mb/s Duplex: Full Port: MII PHYAD: 0 Transceiver: internal Auto-negotiation: on Supports Wake-on: pumbg Wake-on: g Current message level: 0x00000033 (51) Link detected: yes root@Tower:~# ping -c google.com ping: bad number of packets to transmit. root@Tower:~# ping -c 5 google.com PING google.com (74.125.93.105) 56(84) bytes of data. 64 bytes from qw-in-f105.1e100.net (74.125.93.105): icmp_seq=1 ttl=48 time=47.2 ms 64 bytes from qw-in-f105.1e100.net (74.125.93.105): icmp_seq=2 ttl=48 time=51.0 ms 64 bytes from qw-in-f105.1e100.net (74.125.93.105): icmp_seq=3 ttl=48 time=49.6 ms 64 bytes from qw-in-f105.1e100.net (74.125.93.105): icmp_seq=4 ttl=48 time=51.9 ms 64 bytes from qw-in-f105.1e100.net (74.125.93.105): icmp_seq=5 ttl=48 time=52.2 ms --- google.com ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4042ms rtt min/avg/max/mdev = 47.291/50.454/52.276/1.824 ms root@Tower:~# Now when plugged into the trendnet switch root@Tower:~# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:24:1d:2c:a7:5f inet addr:192.168.0.101 Bcast:192.168.0.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:3037 errors:0 dropped:0 overruns:0 frame:0 TX packets:2129 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:338384 (330.4 KiB) TX bytes:585960 (572.2 KiB) Interrupt:26 Base address:0x4000 root@Tower:~# ethtool eth0 Settings for eth0: Supported ports: [ TP MII ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Half 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Half 1000baseT/Full Advertised auto-negotiation: Yes Speed: 1000Mb/s Duplex: Full Port: MII PHYAD: 0 Transceiver: internal Auto-negotiation: on Supports Wake-on: pumbg Wake-on: g Current message level: 0x00000033 (51) Link detected: yes root@Tower:~# ping -c 5 google.com PING google.com (74.125.93.105) 56(84) bytes of data. 64 bytes from qw-in-f105.1e100.net (74.125.93.105): icmp_seq=1 ttl=48 time=52.7 ms 64 bytes from qw-in-f105.1e100.net (74.125.93.105): icmp_seq=2 ttl=48 time=50.3 ms 64 bytes from qw-in-f105.1e100.net (74.125.93.105): icmp_seq=3 ttl=48 time=47.7 ms 64 bytes from qw-in-f105.1e100.net (74.125.93.105): icmp_seq=4 ttl=48 time=48.1 ms 64 bytes from qw-in-f105.1e100.net (74.125.93.105): icmp_seq=5 ttl=48 time=51.8 ms --- google.com ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4035ms rtt min/avg/max/mdev = 47.701/50.176/52.759/2.000 ms root@Tower:~#

The card is a AOC-SAT2-MV8 I was looking for the NIC settings, but I cannot find them so maybe they werent changed, unless someone can not otherwise as a good place to look. Maybe it is in the kernel, outside of unraid itself. One thing I did notice was find was talk about Jumbo Frames, is it work enabling them with the MTU setting specified, 9000?

looks as though you must be running a gigabyte mobo and you have theHPA issue that many speak of and I just dealt with myself. I recommend reverting to a pre 4.7 version to address the HPA issue and then upgrade. Same thing happened to me when I went from 4.6 to 4.7. I went back to 4.6, fixed HPA and then upgraded without issue. 4.7 I hear is a bit more strict so it throws this error while the earlier versions do not.

One other thought I just had was could this be related to NIC speed settings, for some reason I want to say I may have things forced to gig. Recently I thought I heard something that with the new gig standards it is best to have it auto negotiate. Any thoughts? Maybe the D-link just handles it better than the trendnet. As another test, I just sent the same files that crashed the unRAID to my WHS (home server machine and it worked. that system is auto running at a gig connected to the trendnet, so the data took the same path that locked up the unraid. So I am thinking it is something between the unraid and trendnet. I just hate the hard resets to get the system back. I will reboot, and report a new syslog and also find the network settings.