June 14, 201313 yr I've been trying to copy a few movies to my server and have been having some problems. Here's the scenario. I am currently pre-clearing a 4TB hard drive and have been throughout this problem, but that never caused me any issues before. When I initiate the copy to from my Win7 desktop, it starts as you would expect and I get ~35MB/s transfer speed, but after a minute or two it drops to 300-400 KB/s and mostly stays there. I've seen it bump back up to ~35MB/s, but only for a few seconds and then drop back down. System: unRAID 5.0rc10 running as a VM in ESXi 2GB RAM drives are all running on IBM M1015's in passthrough mode Log is attached. syslog-2013-06-14.txt
June 14, 201313 yr unRAID 5.0rc10 running as a VM in ESXi 2GB RAM Are the symptoms the same running bare metal? Do the symptoms change with different RC versions? No offense intended, but if you aren't running the latest RC on bare metal, I wouldn't expect much help in the RC support forum.
June 14, 201313 yr Author unRAID 5.0rc10 running as a VM in ESXi 2GB RAM Are the symptoms the same running bare metal? Do the symptoms change with different RC versions? I can't be certain. As I said, I am currently in the middle of a preclear which has been runnning for ~38 hours and has a couple days to go. I don't want to stop that right now to reboot and try new configurations. Plus, like I said, this hasn't happened before although I can't be 100% certain I tried copying files to the server during a preclear. No offense intended, but if you aren't running the latest RC on bare metal, I wouldn't expect much help in the RC support forum. Maybe I should have posted this on the the plain General Support forum? If so, my apologies. Not really sure the difference between the two. I posted here because I'm running a 5.0rc. If it needs moved, that's fine.
June 14, 201313 yr The preclear will read or write until the buffer cache is full and the buffer cache starts getting flushed. This can, and will, impact performance on drives, if the controllers are shared. I.E. If you are preclearing a drive on the same controller as the parity and data drive. You could try and renice the DD and preclear parent via a telnet session. Make it the lowest priority. I think you can do this via top also. Also do you have any kernel tunings enabled?
June 14, 201313 yr You might look at the smart report(s) for the drive you are writing to and the parity drive if you are writing directly to the array and not to a cache drive first. I doubt you will see anything but it is worth a look.
June 14, 201313 yr Author The preclear will read or write until the buffer cache is full and the buffer cache starts getting flushed. This can, and will, impact performance on drives, if the controllers are shared. I.E. If you are preclearing a drive on the same controller as the parity and data drive. I'm pretty sure the drive I was writing to and the drive I am preclearing are both on the same controller. You could try and renice the DD and preclear parent via a telnet session. Make it the lowest priority. Not sure what renice and DD are? I think you can do this via top also. Also do you have any kernel tunings enabled? No. I've read a bit about them in the past couple days, but frankly I don't understand them and have been afraid to mess with them.
June 14, 201313 yr Author You might look at the smart report(s) for the drive you are writing to and the parity drive if you are writing directly to the array and not to a cache drive first. I doubt you will see anything but it is worth a look. Thanks I'll try that. Can you run a smart test on a drive that is preclearing?
June 14, 201313 yr Can you run a smart test on a drive that is preclearing?I don't think so, but that's not what's needed. You can definitely get a smart report while it's preclearing. Testing commands the drive to do things, getting a report just reads the smart statistics as they are right now. A preclear probably does at least as much as a long smart test anyway.
June 14, 201313 yr Can you run a smart test on a drive that is preclearing?I don't think so, but that's not what's needed. You can definitely get a smart report while it's preclearing. Testing commands the drive to do things, getting a report just reads the smart statistics as they are right now. A preclear probably does at least as much as a long smart test anyway. you can grab a smart report with -a while you can trigger a test, I would not recommend it, It will compete with the preclear. It could cause the test to be aborted. In the past with my hosting company, we had scheduled smart tests that would cause all sorts of grief on busy servers. do one or the other, not both at the same time. accessing the smart report is fine.
June 14, 201313 yr What they said. I didn't look at your syslog and any errors should be reported there but if a drive was going bad you might be able to see that in the smart reports. I check smart reports while drives are preclearing and I'm writing to the drive all the time. It might slow down access some but you are already having that problem so that shouldn't matter. But as jonathanm and WeeboTech said it is not likely if they didn't see anything in the log.
June 14, 201313 yr Author Bob - I am not using a cache drive. The SMART reports look fine to me, but I'm no expert. SMART report for the preclearing drive: SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_ FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 118 100 006 Pre-fail Always - 196324336 3 Spin_Up_Time 0x0003 096 096 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 4 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 100 253 030 Pre-fail Always - 524116 9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 39 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 4 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0 189 High_Fly_Writes 0x003a 094 094 000 Old_age Always - 6 190 Airflow_Temperature_Cel 0x0022 072 071 045 Old_age Always - 28 (Min/Max 24/29) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 4 193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 8 194 Temperature_Celsius 0x0022 028 040 000 Old_age Always - 28 (0 23 0 0) 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 177485228539942 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 10614449992 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 23846290590 SMART Error Log Version: 1 No Errors Logged SMART report for the drive I was writing to: D# ATTRIBUTE NAME FLAG VALUE WORST THRESH TYPE UPDATED FAILED RAW VALUE 1 Raw Read Error Rate 0x002f 200 200 051 Pre-fail Always Never 0 3 Spin Up Time 0x0027 172 168 021 Pre-fail Always Never 6391 4 Start Stop Count 0x0032 098 098 000 Old age Always Never 2660 5 Reallocated Sector Ct 0x0033 200 200 140 Pre-fail Always Never 0 7 Seek Error Rate 0x002e 200 200 000 Old age Always Never 0 9 Power On Hours 0x0032 084 084 000 Old age Always Never 11855 10 Spin Retry Count 0x0032 100 100 000 Old age Always Never 0 11 Calibration Retry Count 0x0032 100 100 000 Old age Always Never 0 12 Power Cycle Count 0x0032 100 100 000 Old age Always Never 233 192 Power-Off Retract Count 0x0032 200 200 000 Old age Always Never 222 193 Load Cycle Count 0x0032 194 194 000 Old age Always Never 19559 194 Temperature Celsius 0x0022 122 095 000 Old age Always Never 28 196 Reallocated Event Count 0x0032 200 200 000 Old age Always Never 0 197 Current Pending Sector 0x0032 200 200 000 Old age Always Never 0 198 Offline Uncorrectable 0x0030 200 200 000 Old age Offline Never 0 199 UDMA CRC Error Count 0x0032 200 200 000 Old age Always Never 0 200 Multi Zone Error Rate 0x0008 200 200 000 Old age Offline Never 0 Is there anything in my log that indicates any problem?
June 14, 201313 yr They look fine to me too and doubt jonathanm and WeeboTech would disagree. Was a long shot. Most likely what WeeboTech said. I preclear on a separate box so I have no practical experience with preclearing and reading/writing to the array at the same time. Really need to update my M1015 and RES2SV240 firmwares to see if that really will fix my problem with running preclear on my main unRAID boxes. FYI: Some times drives are just slow. The 250GB drive I got with my HP N40L is one of those. It preclears just fine ran ~6 cycles without errors but the most it got was 40-60MB/s if I remember correctly. The 1.5TB WD Green I replaced it with was 70-120MB/s on a preclear.
June 14, 201313 yr Author I understand some drives are just slower, but 300-400 KB/s? That's unbearable. I'll test the speed again once the preclear is completed to verify, but it probably is because of that. I've not had a problem writing to that drive before.
June 14, 201313 yr Log is attached.Unrelated to the current speed issue, but you need to fix whatever is causing your duplicate name problem. Apparently you have a few file names duplicated across your disks in the same relative path. Notice I did NOT say you have duplicate files, because you very easily could have 2 different files with the same name, so you need to check before you delete them. The easiest way to resolve it is to change the name of the containing folder, so the full relative path is different. Then you can easily examine both files and delete the one you don't need.
June 14, 201313 yr I'm not sure it's a problem writing to the drive. There may be something starved. i.E. If there is tremendous I/O going on, A smaller I/O has to wait. If you issue the top command you can monitor the processes. look at the %WA colum, if that is very high then you have allot of data waiting to be flushed. While I have done preclears on live machines. They were almost always limited in scope. i.e. separated by controllers. So I have no experience to help regarding preclearing on a controller that has parity and data alongside it.
June 14, 201313 yr Author Log is attached.No relevant thoughts on the speed, but you need to do 2 things relatively quickly. Remove the attached syslog from your first post, and change your email password. Then before you update your notification email settings in simplefeatures, turn off the extended logging, or whatever it is that puts your email and password in clear text in the logs. Thanks for the heads up. I actually caught the email inclusion shortly after I made the post and replaced the attached file, but, like a moron, I forgot all about the password. Fixed now. Thanks again!
June 14, 201313 yr Author WeeboTech and BobPhoenix - You guys don't preclear on your production unRAID. What do you do it on? Is it just a second unRIAD machine or VM, or do you do it in a separate OS?
June 14, 201313 yr WeeboTech and BobPhoenix - You guys don't preclear on your production unRAID. What do you do it on? Is it just a second unRIAD machine or VM, or do you do it in a separate OS? I've done it on a separate machine over the week. In fact I did 4 at a time. But i did not access unRAID. I imagine you could do it in a separate VM. When I had my large server, I was able to do the preclear on the production server. That was running 4.7 but I also had 4 different controllers. I had one controller just for the parity drive. 2 Supermicros for the data 6 ports on the motherboard 2 drives on the areca arc-1200 In my case, you aren't comparing apples and apples.
June 14, 201313 yr WeeboTech and BobPhoenix - You guys don't preclear on your production unRAID. What do you do it on? Is it just a second unRIAD machine or VM, or do you do it in a separate OS? I run the free version of unRAID on a separate PC in my office/2nd bedroom to do my preclears. My unRAID servers are in the basement.
June 15, 201313 yr Check out this thread. http://lime-technology.com/forum/index.php?topic=27942.0 perhaps using this kernel tuning will leave some memory available for other tasks. echo 65536 > /proc/sys/vm/min_free_kbytes
June 16, 201313 yr There was a suggestion for upping the value to 128. The person said his plugins were failing with out of memory until he put this value in. Since memory is pretty cheap these days, you can try this one out if want. sysctl vm.min_free_kbytes=131072
Archived
This topic is now archived and is closed to further replies.