speedkills

Looks like you guys pointed me in the right direction with the disk issues. After running more tests read errors popped up on a different disk. Seems possible but unlikely that this is different disks having issues within days, so I am going to investigate the controller next. I am heading out of town on vacation for a couple of weeks but I will report back when I am back in town just to make sure this isn't left an open issue. I did just switch to running my disks through an internal to external sas adapter (sff8087 to sff8088) and that change happened about the same time this started so that is my leadng suspect right now.

Thanks. I am going to go ahead and remove Disk1 from the equation even though it doesn't show errors anymore. Can't hurt and won't be any trouble. I'll report back what I find after tests are run, thanks again for the link.

Updated syslog from today, no more read errors, performance is still no good and after an hour the webgui went unresponsive again. Seems like the next step is to uninstall the simplefeatures plugins and see what happens. syslog.zip

I saw the read errors on that drive during that dump of syslog. The performance issues started before then, and there have been no other read errors shown any other time (performance is still about 1MB/s today as well with no read errors). I double checked my cables and haven't seen the issue crop up again, all smart tests show ok. Performance is still bad. Treat me like I am an idiot, I think you have it in you, and break it down for me if you don't mind. It can help us both at the same time, you can feel superior and I can get a hand figuring this out. Basics for me, which may seem stupid compared to someone elses basics. 1. Performance is bad. 2. Check some cables. 3. Reboot a few times. 4. Run smart tests, all ok.. 5. See some errors on a disk. 6. Shut down, swap sata cable on that disk. 7. Start up, no error for a day, clean smart test, performance is still bad. Should I consider this permanent and not transient? Is the disk garbage even if it says it is ok now and doesn't continue to show errors? Should I just remove the disk from the array for now and verify it is not the source of the issue?

Just did one more test since my parity was already untrustworthy at this point, I removed my parity drive in the webgui, and now I am back up to normal speeds, reads and writes of about 50MB/s. I am out of time to play with this more tonight but tomorrow I will re-enable the parity drive and see if that tanks my read and write speeds again. Another positive, I was able to get through the new permissions script this time, I tried it right before disabling my parity and ended up with an out of memory error. Ran through no problems, and much faster with parity disabled. Obviously not a long term fix but at least this tells me there isn't something fundamentally wrong with reads and writes on the system, it definitely seems to be parity related but hopefully I can find more confirmation tomorrow.

Just an update to a question I forgot to answer previously, yes the new permissions script has been run. I am running it again right now just in case. I did find I can run it directly from the command line, it is at usr/local/sbin/newperms.

You set your selfup for a world of hurt. I say this because you know a bit but not enough and went down a very complex path. First and for most, did you apply the VMWare U2 patch for esx5.1? How is your unRAID guest VM setup (need to be very specific here)? How are your unRAID Disk and Shares setup (SMB/AFP settings secure/public, etc for each)? Did you run the new permissions script before anything else when you upgrade to RC11? ... Many more questions, but this would be a start to lead you in the right direction. Yes, I am on U2 (build 914609). Thanks for the tip to recheck my shares, I was thrown off by the fact that my video share still existed and that it has a few but not all files accessible through it that when I upgraded unraid a couple of days ago my user shares actually got turned off. I completely missed they were turned off, because one of them still existed. You are correct, I am in a bit over my head knowing how to turn things on and off but not knowing enough to know that when user shares are turned off some of them will still be available, sorta, sometimes, partially. Luckily there are helpful people like you here who know enough to explain why that is probably perfectly normal and I just didn't understand. Now that shares are turned on again they are fully available, not just sorta available. One problem solved so that is progress.

Err... you are aware that once mover has moved files from the cache drive to the data drives, the whole directory tree is removed from the cache drive? The cache drive is temporary storage for the cached user shares - the directory trees are not permanently held on the cache drive. Yep, I am aware of how the move script moves, but still don't understand what that would have to do with the behavior I am seeing. Say Drive 1 has items A and B Drive 2 has items C and D Drive 3 has items E and F. and my share that should show all of them has only item C. What does that have to do with mover? Honest question, I am missing the connection.

Stranger still, I was wrong, the video share that is showing is only showing one folder of the two folders on my cache drive. There are no drives on my system that only show the one folder, so why would the video share be the only share of my shares that is showing up, and why would it only show one folder of the many folders in that share?

Just throwing the link up in case anyone else didn't finds this thread and doesn't know how to dump the syslog without the web gui. Instructions are up at http://lime-technology.com/wiki/index.php/Troubleshooting#Capturing_your_syslog. The gist of it is use cp /var/log/syslog /boot to dump the syslog to a file on the root of your unraid flash drive.

Syslog was too large to attach here, but this link to it on dropbox should work. https://dl.dropbox.com/u/17526/syslog

Phew.....what a day with unRAID. Now that it is back online....ish and upgraded to 5.0 RC11 the main web ui is intermittent but mostly unresponsive. User shares are....odd, not quite offline, not quite online. Mostly offline though. As an example, I have a few user shares, photo, music, video, etc. All of my disk shares are online and accessible. All of my user shares are missing when browsed from windows or osx except my video share, but it only has one directory it in, (I believe the directory from my cache drive), all of the files and directories are inaccessible though. I am unable to get to the user shares web gui to try to investigate further. I did run top from the console and nothing is taking more than 2% cpu so I can't tell why it is being unresponsive. Any ideas on what to do next? Can I run the reset permissions script from the command line? I looked for it on my flash share but couldn't identify it. I am trying to avoid another dirty shutdown but if the webgui is just unresponsive (by the way, it this common for others, it has been an issue for over two years for me, just completely unreliable webgui even though I have tried stock, unmenu, simplefeatures, nothing seems reliable). I am still reading up trying to figure out how to get a syslog with the webgui unresponsive but will follow up when I can get it. I am running on esxi 5.1 which I believe should obscure my actual motherboard from unraid and masquerade it as an intel 440BX but in case I am wrong about that the actual motherboard is a SuperMicro X9DRL-3F-O.

No luck. I updated to RC11 (itself a pain in the ass. I wonder if by mid-2015, sometime around 5.0 RC28 if unraid will ever be able to unmount it's shares successfully?), then tried the 4GB limit option, and directly typing in the sysctl vm.highmem_is_dirtyable=1 and am still getting horrible performance. My parity check is humming along at 1MB/s and is scheduled to finish in three weeks. My fiancee just heard me grumbling "....grumble grumble, crappiest software I have ever spent $140 dollars on...grumble grumble" and was immediately like "What?!? With the amount of trouble you have every single time you touch unRAID I always just assumed it was one of those unsupported open source projects." Oh well, another day working on unraid, another unsatisfactory result. I'll keep banging away at it but man does this project leave a bad taste in my mouth every time I work with it. After being unable to do a simple shutdown with my last build I went out of my way to make sure I picked a well supported configuration this time and spent the extra bucks on a supermicro board, even went to the trouble of installing it in esxi so I would have a hardware abstraction layer to see if that helped since a few people are going that route but once again I still can't make unraid do something as simple as shut down by pressing the unmount button. A clean unmount seems really important to this OS, so you would think that would be something it would be able to do, and I am sure there are many people it works great for, but never for me. Whole new server, version, motherboard, everything....exact same problems. But now slower. Awesome. I will keep an eye on the other forum to see if this is a related problem but it seems like the fact that this isn't helping may indicate I have my own unique issue not related to the problem so many other people are seeing related to memory over 4GB.

I have not, I will try upgrading in the next couple of days. Figured I would see if this was a known issue before changing things further but I was hoping to get to the latest version anyway to have the vmware tools available.

I have been troubleshooting a strange performance issue with unraid that has cropped up recently after I upgraded to unraid 5.0 rc8 (from 4.7) and moved to a virtual machine in esxi server. Performance is abysmal on user shares but normal to the cache drive. I am seeing about 1MB/s reads and writes to user shares, and it appears to correlate with the main web server as well. As an example I can read and write a file to the cache drive at over 40MB/s, but if I try to read or write a file to my video share I get around 1MB/s. I just noticed that while I am trying to write a file to the video share not only is the write very slow but the main web interface and user share browsing is also very slow (intermittently timing out from multiple machines in my house) but the web page for sickbeard and sabnzbd which I have installed using the plugins posted to this forum respond normally. I am also routinely seeing the shfs process using a large amount of cpu, 40% or more. Is this a known issue or can anyone point me to next steps for troubleshooting?

Problem solved. I talked to SuperMicro support and they said that sometimes the ipmi gets in a bad state and needs to be reset. I booted off of a usb key with freedos and ran ipmicfg.exe -fd to reset the ipmi to factory defaults and now I have cpu temps.

Has anyone seen this? I did a bit of googling around and couldn't find anything. I got the motherboard as an open box from NewEgg and now I am wondering if I have a bad motherboard. I just updated the IPMI and BIOS firmware, both were out of date but updating them hasn't changed anything. I am only running one cpu right now but the cpu temp shows not present. It ends up being annoying because everytime I do much of anything on the server the motherboard beeps and the fans speed up for a couple of minutes, along with the board logging a cpu over temp error. How could the cpu be over temp when there is no temperature sensor working? I do show system temp and peripheral temp but no cpu or pch temp (both say "not present" in IPMI). Any ideas? How would I know if this is a bad motherboard or cpu?

Nice, thanks for the link. Documentation is always appreciated.

Wow, thank you Johnm. I would have never guessed in a million years that the add user was only tested with lowercase. Tried adding three different people doing the dumb thing and adding them as Firstname, or Firstname Lastname. No error message, no instructions anywhere that the add user had failed or that users can only be identified in all lowercase. Just setup a couple of users as firstname (all lowercase) and it worked no problem.

I'll take it. Does this card need any modification to run in straight jbod mode for unraid?

The biggest, by far is the lack of an available web gui. The unraid gui won't do much, it won't even shut my unraid box down if there are any plugins installed, which there are. I'm still surprised it couldn't just have written a process to shut down third party apps and then shut the system down. Since there is so little to the unraid web gui I rely on unmenu, which is just really crashy in my experience. Nearly every time I log into it (which is always after a month or two of ignoring it) the unmenu gui has gone offline. I actually just rebooted my unraid box last night and today the unmenu gui doesn't work at all. I end up with the all too frequent e </BODY></HTML> 0 So that would probably be my biggest question, does unraid 5 have a native web gui that is stable or is it still tacked on my someone else? Screw it, I'm just going to upgrade and see if the new version looks better. I'm looking at it now and actually have the evening free for one. Wish me luck!

That sounds promising, I was actually about to tear down my unraid setup and rebuild it with something else but maybe I should give 5 a try first. Have you had better luck with it? It is true that it is sold as a storage device only, I think a lot of the criticism comes from people saying what it "can" do, not what it should do. If I bought was asking friends for advice on vehicles to buy and asked, can I haul a full size mattress with a Toyota Camry they would probably answer, well you could, but it would be a harrowing experience and you would probably regret it. When I was researching unraid initially and someone asked "can it run X" the answer was always, sure, no problem, it's great, I do it all of the time. Had the answer been "yes, it has been done, but it is frought with peril, your system may become unstable, and it wasn't really designed for that" I probably would have either skipped unraid, or used it as storage only, using my mac mini for all server functions and been left with a much better view of it. For storage only I think it is a great system, so for prospective customers I guess you need to ask yourself is storage alone enough these days? Do you really want to run one nas and one downloader? One nas and one streamer? Even nas boxes running arm these days will download and serve media for you, there isn't a huge market for a box that only serves files these days, but does not help you manage them (and creates duplicates but gives you no easy method to clean them up). I was really attracted to the idea of spinning my drives down to save power but realistically given the extra time invested vs my previous solution that did not spin the drives down this only saved me money if my time was worth about 50 cents an hour. All comments are on version 4.7, 5.0 may be an entirely different ball of wax, in which case ignore my comments. is 5.0 even an official release yet, or is 4.7 still the official stable release?

I agree, and that is a good reason why people should be wary of unRAID. I have been using it over a year now, in large part because I have not yet found better options but it is time for me to take another look at flexraid and snapraid. In my experience, you will definitely need to get use to the CLI as the unraid GUI is really unstable. Every time I have an issue, the web interface is useless. The default interface can't close any files, so if you use it for anything, and I mean anything besides a file server files will be locked and it is too dumb to know how to kill the process and close the files. Want to install any sort of streamer, downloader, ripper, transcoder, whatever, expect an error trying to shut down the system with it complaining that it files are locked. Unmenu is much, much better when it works, but every time my system acts up, unmenu has crashed. I just rebooted it tonight (from the cli, with unmenu crashed and the default web interface refusing to shutdown, natch) and right after a fresh reboot and booting into unmenu I find that any links I click in unmenu send me to a page with the super helpful e </BODY></HTML> 0 Which sadly half of the users here have probably seen before. When considering your options think twice about how much time you want to spend on this forum, the default answer I saw when I was investigating it was how helpful and active the forum was, what I didn't realize was that there is a reason the forum is so active, anyone who wants to get anything done has to come here and ask for help. Search around. You might be stunned to see the percentage of threads here asking simple things like how do I reboot, how do I remove duplicate files, how do I install a bit torrent client (or now that I have installed a bittorrent client, why can't I shutdown?) compared to other forums, these are things you wouldn't think to look for on a ubuntu or windows home server forum because you take them for granted, but after a year with unraid I don't take them for granted. In unraid's defense, I was able to add a drive and keep all of my old data. But the rub, as always with unraid, is how much effort it took, I shut the system down, added a drive, started it up and found it couldn't see any of my data, had lost all of my drive mappings and that I had to manually tell it which drive was which based on serial number. Don't worry, you can just post here, someone will walk you through how to do all of it, and you will be fixed up by tomorrow. Unless you wanted it to just work....today, then maybe unraid isn't your best choice. Just offering an alternative perspective, there are a lot of people here who visit the forums every day and will be very helpful with your questions and sing unraid's praises, but I find people who don't live to contribute code to this project might have something else to say about it. Coming here and asking for alternatives is kind of like going to a Mustang forum and asking what people think of the new Camaros.

I've been running unraid 4.7 for over a year now, never really liked it and I know I'm probably asking for trouble here but I am hoping to hear from someone not terribly impartial, maybe someone who runs ubuntu, windows, macs, isn't an OS fanboy, is the new version worth my time to upgrade to? Or to put it another way, if I don't like unRAID 4.7, is there any chance I will like 5 or is it more of the same?

That was very helpful, thank you. I had no idea I could export the cache drive like that.