June 2, 201511 yr I saw this reported by someone in some threads... BUT never (I think...) in "formal way"... unRAID OS Version: 6.0 RC3 - WebGUI v.2015.05.19 (still present in RC4 - solved in RC5?) NO DOCKER installed. Description: Sometimes Often, changing rapidly between WebGUI tabs (more observed between Dahboard & Main tabs... but not only), the WebGUI don't respond anymore and we get an "Unable to load page error" from web browser. How to reproduce: Change rapidly between WebGUI tabs back and forth as if you were (e.g.) searching something. Expected results: It should show the requested web page flawlessly. Actual results: The WebGUI don't respond anymore and we get an "Unable to load page error" from web browser. Top reports emhttp using about 98-99% of CPU usage and the WebGUI doesn't respond anymore. Other information: It happens even with low to nothing load on unRAID server (e.g. array stopped and no background operations). The hardware is ok (well tested and fast enough) and we have this happened on different hardware and on Virtualbox guest too. Stock "powerdown" command is unable to close the system too, requiring an hard reset if issued. Powerdown package plugin v.2.16 is instead able to shutdown. If emhttp process is KILLed and restarted sometimes all will go fine again, often not, giving segmentation fault in syslog.. If needed I'll try to attach a syslog the next time it happens. (Added!) EDIT: this evening I wasn't able to restart either by kill... root@unRAID1:/mnt# kill 3403 root@unRAID1:/mnt# emhttp & [1] 5127 root@unRAID1:/mnt# kill 5127 bash: kill: (5127) - No such process [1]+ Segmentation fault emhttp root@unRAID1:/mnt# emhttp & [1] 5730 root@unRAID1:/mnt# kill 5730 bash: kill: (5730) - No such process [1]+ Segmentation fault emhttp root@unRAID1:/mnt# kill 5730 bash: kill: (5730) - No such process root@unRAID1:/mnt# emhttp & [1] 6148 root@unRAID1:/mnt# syslog-2015-06-02.zip
June 4, 201511 yr emhttp is designed to be non-startable following a kill. You must reboot if emhttp crashes or is killed.
June 4, 201511 yr Author emhttp is designed to be non-startable following a kill. You must reboot if emhttp crashes or is killed. Ok, but don't take it easy... it's surely the worst bug of RC3-4 so far... If I play a bit with the WebGUI I should restart unRAID every about half an hour... It seems - but I'm still not sure... - that this happens mostly when browsing from local network on Gigabit LAN and only sometimes/never when from a remote location by vpn... This is a (bad) behaviour I've already encountered on other web servers in the past, apparently related to the "burst" of data that Gigabit LAN permits and that they wasn't able to manage flawlessly mostly on page change/load... If this will be the case, probably a "throttling" on data sending could solve this issue... I don't know if emhttp is configurable anyway. If you can give me some directions I could investigating and extensive testing on this. I'll anyway report every other impression/behaviour I would get on this issue.
June 4, 201511 yr Author Searching the forum I've seen others have encountered issues with emhttp since 5.0 beta, sometimes related to different browser/antivirus. I'll test different combinations as well, but in an "ideal world" a web client should never able to crash a web server...
June 4, 201511 yr Searching the forum I've seen others have encountered issues with emhttp since 5.0 beta, sometimes related to different browser/antivirus. I'll test different combinations as well, but in an "ideal world" a web client should never able to crash a web server... Could you post up a complete list of your hardware including the Network card (If you are using an off-brand card, try to find the name of the manufacturer of the chip set) and/or the on-board NIC of your server? (If you are using the on-board NIC, it should be easier for you to find this in the Manual than for us.) Have you rebooted the router and switches on your network to see if that action results in any changes? Do you have any plugins installled? List if you do.
June 4, 201511 yr My personal experience with all recent v6 versions is that the webGUI is more responsive than it ever was in v5, and it never crashes, unlike v5 and earlier. No idea what might cause the difference in our experiences.
June 4, 201511 yr My personal experience with all recent v6 versions is that the webGUI is more responsive than it ever was in v5, and it never crashes, unlike v5 and earlier. No idea what might cause the difference in our experiences. We have never seen any such thing as enetec describes. Here lies the frustration with tech support - 99% of the time if we can't reproduce it we can't fix it.
June 4, 201511 yr How to reproduce: Change rapidly between WebGUI tabs back and forth as if you were (e.g.) searching something. I did very rapid page retrieval and can NOT reproduce this problem in rc4. emhttp stays solid while getting requests fired at it. Do you have plugins, dockers or VMs running as well ? Did you scan your syslog for 'oom' (out-of-memory) messages ?
June 4, 201511 yr bonienl ----- I did the scan for you on the syslog that the OP posted in his first post. No oom errors in that one. Frank NOTE to enetec: Go to the 'Tool's page. Click on the 'Diagnostics' icon, download the diagnostics file and attached it to a new post.
June 4, 201511 yr Author My personal experience with all recent v6 versions is that the webGUI is more responsive than it ever was in v5, and it never crashes, unlike v5 and earlier. No idea what might cause the difference in our experiences. We have never seen any such thing as enetec describes. Here lies the frustration with tech support - 99% of the time if we can't reproduce it we can't fix it. I know very well your 'sentiment'... I've produced and repaired PC for over 15 years, now I'm an IT manager who have to mantain over 400 PC and about 8/10 different servers (AD DCs, storage servers, web servers, etc..) installed on 8 different LANs, all interconnected by VPNs and mostly managed by Active Directory. You can imagine the numbers of different issues I have to face every single day! Think that I've found this forum (and unRAID system which I didn't know... bad for me! ) searching for a solution for an issue with my LSI 9240-4i not compatible with the mainboard of my primary workstation (fixed crossflashing it to 9211-8i IR firmware after tens of useless tickets with LSI now Avago Tech... But this is another story! ) Said that, in less than a month from discovering unRAID at all, it's about three days I've my first unRAID server up & running and it will go production in a week or two (mainly for secondary backups storage). The hardware is all brand new and personally chosen, assembled and checked/burned in. It's ok, you can trust me... It's based on an Asus A88XM-Plus mainboard (8 well working SATA ports on board ), a low power dual core processor and 4 GB of perfect working RAM (extensively tested). It's connetted to LAN by onboard Gigabit LAN, one widely used Realtek chipset based NIC, which I've used flawlessly in a tens of machines and connected to Zyxel Gigabit switches with cat.6 cables. It works fine and I was able to fine tuning it up to 6K jumbo frames (it can go over but with minor overall performance... and, before you ask: the issue is present even with standard 1,5K MTU), with performance measured on my unRAID server of sustained rates of about 48 MB/s writing (with parity and no cache) and 68 MB/s reading. This using 2,5" consumer hard drives... good values I think. System is completely stable and hard drives are all new and with perfect Smart values after preclear and other tests. First parity building started @118 MB/s and ended flawlessly in about 8 hours and no error. This to say that... Yes, I'm very happy to have chosen unRAID for my needs and bought a Plus licence. BUT... there is this issue. Browsing from LAN (gigabit as already said...), Dynamix WebGUI on RC3 & 4 - the only I've tested - often becomes unresponsive during page load, mainly between the first two tabs (but it could be a coincidence...), emhttp CPU usage go to 100% (from Top command) and only sometimes I'm able to kill it and restart without a 'segmentation fault' error. Please note that in this condition, unRAID server remains working on shares and I've not found any related error in syslog. More, unMENU remains fully working with only images now shown (I think it uses emhttp only for those or something...). If this had been an hardware issue, even unRAID would hang sometimes... But this never happens. I've tested different browsers (IE, Chrome, etc.) and disabled Avast too, with same results. Until now, If I browse the gui from remote access (outside the LAN, by VPN), I've never seen this issue. I'll eventually update on this... This is making me to believe that the issue is related to 'data burst' on gigabit lan, since I saw in past various web servers having this type of issues when firstly gigabit lans came out. Web servers have been originally developed with (old) internet speeds in mind, and only more recently widely used in intranet environments. When LANs went to gigabit some had gone wild.... I don't know emhttp, never seen before, so I don't know for sure if this is the case... But I suspect in that direction for now. Surely until I will not see WebGUI hanging from remote...! Interesting is the 100% CPU usage... Something hangs inside. Is it waiting for... what? More info: I've no docker or VM installed. RAM usage is ever very low compared with 4GB available. Plugins: only few installed. unMENU, powerdown 2.16 (a note about it: this is able to powerdown the server even with emhttp hanged, stock one wasn't capable of it), Nerd tools (only CLI commands... Not related) and, since two days Dynamix temperatures plugin, which is unrelated too since I had this issue since before installing it. Only a question now: even if it surely can't be a solution, is there a simple way to make NIC to connect at 100 Mbit/s in unraid? by ifconfig? I would like to test it this way too... Attached what you have requested (please note, no hang in this since boot). unraid1-diagnostics-20150604-2222.zip
June 4, 201511 yr Interesting!!! I had(have) a problem with RealTek NIC that tis discussed in this thread: https://lime-technology.com/forum/index.php?topic=39350.0 You will note that it was 'solved' in the end by a installing a inexpensive Intel Network Card. However, what is extremely interesting is that with the Intel card installed, I have an almost limited number of dropped incoming packets. My research indicates that while dropped packets are not an issue in themselves, they are an indicator of other issues. (In thinking about it, I believe that the Intel card is much better at handling unusual high numbers of data request packets and simply drops the duplicates rather than sending them onto the OS to handle.)
June 4, 201511 yr It's connetted to LAN by onboard Gigabit LAN, one widely used Realtek chipset based NIC, which I've used flawlessly in a tens of machines and connected to Zyxel Gigabit switches with cat.6 cables. "flawless" or not, Realtek has been junk over the years with linux. Realtek refuses for some reason to maintain the in-kernel driver. They have their own driver which historically lags behind kernel releases and sometimes won't build with the latest kernel. Over the years we've replaced the Realtek in-kernel driver with the vendor driver (if it builds), result being some users report success, and other report failure. It's been a real PITA.
June 4, 201511 yr Author Never said that Realtek NIC are the best of the bounch (to be clear: they surely don't...!) but knowing how things work, it's hard to me to believe that a NIC capable of trasferring TB of data from and to the unRAID server at high speeds (45-68 MB/s as I said...) with no issue at all is then the 'culprit' of making emhttp go mad. And, please note, only Dynamix WebGUI and NOT unMENU too! Read this extensively: for NIC I intend the hw+driver combination. Anyway, it's not a problem for me to install a second NIC (Zyxel... well known and loved in Linux world too) and test with it. Or to install a 5$ 100 Mbit/s unbranded NIC and use it only to access web interface.... BUT I think this is the wrong direction. When an issue is reported, using the LSI approach of 'it's your (other) hardware to have issue', is the wrong approach, more when after reflashing their faulty firmware card with another, showed at last that... as I ever said, it simply was a firmware problem. Sometimes my users report issues that seems strange to me or that I am not able to reproduce at first... but I've learned - the hard way - that every report deserve the correct attention. More if it comes from a trained user. Even if it were the Realtek NIC to have 'issues' (but I don't think...), they are so widely used that it would be anyway up to who write an OS to manage them. Anyway, said that, it's not my intention to create 'a case' on this. unRAID is a good product and I would like to collaborate in testing beta & RC to have a flawless final release. If I can test something setting different parameters in emhttp (I don't know it, so I don't know where to operate), I would be happy to try. A good starting point could be undestand what emhttp is doing/waiting for when it goes to 100% CPU usage... Otherwise, no prob... I've resolved issues well worst of this... a solution, in a way or another (with two NICs, with a proxy, etc.) I'll be able to find it. BUT it couldn't be the same for others... and there are a LOT of Realtek NICs outside...
June 4, 201511 yr I'm sure most of us who don't experience this problem also have gigabit LAN so I don't think that is enough to explain it.
June 4, 201511 yr Author Interesting!!! I had(have) a problem with RealTek NIC that tis discussed in this thread: https://lime-technology.com/forum/index.php?topic=39350.0 You will note that it was 'solved' in the end by a installing a inexpensive Intel Network Card. However, what is extremely interesting is that with the Intel card installed, I have an almost limited number of dropped incoming packets. My research indicates that while dropped packets are not an issue in themselves, they are an indicator of other issues. (In thinking about it, I believe that the Intel card is much better at handling unusual high numbers of data request packets and simply drops the duplicates rather than sending them onto the OS to handle.) I've read your thread and I'm not surprised at all. Realtek are cheap cards and often shows a lot of peaks during high speed rate traffic. This, if could not be a big issue on data movement, it's probably an issue when the traffic is real time streaming. On Windows environments often is possible to reduce this behaviour by increase send & receive buffers and fine tuning jumbo frame. In unRAID it seems that network.cfg doesn't permit a lot of customization and probably you have to go by ifconfig to obtain something more... But I'm not enough trained on it to say how much. I've already asked if there is a simple way to force 100 Mbit/s connections e.g., but no reply... BUT I really think that my issue is totally unrelated to this: web traffic is not real time streaming and not high data movement neither. More... I don't think dumb Realtek NICs are instead so 'smart' to go mad emhttp only and not, e.g. unMENU! Web traffic the first, web traffic the latter. Same NIC. The difference? Tell me.... I repeat... one good web server should never crash since of a (bad) client, a bad medium or... an antivirus! emhttp has been instead already reported in past to go mad with some of these. Next time it will hang, I'll try to leave it alone to see if timeout at last (some reported of minutes of 100% cpu use in past), or go out with some errors on syslog...
June 4, 201511 yr Author I'm sure most of us who don't experience this problem also have gigabit LAN so I don't think that is enough to explain it. Never said 'it is enough'. Mine is only a suspect based on old bad experiences and the fact that, until now, I had no hang by remote. Surely is more plausible than saying "it's all because of Realtek"! Moreover... consider that outside there are a LOT of bad configured Gigabit LANs (poor switches, cheap cabling, rookie configurations...) that in reality go with speed of 30 MB/s or so... mine, where the un RAID server has been installed, has been tested at sustained transfer of 110 MB/s server to server... so high speed spikes of web traffic ar possible. And what's the moment of max traffic? During page change...
June 5, 201511 yr I have had intermittent problems with the gui locking up. But, it happens so rarely, that I can never reproduce it. I had more issues with v4 and v5 of unRaid than I have with v6. Like Tom says, if you can reproduce it, and he can reproduce it, he should be able to fix it. If you can and he can't, there is something wonky going on. I would suspect the Realtec NIC as they have proven for whatever reason to be a challenge for unRaid. Tom supports with a best efforts basis, but they are always one of the first things the community points to if the server is acting irresponsibly. My first 2 servers used Realtec NIC's but I don't use them any more. I have heard them referred to as the WinModems of this decade. Other things that are known weak areas in our servers are power supplies and disk cabling. However, your symptoms don't point in that direction.
June 5, 201511 yr BUT I think this is the wrong direction. When an issue is reported, using the LSI approach of 'it's your (other) hardware to have issue', is the wrong approach, more when after reflashing their faulty firmware card with another, showed at last that... as I ever said, it simply was a firmware problem. I'm not saying this issue isn't important and I'm not saying "oh it's just the fault of the h/w". Problem is you are the only one reporting this issue, we cannot reproduce, and I suggested perhaps Realtek NIC was a factor because we've had other issues with those NIC's. It's just stab in the dark man.
June 5, 201511 yr BUT I think this is the wrong direction. When an issue is reported, using the LSI approach of 'it's your (other) hardware to have issue', is the wrong approach, more when after reflashing their faulty firmware card with another, showed at last that... as I ever said, it simply was a firmware problem. I'm not saying this issue isn't important and I'm not saying "oh it's just the fault of the h/w". Problem is you are the only one reporting this issue, we cannot reproduce, and I suggested perhaps Realtek NIC was a factor because we've had other issues with those NIC's. It's just stab in the dark man. Well there's that and all the extra software that's being loaded on his system (e.g. unMenu). Those modifications aren't officially supported and could definitely be adding to the instability. @enetec: see if you can recreate the issue on a stock build (no plugins and no custom scripts added). If not, chances are the issue with emhttp hanging is plugin-induced.
June 5, 201511 yr Author I have had intermittent problems with the gui locking up. But, it happens so rarely, that I can never reproduce it. I had more issues with v4 and v5 of unRaid than I have with v6. Like Tom says, if you can reproduce it, and he can reproduce it, he should be able to fix it. If you can and he can't, there is something wonky going on. ... I'm sorry but this is a too 'simpleristic' approach to beta testing: if my prog works fine on my PC it's surely ok. If things worked this way, there wouldn't need of beta testing too: my prog works fine on my PC! But a good beta testing is needed since we have to cover quite the 99,99% of good working hardware beneath well different in configurations and performance. Realtek are cheap cards? Surely! Realtek are bad working hardware? Maybe. Realtek are the cause of WebGUI hangs? I don't think. And I have already exposed why (read: unMENU *ever* working and high speed transfers with no issue at all...) Realtek are mothers of all odds on unRAID? Well, simply remove it's driver and support from kernel. BUT if supported... they should work. And don't forget that there are Realtek and Realtek: we are not talking of a 2$ chinese card with a fake CE sticker on it, but an onboard NIC on decent Asus mainboard. On the other hand, with a short search on forum I got: Docker hangs? -> WebGUI hangs Xyz brand antivirus while browsing? -> WebGUI hangs Low in memory? -> WebGUI hangs Have a Realtek NIC -> WebGUI hangs (as all of you continue to say...) What's the common part in all of these? Yes... that the WebGUI hangs. Anyway, I repeat myself, I'll surely find a good solution suitable for me and surely I'll try different NICs (and when it will hangs even with those? ) I'm reporting this because IMHO there is an issue with emhttp (a module which goes 100% in CPU usage is not usually Linux stuff... is more a Windows behaviour...) and for seeking help from who knows it better than me to help me to isolate the problem and because of it. But if this has to become a 'Realtek affair thread', I'm no more interested on it. I'll patch it myself at my best and for the others... Well... Just remember, it could be sunny or rainy... But WebGUI - if it wanna - will hang anyway!
June 5, 201511 yr Author ... @enetec: see if you can recreate the issue on a stock build (no plugins and no custom scripts added). If not, chances are the issue with emhttp hanging is plugin-induced. Already tested. I've another two keys with Trial licence on it and one of these is a plain 'vanilla' RC3. Same behaviour (and with stock powerdown I *must* kill emhttp before invoking it otherwise it hangs too and no powerdown at all + parity checking starting). I'll test them on different hardware too. I have many... lucky me! And *surely* I'll test other NICs too! On Virtualbox instead I don't remember any hang. But I've that issue with Fuse and user sharing not working, so I don't use it anymore for advanced testing... Anyway, don't forget this: I don't remember a single hang on browsing by remote... Maybe in five minutes it will happen, but until... I ask it again: are there any emhttp/WebGUI performance tunable settings I can edit?
June 5, 201511 yr BUT I think this is the wrong direction. When an issue is reported, using the LSI approach of 'it's your (other) hardware to have issue', is the wrong approach, more when after reflashing their faulty firmware card with another, showed at last that... as I ever said, it simply was a firmware problem. I'm not saying this issue isn't important and I'm not saying "oh it's just the fault of the h/w". Problem is you are the only one reporting this issue, we cannot reproduce, and I suggested perhaps Realtek NIC was a factor because we've had other issues with those NIC's. It's just stab in the dark man. Well there's that and all the extra software that's being loaded on his system (e.g. unMenu). Those modifications aren't officially supported and could definitely be adding to the instability. @enetec: see if you can recreate the issue on a stock build (no plugins and no custom scripts added). If not, chances are the issue with emhttp hanging is plugin-induced. Looking at his posting history, he may be running unRAID in a VM.
June 5, 201511 yr Ah, another potential gotcha. In short reply to enetec, there are no tunable settings for emhttp. If you are running in a VM, please try running unraid as a host, not a guest VM.
June 5, 201511 yr BUT... there is this issue. Browsing from LAN (gigabit as already said...), Dynamix WebGUI on RC3 & 4 - the only I've tested - often becomes unresponsive during page load, mainly between the first two tabs (but it could be a coincidence...), emhttp CPU usage go to 100% (from Top command) and only sometimes I'm able to kill it and restart without a 'segmentation fault' error. With first two tabs you mean "Dashboard" and "Main" ? Interesting is the 100% CPU usage... Something hangs inside. Is it waiting for... what? When this happens can you list all running processes: ps -ef And communication state: lsof -Pni
Archived
This topic is now archived and is closed to further replies.