Reoccuring issue

[Adrian]

I keep running into this issue with unraid. File transfers get really slow through both windows explorer and internally in docker apps like sick beard. This last time it took over an hour to move 3 gigs from a standalone SSD drive to the array. It gets unbearable and then I have to stop sick beard by stopping it in the docker UI. Then I have to stop the array which takes a ridiculous amount of time (about 5 minutes) and then finally I can restart the server. When it comes up (takes several minutes after ping starts to respond), everything seems to be back to normal.

 

Any ideas what's going on? Anyone else running into this issue? Are there any plugins/tools that I can run while this is happening to see what's going on? What drive it's writing to, what write speeds is it getting, etc. Is there a way to run a performance test for network and disk writes/reads for each drive? If it's a hardware issue, I don't mind upgrading. I just have no idea where to start to even see where things are crapping out.

 

Thank you,

Adrian

 

Unraid 6.2.4

[John_M]

The best place to start is to read this: https://lime-technology.com/forum/index.php?topic=39257.0

[Adrian]

Ok, so part of my reoccurring issue just started again where I can't copy any files. It starts writing and then it times out and unraid becomes unresponsive for several minutes. I finally got it to create the diagnostic file.

 

Basically the only way for me to fix it is to reboot the server.

 

Thanks,

Adrian

mediaserver-diagnostics-20161215-1841.zip

[JorgeB]

IRQ16 is being disabled crippling one or both your SAS2LP controllers.

 

Reboot and check what IRQ they are using by running:

 

 lspci -vv -d 1b4b:*

 

Try updating your bios/firmaware , if just one is on IRQ 16 try moving it to another slot if available.

[Adrian]

IRQ16 is being disabled crippling one or both your SAS2LP controllers.

 

Reboot and check what IRQ they are using by running:

 

 lspci -vv -d 1b4b:*

 

Try updating your bios/firmaware , if just one is on IRQ 16 try moving it to another slot if available.

 

 

88SE9485 is using IRQ 16

88SE9485 is using IRQ 17

88SE9172 is using IRQ 32

 

 

I'll look into updating the MB BIOS if there are any updates available. I'll have to see if I have any other slots available.

 

One thing I do notice is that when I start the computer I can't access the BIOS. I think I can if I remove one of the SAS2LP cards. Also, after the SATA cards each post, I see this really large blinking line. Almost like a really big cursor dash. I'll open it up today and document exactly how I have the cards and troubleshoot a bit more with their slots.

 

Thank you so much!

Adrian

[gubbgnutten]

Possibly related: You appear to have a bunch of nearly full ReiserFS drives.

 

If for example split levels cause writes to occur on one of those drives, the writes will no doubt be unbearably slow...

[Adrian]

Possibly related: You appear to have a bunch of nearly full ReiserFS drives.

 

If for example split levels cause writes to occur on one of those drives, the writes will no doubt be unbearably slow...

 

 

Well, I have all my shares set to most-free so I'm not really sure how or why it would ever try to write to any of the nearly full drives. I guess if I ever see slow writes, I would need to look at figure out which drive it's writing too and see what it's free space is.

[Adrian]

Well, I removed one of the SATA cards that had 4 drives and moved them to the onboard sata. So now I'm just using the onboard and the 2 SAS2LP controllers. Once I did that that weird issue with the large white blinking cursor went away and now it gets to the unraid booting faster. It's running a parity check now. So I'll see what happens with this new setup. Right now I'd be happy for it to work for another month or so to give me some time to research doing a major upgrade. Ideally I'd like to run off a Supermicro MB without any of the limitations with the PCI Express lanes that this Gigabyte motherboard has.

[Adrian]

So  I found some posts that describe my issue with talks about a bug related Sandybridge motherboards and additional ethernet cards (exactly what I have)

 

https://bugzilla.kernel.org/show_bug.cgi?id=38632

 

The last post mentions running a command to get the IRQ working again, but I'm don't know enough about linux to modify this to try on my  server.

 

I tried to run "rmmod e100;modprobe e100;sleep 0.1;ifconfig eth2 up" after error - it helped. IRQ works again:

 

I'm probably going to try again to get the server working without the additional ethernet card.

[Adrian]

IRQ16 is being disabled crippling one or both your SAS2LP controllers.

 

Reboot and check what IRQ they are using by running:

 

 lspci -vv -d 1b4b:*

 

Try updating your bios/firmaware , if just one is on IRQ 16 try moving it to another slot if available.

 

I just wanted to let you know that I was able to eliminate the IRQ 16 issue by removing the secondary Intel NIC. It took a bit, but I finally got the on-board NIC working. This MB really sucks, but at least I've bought me some time before I have to drop almost a grand on a replacement. So far I haven't had this crazy issue which would happen at least once or twice a week. I've written to drives that used to always cause the issue and so far everything is running really nice.

 

Thank you all!

Adrian

[Adrian]

ok, sooo I haven't exactly eliminated the issue. It's still keeps coming up.

 

I ran "cat /proc/interrupts" before int16 got disabled and this is what it listed.

 

IO-APIC  16-fasteoi   ehci_hcd:usb1, mvsas

 

So I'm thinking, maybe I need to disable that USB and find another one to boot off of? Would I need to disable it in the BIOS or would simply not plugging anything in be enough?

[Squid]

13 minutes ago, Adrian said:

ok, sooo I haven't exactly eliminated the issue. It's still keeps coming up.

 

I ran "cat /proc/interrupts" before int16 got disabled and this is what it listed.

 

IO-APIC  16-fasteoi   ehci_hcd:usb1, mvsas

 

So I'm thinking, maybe I need to disable that USB and find another one to boot off of? Would I need to disable it in the BIOS or would simply not plugging anything in be enough?

Or try moving your marvel controller to a different slot.  Most decent m/b manuals detail the shared IRQ's between slots and the USB controllers

[Adrian]

Can't move the controller, there aren't any other slots where they work. I'll go over the manual and see if I can find this info.

Edited March 1, 2017 by Adrian

[Adrian]

The motherboard is a Gigabyte GA-Z77X-UD3H

 

Link to manual: http://www.gigabyte.com/Motherboard/GA-Z77X-UD3H-rev-10#support-manual

 

I just read through it and I don't think I see any mention of which IRQ's are shared between slots and USB controllers. If you can, take a look and see if you find it or maybe you can see what options I might have available.

 

I see some options regarding USB on page 52. xHCI Pre-Boot, xHCI Mode, HS Port #1~#4, xHCI Streams. Not sure if those settings could help deal with this issue.

 

Thank you,

Adrian

 

 

Edited March 1, 2017 by Adrian