• [6.8.3] - NGINX - Out of shared memory

    Struck

    By Struck

      • Minor

    Hi

    I am in the process of moving data off my array onto a couple unassigned disks, so i can reformat my array to xfx encrypted (from regular xfx)

    When i got up this morning , i saw that the server log is filled with errors like:

    Quote

    Oct 22 11:18:10 Hotbox nginx: 2020/10/22 11:18:10 [crit] 5726#5726: ngx_slab_alloc() failed: no memory
    Oct 22 11:18:10 Hotbox nginx: 2020/10/22 11:18:10 [error] 5726#5726: shpool alloc failed
    Oct 22 11:18:10 Hotbox nginx: 2020/10/22 11:18:10 [error] 5726#5726: nchan: Out of shared memory while allocating channel /disks. Increase nchan_max_reserved_memory.
    Oct 22 11:18:10 Hotbox nginx: 2020/10/22 11:18:10 [error] 5726#5726: *3589872 nchan: error publishing message (HTTP status code 507), client: unix:, server: , request: "POST /pub/disks?buffer_length=1 HTTP/1.1", host: "localhost"
    Oct 22 11:18:10 Hotbox nginx: 2020/10/22 11:18:10 [crit] 5726#5726: ngx_slab_alloc() failed: no memory
    Oct 22 11:18:10 Hotbox nginx: 2020/10/22 11:18:10 [error] 5726#5726: shpool alloc failed
    Oct 22 11:18:10 Hotbox nginx: 2020/10/22 11:18:10 [error] 5726#5726: nchan: Out of shared memory while allocating channel /cpuload. Increase nchan_max_reserved_memory.
    Oct 22 11:18:10 Hotbox nginx: 2020/10/22 11:18:10 [error] 5726#5726: *3589877 nchan: error publishing message (HTTP status code 507), client: unix:, server: , request: "POST /pub/cpuload?buffer_length=1 HTTP/1.1", host: "localhost"
    Oct 22 11:18:10 Hotbox nginx: 2020/10/22 11:18:10 [crit] 5726#5726: ngx_slab_alloc() failed: no memory
    Oct 22 11:18:10 Hotbox nginx: 2020/10/22 11:18:10 [error] 5726#5726: shpool alloc failed
    Oct 22 11:18:10 Hotbox nginx: 2020/10/22 11:18:10 [error] 5726#5726: nchan: Out of shared memory while allocating channel /var. Increase nchan_max_reserved_memory.
    Oct 22 11:18:10 Hotbox nginx: 2020/10/22 11:18:10 [alert] 5726#5726: *3589878 header already sent while keepalive, client: 192.168.1.24, server: 0.0.0.0:443
    Oct 22 11:18:10 Hotbox kernel: nginx[5726]: segfault at 0 ip 0000000000000000 sp 00007ffc7c827cf8 error 14 in nginx[400000+21000]

    And much more of stuff like that. atm the logfile is 70% full and is still filling.

    The data migration is still ongoing in its normal pace, and is expected to finish in less than an hour from the time of writing.

     

    As far as i know ngnix is used for the web service, and might explain why the UI is slow and unresponsive. I was not able to download the diagnostics file with the UI, and used a SSH connection with Putty instead to get it generated, then downloaded it off the flash drive.

     

    The system uptime is less than 7 days, so wouldn't expect it to be the problem. But i would expect the problem to solve itself with a simple restart, but that might not be solving the problem in the longer term, (for me and other users)

     

    So basically this report is mostly used to help find the cause of the problem, since i think i know the quick solution ( reboot )


    For ease of use i can post my specs here:

    Intel Xeon E5-2680v4 

    Asrock X99 Extreme4

    4x16GB Corsair LPX 3200Mz@2400MHz QUAD channel.

    EVGA SuperNOVA 550W G3

    4x8TB WD Red (3+1)

    Samsung 850 EVO 1TB Cache

    Unassigned disks:

    2x8TB WD Red

    2x4TB WD Red

    300GB Intel SSD secondary

     

    hotbox-diagnostics-20201022-1133.zip

    Go to reports Stable Releases

    User Feedback

    Recommended Comments

    Struck

    As expected a restart solved the problem.

    But i had problems stopping the array, (which is normally not a problem)

    So had to powerdown from the terminal and pressing the power buttom - still a safe shutdown though.

    The problem seemed to be unmounting disk shares, of which i have none i think. 

    Link to comment
    itimpi
    6 hours ago, Struck said:

    The problem seemed to be unmounting disk shares, of which i have none i think

    It would have been unmounting disk drives (not shares) which is a standard step in stopping the array.

     

    Link to comment
    Struck
    1 hour ago, itimpi said:

    It would have been unmounting disk drives (not shares) which is a standard step in stopping the array.

     

    It could have been that. I can't remember.

    Link to comment


    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.

    ×

  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.

    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.