June 27, 201016 yr I've been using unRAID for a few weeks now, and today I decided to install some new stuff. But now everything I run seems to be killed off due to low memory. I have 4GB of real memory in the machine, and almost all of it seems to be used. The following is some outputs that might help: (from /usr/bin/free) total used free shared buffers cached Mem: 3886268 3773544 112724 0 7564 3700208 -/+ buffers/cache: 65772 3820496 Swap: 2097144 7460 2089684 top - 18:43:25 up 3:05, 1 user, load average: 1.71, 1.58, 1.53 Tasks: 101 total, 1 running, 96 sleeping, 0 stopped, 4 zombie Cpu(s): 1.2%us, 1.6%sy, 0.0%ni, 66.8%id, 28.8%wa, 0.7%hi, 0.8%si, 0.0%st Mem: 3886268k total, 3773484k used, 112784k free, 7468k buffers Swap: 2097144k total, 7460k used, 2089684k free, 3698508k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2710 root 20 0 14584 4088 3092 D 4 0.1 6:31.70 smbd 124 root 20 0 0 0 0 S 2 0.0 0:45.11 sync_supers 1773 root 20 0 48260 1324 856 S 2 0.0 5:41.37 shfs 1 root 20 0 700 288 264 S 0 0.0 0:01.87 init 2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd 3 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/0 4 root 20 0 0 0 0 S 0 0.0 0:00.28 ksoftirqd/0 5 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/1 6 root 20 0 0 0 0 S 0 0.0 0:00.15 ksoftirqd/1 7 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/2 8 root 20 0 0 0 0 S 0 0.0 0:00.51 ksoftirqd/2 9 root RT 0 0 0 0 S 0 0.0 0:00.01 migration/3 10 root 20 0 0 0 0 S 0 0.0 0:00.25 ksoftirqd/3 11 root 20 0 0 0 0 S 0 0.0 0:00.00 events/0 12 root 20 0 0 0 0 S 0 0.0 0:00.00 events/1 13 root 20 0 0 0 0 S 0 0.0 0:00.00 events/2 14 root 20 0 0 0 0 S 0 0.0 0:00.00 events/3 15 root 20 0 0 0 0 S 0 0.0 0:00.00 khelper 20 root 20 0 0 0 0 S 0 0.0 0:00.00 async/mgr 126 root 20 0 0 0 0 S 0 0.0 0:00.01 bdi-default 128 root 20 0 0 0 0 S 0 0.0 0:00.00 kblockd/0 129 root 20 0 0 0 0 S 0 0.0 0:00.00 kblockd/1 130 root 20 0 0 0 0 S 0 0.0 0:00.00 kblockd/2 131 root 20 0 0 0 0 S 0 0.0 0:00.00 kblockd/3 132 root 20 0 0 0 0 S 0 0.0 0:00.00 kacpid 133 root 20 0 0 0 0 S 0 0.0 0:00.00 kacpi_notify 134 root 20 0 0 0 0 S 0 0.0 0:00.00 kacpi_hotplug 259 root 20 0 0 0 0 S 0 0.0 0:00.00 ata/0 260 root 20 0 0 0 0 S 0 0.0 0:00.00 ata/1 261 root 20 0 0 0 0 S 0 0.0 0:00.00 ata/2 262 root 20 0 0 0 0 S 0 0.0 0:00.00 ata/3 263 root 20 0 0 0 0 S 0 0.0 0:00.00 ata_aux 267 root 20 0 0 0 0 S 0 0.0 0:00.00 ksuspend_usbd 272 root 20 0 0 0 0 S 0 0.0 0:00.00 khubd 275 root 20 0 0 0 0 S 0 0.0 0:00.00 kseriod 312 root 20 0 0 0 0 S 0 0.0 0:00.00 rpciod/0 313 root 20 0 0 0 0 S 0 0.0 0:00.00 rpciod/1 314 root 20 0 0 0 0 S 0 0.0 0:00.00 rpciod/2 315 root 20 0 0 0 0 S 0 0.0 0:00.00 rpciod/3 363 root 20 0 0 0 0 D 0 0.0 0:10.34 kswapd0 408 root 20 0 0 0 0 S 0 0.0 0:00.00 aio/0 409 root 20 0 0 0 0 S 0 0.0 0:00.00 aio/1 410 root 20 0 0 0 0 S 0 0.0 0:00.00 aio/2 411 root 20 0 0 0 0 S 0 0.0 0:00.00 aio/3 417 root 20 0 0 0 0 S 0 0.0 0:00.00 nfsiod 422 root 15 -5 0 0 0 S 0 0.0 0:00.00 kslowd000 423 root 15 -5 0 0 0 S 0 0.0 0:00.00 kslowd001 691 root 20 0 0 0 0 S 0 0.0 0:00.00 usbhid_resumer 730 root 16 -4 1892 576 480 S 0 0.0 0:00.13 udevd 834 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_0 835 root 20 0 0 0 0 S 0 0.0 0:01.53 usb-storage 959 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_1 960 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_2 961 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_3 962 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_4 963 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_5 964 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_6 1338 root 20 0 1684 532 504 S 0 0.0 0:00.08 syslogd 1342 root 20 0 1636 360 356 S 0 0.0 0:00.13 klogd 1381 root 20 0 1656 412 392 S 0 0.0 0:00.00 ifplugd 1445 root 20 0 1668 464 460 S 0 0.0 0:00.00 inetd 1455 root 20 0 1636 472 468 S 0 0.0 0:00.00 acpid 1462 root 20 0 1816 636 548 S 0 0.0 0:00.00 crond 1464 daemon 20 0 1672 320 304 S 0 0.0 0:00.00 atd 1469 root 20 0 8484 1424 1296 S 0 0.0 0:00.03 nmbd 1471 root 20 0 13620 2712 2604 S 0 0.1 0:00.03 smbd 1587 root 20 0 0 0 0 S 0 0.0 0:00.00 mdrecoveryd 1589 root 20 0 0 0 0 S 0 0.0 0:00.00 spinupd 1590 root 20 0 0 0 0 S 0 0.0 0:00.00 spinupd 1601 root 20 0 0 0 0 S 0 0.0 0:23.41 unraidd 1619 root 20 0 0 0 0 S 0 0.0 0:00.03 reiserfs/0 1620 root 20 0 0 0 0 S 0 0.0 0:00.02 reiserfs/1 1621 root 20 0 0 0 0 S 0 0.0 0:00.00 reiserfs/2 1622 root 20 0 0 0 0 S 0 0.0 0:00.00 reiserfs/3 1988 root 20 0 3792 720 716 S 0 0.0 0:00.00 sshd 2229 root 20 0 1480 244 240 S 0 0.0 0:00.00 lighttpd-angel 2230 root 20 0 2092 716 712 S 0 0.0 0:00.00 awk 2232 root 20 0 4696 1416 1364 S 0 0.0 0:00.01 lighttpd 2233 root 20 0 0 0 0 Z 0 0.0 0:00.03 php-cgi 2235 root 20 0 0 0 0 Z 0 0.0 0:00.02 php-cgi 2237 root 20 0 0 0 0 Z 0 0.0 0:00.04 php-cgi 2239 root 20 0 0 0 0 Z 0 0.0 0:00.02 php-cgi 2251 root 20 0 10632 2452 2244 S 0 0.1 0:00.96 sshd 2257 root 20 0 2392 1108 1104 S 0 0.0 0:00.76 bash 2273 root 20 0 2292 320 316 S 0 0.0 0:00.00 uu 2274 root 20 0 1620 472 468 S 0 0.0 0:00.01 logger 2278 root 20 0 4832 2712 884 S 0 0.1 0:03.72 awk 2523 root 20 0 1636 460 456 S 0 0.0 0:00.00 agetty 2524 root 20 0 1636 460 456 S 0 0.0 0:00.00 agetty 2525 root 20 0 1636 460 456 S 0 0.0 0:00.00 agetty 2526 root 20 0 1628 460 456 S 0 0.0 0:00.00 agetty 2527 root 20 0 1632 460 456 S 0 0.0 0:00.00 agetty 2528 root 20 0 1632 460 456 S 0 0.0 0:00.00 agetty 2738 root 20 0 3912 1092 996 S 0 0.0 0:00.02 ntpd 3159 root 20 0 2640 1244 1072 S 0 0.0 0:00.50 bash 6828 root 20 0 0 0 0 S 0 0.0 0:01.77 flush-9:2 7192 root 20 0 161m 13m 3360 S 0 0.4 0:02.67 python 7887 root 20 0 0 0 0 S 0 0.0 0:00.00 flush-8:32 7936 root 20 0 2092 904 728 S 0 0.0 0:00.00 gawk 7939 root 20 0 2264 1044 904 S 0 0.0 0:00.00 sh 7940 root 20 0 2112 908 700 R 0 0.0 0:00.01 top Can anyone help me out on this?
June 27, 201016 yr Author Sorry, syslog: Jun 27 18:50:31 EpicStorage kernel: smbd invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0 Jun 27 18:50:31 EpicStorage kernel: Pid: 2710, comm: smbd Not tainted 2.6.32.9-unRAID #2 Jun 27 18:50:31 EpicStorage kernel: Call Trace: Jun 27 18:50:31 EpicStorage kernel: [] oom_kill_process+0x59/0x1cd Jun 27 18:50:31 EpicStorage kernel: [] __out_of_memory+0xef/0x102 Jun 27 18:50:31 EpicStorage kernel: [] out_of_memory+0x5e/0x83 Jun 27 18:50:31 EpicStorage kernel: [] __alloc_pages_nodemask+0x375/0x42f Jun 27 18:50:31 EpicStorage kernel: [] ? tcp_v4_do_rcv+0x21/0x162 Jun 27 18:50:31 EpicStorage kernel: [] grab_cache_page_write_begin+0x54/0x8e Jun 27 18:50:31 EpicStorage kernel: [] fuse_file_aio_write+0x286/0x4fa Jun 27 18:50:31 EpicStorage kernel: [] ? sock_common_recvmsg+0x31/0x4a Jun 27 18:50:31 EpicStorage kernel: [] do_sync_write+0xbb/0xf9 Jun 27 18:50:31 EpicStorage kernel: [] ? autoremove_wake_function+0x0/0x30 Jun 27 18:50:31 EpicStorage kernel: [] ? autoremove_wake_function+0x0/0x30 Jun 27 18:50:31 EpicStorage kernel: [] ? vfs_read+0xfd/0x114 Jun 27 18:50:31 EpicStorage kernel: [] ? do_sync_write+0x0/0xf9 Jun 27 18:50:31 EpicStorage kernel: [] vfs_write+0x8c/0x116 Jun 27 18:50:31 EpicStorage kernel: [] sys_pwrite64+0x44/0x5d Jun 27 18:50:31 EpicStorage kernel: [] syscall_call+0x7/0xb Jun 27 18:50:31 EpicStorage kernel: Mem-Info: Jun 27 18:50:31 EpicStorage kernel: DMA per-cpu: Jun 27 18:50:31 EpicStorage kernel: CPU 0: hi: 0, btch: 1 usd: 0 Jun 27 18:50:31 EpicStorage kernel: CPU 1: hi: 0, btch: 1 usd: 0 Jun 27 18:50:31 EpicStorage kernel: CPU 2: hi: 0, btch: 1 usd: 0 Jun 27 18:50:31 EpicStorage kernel: CPU 3: hi: 0, btch: 1 usd: 0 Jun 27 18:50:31 EpicStorage kernel: Normal per-cpu: Jun 27 18:50:31 EpicStorage kernel: CPU 0: hi: 186, btch: 31 usd: 175 Jun 27 18:50:31 EpicStorage kernel: CPU 1: hi: 186, btch: 31 usd: 89 Jun 27 18:50:31 EpicStorage kernel: CPU 2: hi: 186, btch: 31 usd: 159 Jun 27 18:50:31 EpicStorage kernel: CPU 3: hi: 186, btch: 31 usd: 168 Jun 27 18:50:31 EpicStorage kernel: HighMem per-cpu: Jun 27 18:50:31 EpicStorage kernel: CPU 0: hi: 186, btch: 31 usd: 64 Jun 27 18:50:31 EpicStorage kernel: CPU 1: hi: 186, btch: 31 usd: 72 Jun 27 18:50:31 EpicStorage kernel: CPU 2: hi: 186, btch: 31 usd: 171 Jun 27 18:50:31 EpicStorage kernel: CPU 3: hi: 186, btch: 31 usd: 68 Jun 27 18:50:31 EpicStorage kernel: active_anon:7533 inactive_anon:2456 isolated_anon:0 Jun 27 18:50:31 EpicStorage kernel: active_file:803 inactive_file:852 isolated_file:0 Jun 27 18:50:31 EpicStorage kernel: unevictable:918585 dirty:6 writeback:2454 unstable:0 Jun 27 18:50:31 EpicStorage kernel: free:28302 slab_reclaimable:3419 slab_unreclaimable:2146 Jun 27 18:50:31 EpicStorage kernel: mapped:2641 shmem:4 pagetables:188 bounce:601 Jun 27 18:50:31 EpicStorage kernel: DMA free:15196kB min:64kB low:80kB high:96kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:28kB unevictable:728kB isolated(anon):0kB isolated(file):0kB present:15860kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jun 27 18:50:31 EpicStorage kernel: lowmem_reserve[]: 0 867 3784 3784 Jun 27 18:50:31 EpicStorage kernel: Normal free:96400kB min:3732kB low:4664kB high:5596kB active_anon:2648kB inactive_anon:2812kB active_file:3092kB inactive_file:3188kB unevictable:698648kB isolated(anon):0kB isolated(file):0kB present:887976kB mlocked:0kB dirty:24kB writeback:2784kB mapped:4kB shmem:0kB slab_reclaimable:13676kB slab_unreclaimable:8584kB kernel_stack:1008kB pagetables:4kB unstable:0kB bounce:2404kB writeback_tmp:0kB pages_scanned:8383 all_unreclaimable? no Jun 27 18:50:31 EpicStorage kernel: lowmem_reserve[]: 0 0 23342 23342 Jun 27 18:50:31 EpicStorage kernel: HighMem free:1612kB min:512kB low:3652kB high:6792kB active_anon:27484kB inactive_anon:7012kB active_file:120kB inactive_file:192kB unevictable:2974964kB isolated(anon):0kB isolated(file):0kB present:2987860kB mlocked:0kB dirty:0kB writeback:7032kB mapped:10560kB shmem:16kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:748kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:9360 all_unreclaimable? no Jun 27 18:50:31 EpicStorage kernel: lowmem_reserve[]: 0 0 0 0 Jun 27 18:50:31 EpicStorage kernel: DMA: 1*4kB 1*8kB 1*16kB 2*32kB 2*64kB 1*128kB 2*256kB 0*512kB 2*1024kB 2*2048kB 2*4096kB = 15196kB Jun 27 18:50:31 EpicStorage kernel: Normal: 21394*4kB 870*8kB 2*16kB 6*32kB 3*64kB 1*128kB 1*256kB 2*512kB 0*1024kB 1*2048kB 0*4096kB = 96408kB Jun 27 18:50:31 EpicStorage kernel: HighMem: 359*4kB 8*8kB 1*16kB 1*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1612kB Jun 27 18:50:31 EpicStorage kernel: 922859 total pagecache pages Jun 27 18:50:31 EpicStorage kernel: 2593 pages in swap cache Jun 27 18:50:31 EpicStorage kernel: Swap cache stats: add 26392, delete 23799, find 16631/19377 Jun 27 18:50:31 EpicStorage kernel: Free swap = 2079908kB Jun 27 18:50:31 EpicStorage kernel: Total swap = 2097144kB Jun 27 18:50:31 EpicStorage kernel: 1146880 pages RAM Jun 27 18:50:31 EpicStorage kernel: 919042 pages HighMem Jun 27 18:50:31 EpicStorage kernel: 175313 pages reserved Jun 27 18:50:31 EpicStorage kernel: 9548 pages shared Jun 27 18:50:31 EpicStorage kernel: 937669 pages non-shared Jun 27 18:50:31 EpicStorage kernel: Out of memory: kill process 7192 (python) score 2577 or a child Jun 27 18:50:31 EpicStorage kernel: Killed process 7192 (python) Jun 27 18:53:35 EpicStorage kernel: smbd invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0 Jun 27 18:53:35 EpicStorage kernel: Pid: 2710, comm: smbd Not tainted 2.6.32.9-unRAID #2 Jun 27 18:53:35 EpicStorage kernel: Call Trace: Jun 27 18:53:35 EpicStorage kernel: [] oom_kill_process+0x59/0x1cd Jun 27 18:53:35 EpicStorage kernel: [] __out_of_memory+0xef/0x102 Jun 27 18:53:35 EpicStorage kernel: [] out_of_memory+0x5e/0x83 Jun 27 18:53:35 EpicStorage kernel: [] __alloc_pages_nodemask+0x375/0x42f Jun 27 18:53:35 EpicStorage kernel: [] grab_cache_page_write_begin+0x54/0x8e Jun 27 18:53:35 EpicStorage kernel: [] fuse_file_aio_write+0x286/0x4fa Jun 27 18:53:35 EpicStorage kernel: [] ? sock_common_recvmsg+0x31/0x4a Jun 27 18:53:35 EpicStorage kernel: [] do_sync_write+0xbb/0xf9 Jun 27 18:53:35 EpicStorage kernel: [] ? autoremove_wake_function+0x0/0x30 Jun 27 18:53:35 EpicStorage kernel: [] ? autoremove_wake_function+0x0/0x30 Jun 27 18:53:35 EpicStorage kernel: [] ? vfs_read+0xfd/0x114 Jun 27 18:53:35 EpicStorage kernel: [] ? do_sync_write+0x0/0xf9 Jun 27 18:53:35 EpicStorage kernel: [] vfs_write+0x8c/0x116 Jun 27 18:53:35 EpicStorage kernel: [] sys_pwrite64+0x44/0x5d Jun 27 18:53:35 EpicStorage kernel: [] syscall_call+0x7/0xb Jun 27 18:53:35 EpicStorage kernel: Mem-Info: Jun 27 18:53:35 EpicStorage kernel: DMA per-cpu: Jun 27 18:53:35 EpicStorage kernel: CPU 0: hi: 0, btch: 1 usd: 0 Jun 27 18:53:35 EpicStorage kernel: CPU 1: hi: 0, btch: 1 usd: 0 Jun 27 18:53:35 EpicStorage kernel: CPU 2: hi: 0, btch: 1 usd: 0 Jun 27 18:53:35 EpicStorage kernel: CPU 3: hi: 0, btch: 1 usd: 0 Jun 27 18:53:35 EpicStorage kernel: Normal per-cpu: Jun 27 18:53:35 EpicStorage kernel: CPU 0: hi: 186, btch: 31 usd: 159 Jun 27 18:53:35 EpicStorage kernel: CPU 1: hi: 186, btch: 31 usd: 160 Jun 27 18:53:35 EpicStorage kernel: CPU 2: hi: 186, btch: 31 usd: 183 Jun 27 18:53:35 EpicStorage kernel: CPU 3: hi: 186, btch: 31 usd: 166 Jun 27 18:53:35 EpicStorage kernel: HighMem per-cpu: Jun 27 18:53:35 EpicStorage kernel: CPU 0: hi: 186, btch: 31 usd: 184 Jun 27 18:53:35 EpicStorage kernel: CPU 1: hi: 186, btch: 31 usd: 134 Jun 27 18:53:35 EpicStorage kernel: CPU 2: hi: 186, btch: 31 usd: 180 Jun 27 18:53:35 EpicStorage kernel: CPU 3: hi: 186, btch: 31 usd: 158 Jun 27 18:53:35 EpicStorage kernel: active_anon:3648 inactive_anon:1240 isolated_anon:0 Jun 27 18:53:35 EpicStorage kernel: active_file:910 inactive_file:754 isolated_file:32 Jun 27 18:53:35 EpicStorage kernel: unevictable:922274 dirty:6 writeback:1226 unstable:0 Jun 27 18:53:35 EpicStorage kernel: free:29070 slab_reclaimable:3414 slab_unreclaimable:2597 Jun 27 18:53:35 EpicStorage kernel: mapped:2250 shmem:0 pagetables:183 bounce:374 Jun 27 18:53:35 EpicStorage kernel: DMA free:14232kB min:64kB low:80kB high:96kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:728kB isolated(anon):0kB isolated(file):0kB present:15860kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:992kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jun 27 18:53:35 EpicStorage kernel: lowmem_reserve[]: 0 867 3784 3784 Jun 27 18:53:35 EpicStorage kernel: Normal free:96468kB min:3732kB low:4664kB high:5596kB active_anon:808kB inactive_anon:952kB active_file:3448kB inactive_file:3016kB unevictable:701664kB isolated(anon):128kB isolated(file):0kB present:887976kB mlocked:0kB dirty:20kB writeback:1040kB mapped:4kB shmem:0kB slab_reclaimable:13656kB slab_unreclaimable:9396kB kernel_stack:1008kB pagetables:16kB unstable:0kB bounce:1496kB writeback_tmp:0kB pages_scanned:5159 all_unreclaimable? no Jun 27 18:53:35 EpicStorage kernel: lowmem_reserve[]: 0 0 23342 23342 Jun 27 18:53:35 EpicStorage kernel: HighMem free:5580kB min:512kB low:3652kB high:6792kB active_anon:13784kB inactive_anon:3860kB active_file:44kB inactive_file:0kB unevictable:2986704kB isolated(anon):0kB isolated(file):0kB present:2987860kB mlocked:0kB dirty:4kB writeback:3864kB mapped:8996kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:716kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4343 all_unreclaimable? no Jun 27 18:53:35 EpicStorage kernel: lowmem_reserve[]: 0 0 0 0 Jun 27 18:53:35 EpicStorage kernel: DMA: 2*4kB 2*8kB 2*16kB 13*32kB 5*64kB 5*128kB 2*256kB 0*512kB 2*1024kB 1*2048kB 2*4096kB = 14232kB Jun 27 18:53:35 EpicStorage kernel: Normal: 23443*4kB 25*8kB 2*16kB 0*32kB 1*64kB 2*128kB 4*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 96372kB Jun 27 18:53:35 EpicStorage kernel: HighMem: 1302*4kB 46*8kB 2*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 5704kB Jun 27 18:53:35 EpicStorage kernel: 925536 total pagecache pages Jun 27 18:53:35 EpicStorage kernel: 1633 pages in swap cache Jun 27 18:53:35 EpicStorage kernel: Swap cache stats: add 169386, delete 167753, find 50818/81842 Jun 27 18:53:35 EpicStorage kernel: Free swap = 2066680kB Jun 27 18:53:35 EpicStorage kernel: Total swap = 2097144kB Jun 27 18:53:35 EpicStorage kernel: 1146880 pages RAM Jun 27 18:53:35 EpicStorage kernel: 919042 pages HighMem Jun 27 18:53:35 EpicStorage kernel: 175313 pages reserved Jun 27 18:53:35 EpicStorage kernel: 8591 pages shared Jun 27 18:53:35 EpicStorage kernel: 936554 pages non-shared Jun 27 18:53:35 EpicStorage kernel: Out of memory: kill process 8067 (python) score 3186 or a child Jun 27 18:53:35 EpicStorage kernel: Killed process 8067 (python) Jun 27 18:53:35 EpicStorage kernel: python invoked oom-killer: gfp_mask=0x200d2, order=0, oom_adj=0 Jun 27 18:53:35 EpicStorage kernel: Pid: 8089, comm: python Not tainted 2.6.32.9-unRAID #2 Jun 27 18:53:35 EpicStorage kernel: Call Trace: Jun 27 18:53:35 EpicStorage kernel: [] oom_kill_process+0x59/0x1cd Jun 27 18:53:35 EpicStorage kernel: [] __out_of_memory+0xef/0x102 Jun 27 18:53:35 EpicStorage kernel: [] out_of_memory+0x5e/0x83 Jun 27 18:53:35 EpicStorage kernel: [] __alloc_pages_nodemask+0x375/0x42f Jun 27 18:53:35 EpicStorage kernel: [] grab_cache_page_write_begin+0x54/0x8e Jun 27 18:53:35 EpicStorage kernel: [] simple_write_begin+0x1e/0x50 Jun 27 18:53:35 EpicStorage kernel: [] generic_file_buffered_write+0xb9/0x1de Jun 27 18:53:35 EpicStorage kernel: [] __generic_file_aio_write+0x3ca/0x404 Jun 27 18:53:35 EpicStorage kernel: [] ? mntput_no_expire+0x16/0x74 Jun 27 18:53:35 EpicStorage kernel: [] generic_file_aio_write+0x54/0x95 Jun 27 18:53:35 EpicStorage kernel: [] do_sync_write+0xbb/0xf9 Jun 27 18:53:35 EpicStorage kernel: [] ? cp_new_stat64+0xe8/0xfa Jun 27 18:53:35 EpicStorage kernel: [] ? autoremove_wake_function+0x0/0x30 Jun 27 18:53:35 EpicStorage kernel: [] ? __rcu_process_callbacks+0xce/0x1f9 Jun 27 18:53:35 EpicStorage kernel: [] ? do_sync_write+0x0/0xf9 Jun 27 18:53:35 EpicStorage kernel: [] vfs_write+0x8c/0x116 Jun 27 18:53:35 EpicStorage kernel: [] sys_write+0x3b/0x60 Jun 27 18:53:35 EpicStorage kernel: [] syscall_call+0x7/0xb Jun 27 18:53:35 EpicStorage kernel: Mem-Info: Jun 27 18:53:35 EpicStorage kernel: DMA per-cpu: Jun 27 18:53:35 EpicStorage kernel: CPU 0: hi: 0, btch: 1 usd: 0 Jun 27 18:53:35 EpicStorage kernel: CPU 1: hi: 0, btch: 1 usd: 0 Jun 27 18:53:35 EpicStorage kernel: CPU 2: hi: 0, btch: 1 usd: 0 Jun 27 18:53:35 EpicStorage kernel: CPU 3: hi: 0, btch: 1 usd: 0 Jun 27 18:53:35 EpicStorage kernel: Normal per-cpu: Jun 27 18:53:35 EpicStorage kernel: CPU 0: hi: 186, btch: 31 usd: 156 Jun 27 18:53:35 EpicStorage kernel: CPU 1: hi: 186, btch: 31 usd: 159 Jun 27 18:53:35 EpicStorage kernel: CPU 2: hi: 186, btch: 31 usd: 155 Jun 27 18:53:35 EpicStorage kernel: CPU 3: hi: 186, btch: 31 usd: 168 Jun 27 18:53:35 EpicStorage kernel: HighMem per-cpu: Jun 27 18:53:35 EpicStorage kernel: CPU 0: hi: 186, btch: 31 usd: 140 Jun 27 18:53:35 EpicStorage kernel: CPU 1: hi: 186, btch: 31 usd: 21 Jun 27 18:53:35 EpicStorage kernel: CPU 2: hi: 186, btch: 31 usd: 179 Jun 27 18:53:35 EpicStorage kernel: CPU 3: hi: 186, btch: 31 usd: 43 Jun 27 18:53:35 EpicStorage kernel: active_anon:3648 inactive_anon:1245 isolated_anon:0 Jun 27 18:53:35 EpicStorage kernel: active_file:822 inactive_file:1011 isolated_file:32 Jun 27 18:53:35 EpicStorage kernel: unevictable:922274 dirty:116 writeback:1226 unstable:0 Jun 27 18:53:35 EpicStorage kernel: free:29070 slab_reclaimable:3414 slab_unreclaimable:2597 Jun 27 18:53:35 EpicStorage kernel: mapped:2250 shmem:0 pagetables:183 bounce:374 Jun 27 18:53:35 EpicStorage kernel: DMA free:14232kB min:64kB low:80kB high:96kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:728kB isolated(anon):0kB isolated(file):0kB present:15860kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:992kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Jun 27 18:53:35 EpicStorage kernel: lowmem_reserve[]: 0 867 3784 3784 Jun 27 18:53:35 EpicStorage kernel: Normal free:96468kB min:3732kB low:4664kB high:5596kB active_anon:808kB inactive_anon:1120kB active_file:3244kB inactive_file:3164kB unevictable:701664kB isolated(anon):0kB isolated(file):128kB present:887976kB mlocked:0kB dirty:20kB writeback:1040kB mapped:4kB shmem:0kB slab_reclaimable:13656kB slab_unreclaimable:9396kB kernel_stack:1008kB pagetables:16kB unstable:0kB bounce:1496kB writeback_tmp:0kB pages_scanned:608 all_unreclaimable? no Jun 27 18:53:35 EpicStorage kernel: lowmem_reserve[]: 0 0 23342 23342 Jun 27 18:53:35 EpicStorage kernel: HighMem free:5580kB min:512kB low:3652kB high:6792kB active_anon:13784kB inactive_anon:3860kB active_file:44kB inactive_file:880kB unevictable:2986704kB isolated(anon):0kB isolated(file):0kB present:2987860kB mlocked:0kB dirty:444kB writeback:3864kB mapped:8996kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:716kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4919 all_unreclaimable? no Jun 27 18:53:35 EpicStorage kernel: lowmem_reserve[]: 0 0 0 0 Jun 27 18:53:35 EpicStorage kernel: DMA: 2*4kB 2*8kB 2*16kB 13*32kB 5*64kB 5*128kB 2*256kB 0*512kB 2*1024kB 1*2048kB 2*4096kB = 14232kB Jun 27 18:53:35 EpicStorage kernel: Normal: 23466*4kB 34*8kB 2*16kB 1*32kB 1*64kB 2*128kB 4*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 96568kB Jun 27 18:53:35 EpicStorage kernel: HighMem: 1271*4kB 46*8kB 2*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 5580kB Jun 27 18:53:35 EpicStorage kernel: 925811 total pagecache pages Jun 27 18:53:35 EpicStorage kernel: 1641 pages in swap cache Jun 27 18:53:35 EpicStorage kernel: Swap cache stats: add 169403, delete 167762, find 50872/81902 Jun 27 18:53:35 EpicStorage kernel: Free swap = 2066680kB Jun 27 18:53:35 EpicStorage kernel: Total swap = 2097144kB Jun 27 18:53:35 EpicStorage kernel: 1146880 pages RAM Jun 27 18:53:35 EpicStorage kernel: 919042 pages HighMem Jun 27 18:53:35 EpicStorage kernel: 175313 pages reserved Jun 27 18:53:35 EpicStorage kernel: 8866 pages shared Jun 27 18:53:35 EpicStorage kernel: 936530 pages non-shared Jun 27 18:53:35 EpicStorage kernel: Out of memory: kill process 1464 (atd) score 418 or a child Jun 27 18:53:35 EpicStorage kernel: Killed process 1464 (atd)
June 27, 201016 yr Sorry, syslog: Trust me, that is not the full syslog, but you are correct, even with the huge swapfile, you are out of memory. Obiously, you are running your own distribution... You might need to adjust some tunables to permit the swapfile to be used. Right now, it looks like your buffer cache is using all the physical RAM. Joe L.
June 27, 201016 yr What stuff did you install? I had the same issue, I had to change the vm.cache_pressure to a higher value.
June 27, 201016 yr Author Trust me, that is not the full syslog, but you are correct, even with the huge swapfile, you are out of memory. Obiously, you are running your own distribution... You might need to adjust some tunables to permit the swapfile to be used. Right now, it looks like your buffer cache is using all the physical RAM. Nope, using the standard unRAID, I installed the swap space from unRaid menu. I also install nano, unraid-web and sabnzbd
June 27, 201016 yr Trust me, that is not the full syslog, but you are correct, even with the huge swapfile, you are out of memory. Obiously, you are running your own distribution... You might need to adjust some tunables to permit the swapfile to be used. Right now, it looks like your buffer cache is using all the physical RAM. Nope, using the standard unRAID, I installed the swap space from unRaid menu. I also install nano, unraid-web and sabnzbd I'm assuming you meant the unMENU package manager. Try adjusting the cache_pressure, as Weebotech mentioned. Type: sysctl vm.vfs_cache_pressure=200
June 27, 201016 yr I do these massive rsync backups. before I do them I have to drop the cache with echo 3 > /proc/sys/vm/drop_caches You can try adjusting the following tunables /proc/sys/vm/swappiness /proc/sys/vm/vfs_cache_pressure I set mine to the following values until I get to the bottom of the situation. root@Atlas /mnt/disk2/backups/bin #more /proc/sys/vm/vfs_cache_pressure /proc/sys/vm/swappiness :::::::::::::: /proc/sys/vm/vfs_cache_pressure :::::::::::::: 50 :::::::::::::: /proc/sys/vm/swappiness :::::::::::::: 100 I just do not have the time to debug this. My issue is when I do an updatedb with mlocate across the whole filesystem. (another reason I'm writing my own locate tool).
June 27, 201016 yr Author I sloved the problem, two fold; vm.vfs_cache_pressure made it better, but the main thing was that sabnzbd was caching everything it downloaded into memory, so the 9GB file i was downloading was all being put into memory. Cheers for the help.
Archived
This topic is now archived and is closed to further replies.