KriS Posted May 28, 2020 Share Posted May 28, 2020 Hi, from 6.8.x version I've problem with CPU usage. After few days from boot one core is utilized at 100%... sometimes it's happen after 5 days.. sometimes after 2 weeks. The case before it was core 10, but now after about 2 weeks from boot it was core 15. I didn't restart server this time, and after two weeks more, one more core was utilized at 100%, so I did have core 11, core 15 used by something and one more at ~60%. I start digging and found this: top - 01:30:06 up 43 days, 15:23, 4 users, load average: 3.58, 3.08, 3.26 Tasks: 3 total, 0 running, 3 sleeping, 0 stopped, 0 zombie %Cpu0 : 16.7 us, 1.4 sy, 0.0 ni, 81.6 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0 st %Cpu1 : 9.1 us, 0.7 sy, 0.0 ni, 90.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu2 : 19.5 us, 1.7 sy, 0.0 ni, 77.4 id, 1.0 wa, 0.0 hi, 0.3 si, 0.0 st %Cpu3 : 1.7 us, 1.0 sy, 0.0 ni, 97.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu4 : 30.0 us, 2.0 sy, 0.0 ni, 67.0 id, 0.7 wa, 0.0 hi, 0.3 si, 0.0 st %Cpu5 : 9.7 us, 1.0 sy, 0.0 ni, 89.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu6 : 7.1 us, 10.2 sy, 0.0 ni, 82.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu7 : 10.1 us, 0.7 sy, 0.0 ni, 89.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu8 : 2.0 us, 5.0 sy, 0.0 ni, 89.6 id, 3.3 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu9 : 8.0 us, 4.3 sy, 0.0 ni, 82.7 id, 4.7 wa, 0.0 hi, 0.3 si, 0.0 st %Cpu10 : 3.0 us, 4.1 sy, 0.0 ni, 90.5 id, 2.4 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu11 : 2.0 us, 2.7 sy, 0.0 ni, 0.0 id, 95.3 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu12 : 5.3 us, 4.3 sy, 0.0 ni, 87.0 id, 3.0 wa, 0.0 hi, 0.3 si, 0.0 st %Cpu13 : 1.0 us, 3.0 sy, 0.0 ni, 93.3 id, 2.7 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu14 : 26.7 us, 23.3 sy, 0.0 ni, 49.3 id, 0.7 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu15 : 12.7 us, 10.7 sy, 0.0 ni, 0.0 id, 76.6 wa, 0.0 hi, 0.0 si, 0.0 st MiB Mem : 32160.7 total, 230.6 free, 22955.5 used, 8974.6 buff/cache MiB Swap: 0.0 total, 0.0 free, 0.0 used. 7836.4 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 20710 root 20 0 3830296 3.2g 22160 S 71.3 10.1 12117:42 /usr/bin/qemu-system-x86_64 -name guest=OPNsense,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-OPNsense/mast+ 20595 root 20 0 4995344 4.2g 22404 S 30.7 13.3 19528:28 /usr/bin/qemu-system-x86_64 -name guest=Windows 10,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-Windows 10/+ 7680 root 20 0 4863852 4.1g 22224 S 9.3 13.1 5473:36 /usr/bin/qemu-system-x86_64 -name guest=appliance,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-ap+ OPNsens/Windows it's for tests... not used and was at idle... only turned on. Third VM is used little bit, but CPU usage is at ~3% when I check it from VM site. So I turned off all VM, but CPU still was used... so I turned off docker also.. and still get hig cpu usage at cores 11/15. top - 02:03:49 up 43 days, 15:57, 4 users, load average: 3.00, 3.11, 3.38 Tasks: 363 total, 1 running, 362 sleeping, 0 stopped, 0 zombie %Cpu0 : 0.5 us, 0.0 sy, 0.0 ni, 99.5 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu1 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu2 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu3 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu4 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu5 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu6 : 0.5 us, 0.0 sy, 0.0 ni, 99.5 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu7 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu8 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu9 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu10 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu11 : 0.0 us, 0.0 sy, 0.0 ni, 0.0 id,100.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu12 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu13 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu14 : 0.5 us, 0.0 sy, 0.0 ni, 99.5 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st %Cpu15 : 0.0 us, 0.0 sy, 0.0 ni, 0.0 id,100.0 wa, 0.0 hi, 0.0 si, 0.0 st MiB Mem : 32160.7 total, 22289.2 free, 680.0 used, 9191.5 buff/cache MiB Swap: 0.0 total, 0.0 free, 0.0 used. 30114.5 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5614 root 20 0 283636 4124 3352 S 1.1 0.0 315:48.99 emhttpd 781 root 20 0 0 0 0 I 0.5 0.0 0:01.72 kworker/3:1-events 15903 root 20 0 9120 5372 4852 S 0.5 0.0 0:00.37 sshd 32758 root 20 0 0 0 0 I 0.5 0.0 0:03.59 kworker/2:0-events So I turned off everything, not users connected and still CPU is used by something.. because I can't find what use my processor, I did try another cpu-tools. Iit's result from htop: (instead of top) 1 [ 0.0%] 5 [| 0.7%] 9 [ 0.0%] 13 [ 0.0%] 2 [ 0.0%] 6 [ 0.0%] 10 [ 0.0%] 14 [|| 1.3%] 3 [ 0.0%] 7 [ 0.0%] 11 [ 0.0%] 15 [ 0.0%] 4 [ 0.0%] 8 [ 0.0%] 12 [ 0.0%] 16 [ 0.0%] Mem[|||||||||||||||| 1.56G/31.4G] Tasks: 53, 21 thr; 1 running Swp[ 0K/0K] Load average: 3.00 3.02 3.24 Uptime: 43 days, 16:03:17 PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command 5614 root 20 0 276M 4124 3352 S 1.3 0.0 5h15:50 /usr/local/sbin/emhttpd 5731 root 20 0 276M 4124 3352 S 1.3 0.0 4h50:57 /usr/local/sbin/emhttpd 14008 root 20 0 6432 4836 2872 R 0.7 0.0 0:00.55 htop 13996 root 20 0 102M 13076 7304 S 0.7 0.0 0:00.06 php-fpm: pool www 7736 root 20 0 1264M 40588 1060 S 0.7 0.1 0:00.40 /usr/local/sbin/shfs /mnt/user -disks 511 204 13293 root 20 0 102M 13080 7304 S 0.7 0.0 0:00.16 php-fpm: pool www 8508 root 20 0 1264M 40588 1060 S 0.7 0.1 0:00.33 /usr/local/sbin/shfs /mnt/user -disks 511 204 2539 root 20 0 1264M 40588 1060 S 0.7 0.1 0:00.57 /usr/local/sbin/shfs /mnt/user -disks 511 204 6021 root 20 0 1264M 40588 1060 S 0.0 0.1 14h02:16 /usr/local/sbin/shfs /mnt/user -disks 511 204 3161 root 20 0 1264M 40588 1060 S 0.0 0.1 0:00.53 /usr/local/sbin/shfs /mnt/user -disks 511 204 13254 root 20 0 102M 13084 7304 S 0.0 0.0 0:00.17 php-fpm: pool www 13971 root 20 0 102M 13076 7304 S 0.0 0.0 0:00.05 php-fpm: pool www 31448 root 20 0 1264M 40588 1060 S 0.0 0.1 0:00.72 /usr/local/sbin/shfs /mnt/user -disks 511 204 13360 root 20 0 102M 12888 7112 S 0.0 0.0 0:00.15 php-fpm: pool www 14069 root 20 0 102M 12532 6780 S 0.0 0.0 0:00.03 php-fpm: pool www 15903 root 20 0 9120 5372 4852 S 0.0 0.0 0:00.46 sshd: root@pts/3 7490 root 20 0 146M 8348 3792 S 0.0 0.0 21:14.61 nginx: worker process 5730 root 20 0 276M 4124 3352 S 0.0 0.0 15:01.49 /usr/local/sbin/emhttpd 30845 root 20 0 1264M 40588 1060 S 0.0 0.1 0:00.68 /usr/local/sbin/shfs /mnt/user -disks 511 204 5803 root 20 0 3788 2820 2548 S 0.0 0.0 26:37.57 /bin/bash /usr/local/emhttp/webGui/scripts/di 9541 root 20 0 1264M 40588 1060 S 0.0 0.1 0:00.30 /usr/local/sbin/shfs /mnt/user -disks 511 204 1880 root 20 0 1264M 40588 1060 S 0.0 0.1 0:00.54 /usr/local/sbin/shfs /mnt/user -disks 511 204 2171 ntp 20 0 77212 4352 3716 S 0.0 0.0 2:18.06 /usr/sbin/ntpd -g -u ntp:ntp 8509 root 20 0 1264M 40588 1060 S 0.0 0.1 0:00.35 /usr/local/sbin/shfs /mnt/user -disks 511 204 10756 root 20 0 1264M 40588 1060 S 0.0 0.1 0:00.28 /usr/local/sbin/shfs /mnt/user -disks 511 204 1967 root 20 0 8212 4436 1308 S 0.0 0.0 8:37.61 /sbin/haveged -w 1024 -v 1 -p /var/run/havege 1 root 20 0 2468 1840 1736 S 0.0 0.0 0:31.52 init 24860 root 20 0 5680 2228 0 S 0.0 0.0 0:00.00 /usr/sbin/rpc.mountd 24765 root 20 0 2524 1684 1580 S 0.0 0.0 0:00.00 /usr/sbin/inetd 24685 root 20 0 8988 3348 2840 S 0.0 0.0 0:00.00 /usr/sbin/sshd 15905 root 20 0 7344 4176 3264 S 0.0 0.0 0:00.05 -bash 13671 root 20 0 3848 2108 1776 S 0.0 0.0 0:03.70 /bin/bash /usr/local/emhttp/plugins/dynamix.s 13274 root 20 0 2448 756 696 S 0.0 0.0 0:00.00 sleep 300 12065 root 20 0 50600 10760 8088 S 0.0 0.0 0:00.00 /usr/sbin/winbindd -D 12267 root 20 0 51016 9472 6816 S 0.0 0.0 0:00.00 /usr/sbin/winbindd -D 12068 root 20 0 50968 15896 13312 S 0.0 0.0 0:00.00 /usr/sbin/winbindd -D 12062 root 20 0 2512 104 0 S 0.0 0.0 0:00.00 /usr/sbin/wsdd 12055 root 20 0 35340 6540 4380 S 0.0 0.0 0:00.01 /usr/sbin/nmbd -D 12050 root 20 0 59900 17552 14712 S 0.0 0.1 0:00.00 /usr/sbin/smbd -D 12054 root 20 0 50340 7284 4568 S 0.0 0.0 0:00.00 /usr/sbin/smbd -D 12053 root 20 0 50332 8140 5424 S 0.0 0.0 0:00.00 /usr/sbin/smbd -D 7489 root 20 0 145M 3812 28 S 0.0 0.0 0:00.00 nginx: master process /usr/sbin/nginx -c /etc 7483 root 20 0 13492 5132 3952 S 0.0 0.0 1:25.22 ttyd -d 0 -i /var/run/ttyd.sock login -f root 7447 root 20 0 99M 11800 6232 S 0.0 0.0 1:31.38 php-fpm: master process (/etc/php-fpm/php-fpm 11706 root 20 0 102M 13288 7432 S 0.0 0.0 0:00.05 php-fpm: pool www 13828 root 20 0 17296 14732 4104 D 0.0 0.0 0:00.24 lshw -xml -sanitize -quiet 6008 root 20 0 141M 744 280 S 0.0 0.0 0:02.62 /usr/local/sbin/shfs /mnt/user0 -disks 510 -o 6010 root 20 0 141M 744 280 S 0.0 0.0 0:01.30 /usr/local/sbin/shfs /mnt/user0 -disks 510 -o 6009 root 20 0 141M 744 280 S 0.0 0.0 0:01.31 /usr/local/sbin/shfs /mnt/user0 -disks 510 -o 5715 root 20 0 4716 108 0 S 0.0 0.0 0:00.00 /usr/sbin/avahi-dnsconfd -D 5704 avahi 20 0 6420 3288 2848 S 0.0 0.0 1:11.14 avahi-daemon: running [VDS.local] 5706 avahi 20 0 6024 264 0 S 0.0 0.0 0:00.00 avahi-daemon: chroot helper 5733 root 20 0 276M 4124 3352 S 0.0 0.0 0:00.00 /usr/local/sbin/emhttpd F1Help F2Setup F3SearchF4FilterF5Tree F6SortByF7Nice -F8Nice +F9Kill F10Quit So, nothing use my CPU.. and now Im confused... my cpu was used or not? When I reboot my server I see normal CPU usage ~3% at dashboard. server-diagnostics-20200525-0138(anony).zip Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.