bubbl3
-
Posts
126 -
Joined
-
Last visited
Content Type
Profiles
Forums
Downloads
Store
Gallery
Posts posted by bubbl3
-
-
3 minutes ago, Atilla0406 said:
on unraid web ui, I've clicked on flash drive and at the bottom it shows this enabled
That's not what I asked, can you try to boot a different USB drive with a different OS?
-
11 minutes ago, Atilla0406 said:
previously i did try alternative usb , also tried ubuntu prior. Unraid server itself has been up and running since December
Ok, but what about now, does another bootable usb boot automatically?
-
Did you try another bootable USB? Like another drive and/or another OS, or both at the same time. Just to make sure it's a bios issue.
-
1
-
-
Recommended steps if you have the ability to get another drive:
- Take disk out of the server and keep in safe space
- Install new 6TB drive and follow rebuild process
- If rebuild successful proceed RMA faulty drive
- When drive comes back from RMA, keep it as spare
-
1
-
For sake of completion, disabling SMT seems to be a better workaround if that's an option in your setup: https://forums.unraid.net/bug-reports/stable-releases/6128-no-webgui-run-full-r2854/?do=findComment&comment=27345
-
2 hours ago, Amane said:
I found a solution:
Thanks, but this is not a solution, just a workaround. Would also suggest you to check your syslog after the change as you may also have DBUS failing (it starts before the GO file), which will cause all sort of underlaying issues:On 2/17/2024 at 11:31 AM, bubbl3 said:Adding these lines to the go file seems to be working and no services complaining about dbus:
#increase /run size /bin/mount -t tmpfs tmpfs /run -o remount,size=64M #start dbus /usr/bin/dbus-uuidgen --ensure /usr/bin/dbus-daemon --system #start elogind /etc/rc.d/rc.elogind start
Still don't think this is a great solution, ideally the size of /run should be increased at the first mount before any service starts.
At least this confirms why the bug is affecting us, you're on a 64 cores Threadripper and I'm on a 64 cores EPYC, we have more devices creating UDEV files than others, I bet their /run/udev is probably abnormally big as well even if it doesn't affect them.
-
Also look here, all 0 bytes files in /run/udev/data are using 4KB:
-
10 hours ago, dlandon said:
Can you take a little closer look and see if you can figure out what is taking all the space? I'm seeing Apparent size: 511.3 KiB.
That is correct, most of the files are 0 bytes and the total data that can be read from them is 512KB (confirmed by the output of tree --du -h -a /run > run.txt , attached here) , that said the block size is 4KB so every file will consume that amount of space on the partition. If you multiply the number of files for the minimum allocation you get a rough idea of why so much space is being taken: 9843 x 4 = 39372
If I learned anything in my years of working with file systems is that disk usage with small files gets often out of hand due to the default block size, NCDU is actually great for this, with tree or df you would be chasing ghosts as they just list the actual size.
This also doesn't happen with 6.12.6, so it must be some change made between that and 6.12.8
-
Bunch of 4KB files in there, weird:
-
-
Looks like udev is taking all the space:
-
13 minutes ago, dlandon said:
PID files should not really take that much space. Post the output of this command:
ls -la /run/root@Tower:/# ls -la /run/ total 80 drwxr-xr-x 19 root root 1040 Feb 17 17:59 ./ drwxr-xr-x 21 root root 460 Feb 17 14:55 ../ -rw-r--r-- 1 root root 0 Feb 17 14:52 acpid.pid srw-rw-rw- 1 root root 0 Feb 17 14:52 acpid.socket= -rw------- 1 root root 0 Feb 17 14:53 agetty.reload -rw-r--r-- 1 root root 6 Feb 17 14:53 apcupsd.pid -rw-r--r-- 1 root root 0 Feb 17 14:52 atd.pid drwxr-xr-x 2 avahi avahi 80 Feb 17 14:54 avahi-daemon/ -rw-r--r-- 1 root root 6 Feb 17 14:54 avahi-dnsconfd.pid drwxr-xr-x 2 root root 80 Feb 17 14:55 blkid/ drw------- 3 root root 60 Feb 17 14:54 containerd/ drwxr-xr-x 2 root root 40 Feb 17 14:52 cron/ drwx------ 2 root root 40 Feb 17 14:53 cryptsetup/ drwxr-xr-x 2 root root 80 Feb 17 14:53 dbus/ drwx------ 8 root root 180 Feb 17 14:54 docker/ srw-rw---- 1 root docker 0 Feb 17 14:54 docker.sock= -rw-r--r-- 1 root root 5 Feb 17 14:54 dockerd.pid drwxr-xr-x 8 root root 160 Feb 17 14:54 elogind/ -rw-r--r-- 1 root root 6 Feb 17 14:53 elogind.pid srw-rw-rw- 1 root root 0 Feb 17 14:53 emhttpd.socket= drwxr-xr-x 2 root root 40 Feb 17 14:52 faillock/ -rw-r--r-- 1 root root 6 Feb 17 14:53 inetd.pid drwxr-xr-x 12 root root 440 Feb 17 14:54 libvirt/ drwxr-xr-x 4 root root 80 Feb 17 14:52 lock/ drwx------ 2 root root 40 Feb 17 14:52 lvm/ drwxr-xr-x 2 root root 40 Feb 17 14:51 mount/ -rw-r--r-- 1 root root 216 Feb 17 17:30 nchan.pid -rw-r--r-- 1 root root 6 Feb 17 14:53 nginx.pid srw-rw-rw- 1 root root 0 Feb 17 14:53 nginx.socket= -rw-r--r-- 1 root root 0 Feb 17 14:52 ntpd.pid -rw-r--r-- 1 root root 5 Feb 17 14:53 php-fpm.pid srw-rw---- 1 root users 0 Feb 17 14:53 php5-fpm.sock= -rw------- 1 root root 25 Feb 17 14:53 qga.state -rw-r--r-- 1 rpc rpc 6 Feb 17 14:53 rpc.statd.pid drwxr-x--- 2 rpc root 40 Feb 17 14:53 rpcbind/ -r--r--r-- 1 root root 0 Feb 17 14:53 rpcbind.lock srw-rw-rw- 1 root root 0 Feb 17 14:53 rpcbind.sock= -rw-r--r-- 1 root root 5 Feb 17 14:54 rsyslogd.pid -rw-r--r-- 1 root root 1 Feb 17 14:52 runlevel drwxr-xr-x 4 root root 80 Feb 17 14:52 samba/ -rw-r--r-- 1 root root 6 Feb 17 17:58 samba-dcerpcd.pid -rw------- 1 root root 6 Feb 17 14:53 sm-notify.pid -rw-r--r-- 1 root root 6 Feb 17 14:54 smbd.pid -rw-r--r-- 1 root root 6 Feb 17 14:53 sshd.pid -rw-r--r-- 1 root root 5 Feb 17 14:54 syslogd.pid lrwxrwxrwx 1 root root 7 Feb 17 14:52 systemd -> elogind/ drwxr-xr-x 8 root root 180 Feb 17 16:56 udev/ srwxr-xr-x 1 root root 0 Feb 17 14:53 unraid-api.sock= drwxr-xr-x 3 root root 60 Feb 17 14:55 user/ -rw-rw-r-- 1 root utmp 4224 Feb 17 14:55 utmp -rw-r--r-- 1 root root 6 Feb 17 14:54 winbindd.pid -rw------- 1 root root 0 Feb 17 14:52 xtables.lock
-
9 minutes ago, dlandon said:
One of the things you should rethink is the stuff you have in your go file. If it were me, I'd put that in a user script (User Scripts Plugin) and have it run on first array start. Also review what you are doing and see if it is really necessary.
Adding stuff to the go file is not recommended.
I agree, but since the upgrade to 6.12.8 the 32MB of the /run mount are not enough and there's no other way to increase it, if I don't do that many services won't start, not to mention i have to manually start dbus as it runs before the go file and fails to create a .pid.
Have a thread open but no help so far 🤷♂️ -
18 minutes ago, dlandon said:
Everyrthing looks right. Is the share showing up in the network shares? If not, go to the device settings and toggle the 'Share' switch off and then back on and see if the share shows in network shares.
That worked 👍
-
5 minutes ago, dlandon said:
Thanks. Looks like I gave you the wrong path. Now show the output of this command:
cat /etc/samba/unassigned-shares/ZCH0859H.conf and cat /etc/samba/smb-unassigned.confroot@Tower:/etc/samba/unassigned-shares# cat /etc/samba/unassigned-shares/ZCH0859H.conf [ZCH0859H] comment = ZCH0859H path = /mnt/disks/ZCH0859H browseable = yes valid users = emanuele write list = emanuele vfs objects = catia fruit streams_xattr fruit:encoding = native case sensitive = auto preserve case = yes short preserve case = yes [global]root@Tower:/etc/samba/unassigned-shares# cat /etc/samba/smb-unassigned.conf include = /etc/samba/unassigned-shares/ZCH0859H.conf
-
27 minutes ago, dlandon said:
Give me the output of this command:
ls /etc/samba/unassigned.devices/Looks like it's not there:
root@Tower:~# ls /etc/samba/unassigned.devices/ /bin/ls: cannot access '/etc/samba/unassigned.devices/': No such file or directoryThe one there is called unassigned-shares
root@Tower:/etc/samba# ls lmhosts smb-fruit.conf smb-names.conf smb-shares.conf smb-unassigned.conf smb.conf unassigned-shares/ root@Tower:/etc/samba# cd unassigned-shares/ root@Tower:/etc/samba/unassigned-shares# ls ZCH0859H.conf
-
@dlandon can confirm the UI is fixed and reflects the disk being mounted, but the SMB share was not created at boot. See the diagnostics attached and the output of cat /proc/mounts , this time I didn't run the add-smb script manually.
root@Tower:~# cat /proc/mounts rootfs / rootfs rw,size=65861692k,nr_inodes=16465423,inode64 0 0 proc /proc proc rw,relatime 0 0 sysfs /sys sysfs rw,relatime 0 0 tmpfs /run tmpfs rw,relatime,size=65536k,mode=755,inode64 0 0 /dev/sda1 /boot vfat rw,noatime,nodiratime,fmask=0177,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,flush,errors=remount-ro 0 0 /dev/loop0 /lib squashfs ro,relatime,errors=continue 0 0 overlay /lib overlay rw,relatime,lowerdir=/lib,upperdir=/var/local/overlay/lib,workdir=/var/local/overlay-work/lib 0 0 /dev/loop1 /usr squashfs ro,relatime,errors=continue 0 0 overlay /usr overlay rw,relatime,lowerdir=/usr,upperdir=/var/local/overlay/usr,workdir=/var/local/overlay-work/usr 0 0 devtmpfs /dev devtmpfs rw,relatime,size=8192k,nr_inodes=16465424,mode=755,inode64 0 0 devpts /dev/pts devpts rw,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /dev/shm tmpfs rw,relatime,inode64 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 hugetlbfs /hugetlbfs hugetlbfs rw,relatime,pagesize=2M 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 tmpfs /var/log tmpfs rw,relatime,size=131072k,mode=755,inode64 0 0 efivarfs /sys/firmware/efi/efivars efivarfs rw,nosuid,nodev,noexec,relatime 0 0 rootfs /mnt rootfs rw,size=65861692k,nr_inodes=16465423,inode64 0 0 tmpfs /mnt/disks tmpfs rw,relatime,size=1024k,inode64 0 0 tmpfs /mnt/remotes tmpfs rw,relatime,size=1024k,inode64 0 0 tmpfs /mnt/addons tmpfs rw,relatime,size=1024k,inode64 0 0 tmpfs /mnt/rootshare tmpfs rw,relatime,size=1024k,inode64 0 0 nfsd /proc/fs/nfs nfsd rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 /dev/md1p1 /mnt/disk1 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/md2p1 /mnt/disk2 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/md3p1 /mnt/disk3 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/md4p1 /mnt/disk4 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/md5p1 /mnt/disk5 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/md6p1 /mnt/disk6 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/md7p1 /mnt/disk7 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/nvme0n1p1 /mnt/cache btrfs rw,noatime,ssd,discard=async,space_cache=v2,subvolid=5,subvol=/ 0 0 shfs /mnt/user0 fuse.shfs rw,nosuid,nodev,noatime,user_id=0,group_id=0,default_permissions,allow_other 0 0 shfs /mnt/user fuse.shfs rw,nosuid,nodev,noatime,user_id=0,group_id=0,default_permissions,allow_other 0 0 /dev/sdg1 /mnt/disks/ZCH0859H xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/loop2 /var/lib/docker btrfs rw,noatime,ssd,space_cache=v2,subvolid=5,subvol=/ 0 0 /dev/loop2 /var/lib/docker/btrfs btrfs rw,noatime,ssd,space_cache=v2,subvolid=5,subvol=/ 0 0 /dev/loop3 /etc/libvirt btrfs rw,noatime,ssd,space_cache=v2,subvolid=5,subvol=/ 0 0 nsfs /run/docker/netns/133a2385364a nsfs rw 0 0 nsfs /run/docker/netns/5e2c60229e3f nsfs rw 0 0 nsfs /run/docker/netns/86b05786ef76 nsfs rw 0 0 nsfs /run/docker/netns/54188258c33f nsfs rw 0 0 nsfs /run/docker/netns/1429a1c211de nsfs rw 0 0 nsfs /run/docker/netns/b06ebc25f551 nsfs rw 0 0 nsfs /run/docker/netns/dcefdc5d840d nsfs rw 0 0 nsfs /run/docker/netns/50061aaae80d nsfs rw 0 0 nsfs /run/docker/netns/58f45d9663bf nsfs rw 0 0 nsfs /run/docker/netns/f1c5a05c5df4 nsfs rw 0 0 nsfs /run/docker/netns/8227e3776579 nsfs rw 0 0 nsfs /run/docker/netns/4583b25e57bb nsfs rw 0 0 nsfs /run/docker/netns/a27582ff11e7 nsfs rw 0 0 /dev/sdg1 /mnt/disks/ZCH0859H/incomplete xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 nsfs /run/docker/netns/4ef83a37736d nsfs rw 0 0 nsfs /run/docker/netns/eaf7b8eddd9e nsfs rw 0 0 tmpfs /run/user/0 tmpfs rw,nosuid,nodev,relatime,size=13175704k,nr_inodes=3293926,mode=700,inode64 0 0
-
1 minute ago, dlandon said:
Give me the output of this command:
cat /proc/mountsroot@Tower:~# cat /proc/mounts rootfs / rootfs rw,size=65861692k,nr_inodes=16465423,inode64 0 0 proc /proc proc rw,relatime 0 0 sysfs /sys sysfs rw,relatime 0 0 tmpfs /run tmpfs rw,relatime,size=65536k,mode=755,inode64 0 0 /dev/sda1 /boot vfat rw,noatime,nodiratime,fmask=0177,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,flush,errors=remount-ro 0 0 /dev/loop0 /lib squashfs ro,relatime,errors=continue 0 0 overlay /lib overlay rw,relatime,lowerdir=/lib,upperdir=/var/local/overlay/lib,workdir=/var/local/overlay-work/lib 0 0 /dev/loop1 /usr squashfs ro,relatime,errors=continue 0 0 overlay /usr overlay rw,relatime,lowerdir=/usr,upperdir=/var/local/overlay/usr,workdir=/var/local/overlay-work/usr 0 0 devtmpfs /dev devtmpfs rw,relatime,size=8192k,nr_inodes=16465424,mode=755,inode64 0 0 devpts /dev/pts devpts rw,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /dev/shm tmpfs rw,relatime,inode64 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 hugetlbfs /hugetlbfs hugetlbfs rw,relatime,pagesize=2M 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 tmpfs /var/log tmpfs rw,relatime,size=131072k,mode=755,inode64 0 0 efivarfs /sys/firmware/efi/efivars efivarfs rw,nosuid,nodev,noexec,relatime 0 0 rootfs /mnt rootfs rw,size=65861692k,nr_inodes=16465423,inode64 0 0 tmpfs /mnt/disks tmpfs rw,relatime,size=1024k,inode64 0 0 tmpfs /mnt/remotes tmpfs rw,relatime,size=1024k,inode64 0 0 tmpfs /mnt/addons tmpfs rw,relatime,size=1024k,inode64 0 0 tmpfs /mnt/rootshare tmpfs rw,relatime,size=1024k,inode64 0 0 nfsd /proc/fs/nfs nfsd rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 /dev/sdg1 /mnt/disks/ZCH0859H xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/md1p1 /mnt/disk1 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/md2p1 /mnt/disk2 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/md3p1 /mnt/disk3 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/md4p1 /mnt/disk4 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/md5p1 /mnt/disk5 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/md6p1 /mnt/disk6 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/md7p1 /mnt/disk7 xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/nvme0n1p1 /mnt/cache btrfs rw,noatime,ssd,discard=async,space_cache=v2,subvolid=5,subvol=/ 0 0 shfs /mnt/user0 fuse.shfs rw,nosuid,nodev,noatime,user_id=0,group_id=0,default_permissions,allow_other 0 0 shfs /mnt/user fuse.shfs rw,nosuid,nodev,noatime,user_id=0,group_id=0,default_permissions,allow_other 0 0 /dev/loop2 /var/lib/docker btrfs rw,noatime,ssd,space_cache=v2,subvolid=5,subvol=/ 0 0 /dev/loop2 /var/lib/docker/btrfs btrfs rw,noatime,ssd,space_cache=v2,subvolid=5,subvol=/ 0 0 /dev/loop3 /etc/libvirt btrfs rw,noatime,ssd,space_cache=v2,subvolid=5,subvol=/ 0 0 nsfs /run/docker/netns/d4f21cba8555 nsfs rw 0 0 nsfs /run/docker/netns/99301db2a02f nsfs rw 0 0 nsfs /run/docker/netns/e2f446f6a4a5 nsfs rw 0 0 nsfs /run/docker/netns/4671951dc6db nsfs rw 0 0 nsfs /run/docker/netns/ea6dae047f70 nsfs rw 0 0 nsfs /run/docker/netns/dcd0116882af nsfs rw 0 0 nsfs /run/docker/netns/3141f5f422ad nsfs rw 0 0 nsfs /run/docker/netns/9130782b52fb nsfs rw 0 0 nsfs /run/docker/netns/7dab268b47f0 nsfs rw 0 0 nsfs /run/docker/netns/6e12348ec98d nsfs rw 0 0 nsfs /run/docker/netns/1b7c59eaca16 nsfs rw 0 0 nsfs /run/docker/netns/86b1c75956dc nsfs rw 0 0 nsfs /run/docker/netns/0f8eedf38d1b nsfs rw 0 0 /dev/sdg1 /mnt/disks/ZCH0859H/incomplete xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 nsfs /run/docker/netns/225a436b9432 nsfs rw 0 0 nsfs /run/docker/netns/5210094db0a7 nsfs rw 0 0 tmpfs /run/user/0 tmpfs rw,nosuid,nodev,relatime,size=13175704k,nr_inodes=3293926,mode=700,inode64 0 0
Of course this is after i manually executed the smbshare script
-
-
Diagnostic file attached here.
-
Unassigned devices is not working correctly for me after upgrading to 6.12.8, device is set for automount and it does mount but is not reflected in the UI:
If i click MOUNT. the button just goes back to the same state
Additionally the SMB share is not created even if it says so in the syslog:
Feb 17 11:36:23 Tower unassigned.devices: Mounting partition 'sdg1' at mountpoint '/mnt/disks/ZCH0859H'... Feb 17 11:36:23 Tower unassigned.devices: Mount cmd: /sbin/mount -t 'xfs' -o rw,relatime '/dev/sdg1' '/mnt/disks/ZCH0859H' Feb 17 11:36:23 Tower unassigned.devices: Successfully mounted '/dev/sdg1' on '/mnt/disks/ZCH0859H'. Feb 17 11:36:23 Tower unassigned.devices: Adding SMB share 'ZCH0859H'. Feb 17 11:36:23 Tower unassigned.devices: Warning: Unassigned Devices are not set to be shared with NFS.
UPDATE:
SMB share is created if I execute the script manually:/tmp/unassigned.devices/add-smb
-
Adding these lines to the go file seems to be working and no services complaining about dbus:
#increase /run size /bin/mount -t tmpfs tmpfs /run -o remount,size=64M #start dbus /usr/bin/dbus-uuidgen --ensure /usr/bin/dbus-daemon --system #start elogind /etc/rc.d/rc.elogind start
Still don't think this is a great solution, ideally the size of /run should be increased at the first mount before any service starts.
-
1
-
-
Added this to the go file and mostly solved the issue:
#increase /run size /bin/mount -t tmpfs tmpfs /run -o remount,size=64M
But dbus and elogind still fail since they run before the go file:
Feb 17 09:24:14 Tower dbus-daemon[3719]: Failed to start message bus: Failed to close "/var/run/dbus/dbus.pid": No space left on device Feb 17 09:24:14 Tower elogind-daemon[3731]: Failed to write PID file /run/elogind.pid: No space left on deviceAnywhere else the size of the /run mount can be changed so that it applies before anything else?
-
1
-
-
What's the best way to increase the size of /run at boot?
Services not starting after 6.12.8 upgrade (pid files cannot be created)
in General Support
Posted
Fixed in 6.12.9