randall526

This is my script to auto check if a docker is responding and restart it. The verification is http based which will catch both when the docker doesn't respond as well as when there is a http error code present. You have to run a sample curl on your own and potentially choose what http content validates the docker. The script comments are fairly self explanatory and the code function is pretty clean to make the script user friendly and easy to add variables and function lines to check other dockers. UnraidDockerCheck.sh

I had to update broken slackware file locations again since it appears new versions are available for some of the downloads. Incase anyone wants the version I am using on unraid 6.4.1, here it is: openVMTools_auto62 (4).plg There is no GUI front end with unraid but I do show the process running and ESXi reporting tools is indeed running. Great plugin so far, i'm waiting for the next kernel release to see if it updates correctly. Good work putting this together.

changing how the udp protocol was defined worked Didn't think to change it seeing as pfsence oepnVPN server generated it that way on behalf of the VPN client and worked fine outside the plugin. I can stop and start the VPN through the WebGUI now, the small things make one happy. Thanks again.

I ended up scripting around this and starting the VPN in the go file the traditional openvpn command line way and using a cron job script to keep an eye on it. GUI plugin still fails to start the VPN by removing my "dev tun" line in my ovpn config file everytime.

The .ovpn file was partially auto generate by pfsense and it defined the UDP protocol this way remote rand526.ddns.net 1198 udp <========= I normally add the udp line separately myself as you suggested however this way works too. I proved this configuration works by starting the VPN by the command line. This works and the .ovpn file is not butchered by the unraid plugin when started this way. The plugin however when starting the VPN via the GUI keeps editing my .ovpn file and removing the "dev tun" line incorrectly just before the VPN starts. The VPN log then bawks that it's missing the "dev tun". The plugin breaks my .ovpn file every time. This proves the plugin is incorrectly messing with my .ovpn file and the configuration works prior to the plugin deciding to update it. I can see updating .ovpn in the lower right corner flash ever so briefly when I use this .ovpn. What in the plugin decides my .ovpn must be updated and removes this line.... I have no idea. My other .ovpn file I use for another VPN, the plugin doesn't touch and thus I don't get my .ovpn butchered by the plugin lol. Summing up, I think someone understanding the plugin logic rather then openVPN would have to comment as to why the plugin is messing with my .ovpn file and removing "dev tun" every-time. I think this is a plugin issue not a openVPN configuration problem. Something about my setup however the plugin is deciding it must incorrectly update my .ovpn file.

One more thing, confident the plugin was incorrectly editing the .ovpn file which was a valid config before the plugin touched it, I decided to fire the VPN up via the command line. The following command and config file connects to my openVPN server and shows up running in the GUI. However if I bounce it via the GUI plugin, it removes the "dev tun" line and mentioned it updated the file and fails to start due to no dev tun/tap adapter defined. Maddening I say. It only does it for this setup, my other VPN server and .ovpn config file the plugin doesn't touch or try to update. Working command openvpn --writepid /var/run/openvpn/openvpn.pid --script-security 3 --mute-replay-warnings --config /boot/openvpn/pfSense-udp-1198-Sister.ovpn Working config via command line dev tun persist-tun persist-key cipher AES-256-CBC auth SHA1 tls-client client resolv-retry infinite remote ######## 1198 udp lport 0 verify-x509-name "pfenseRand" name pkcs12 pfSense-udp-1198-Sister.p12 ns-cert-type server comp-lzo adaptive auth-user-pass /boot/config/plugins/openvpnclient/password.txt status /tmp/openvpn/openvpn-status.log

Got a perplexing problem with the VPN client driving me up a wall. The plugin is dynamically updating my .ovpn config file every time I try to start the VPN Client and it some how removes the dev tun option in the file. The vpn fails to start with, no dev option defined in your .ovpn file. I'll add it back and the plugin updates the .ovpn and removes it every time and then the vpn log complains about the entry not being there. This is what my config file looks like before I start the VPN persist-tun dev tun persist-key cipher AES-256-CBC auth SHA1 tls-client client resolv-retry infinite remote ######### 1198 udp lport 0 verify-x509-name "pfenseRand" name pkcs12 pfSense-udp-1198-Sister.p12 ns-cert-type server comp-lzo adaptive auth-user-pass This is what it looks like after the VPN fails to start and it updates my .ovpn file. It fails to start and says the dev tun/tap option is missing cause the plugin took it out.... persist-tun persist-key cipher AES-256-CBC auth SHA1 tls-client client resolv-retry infinite remote ############## 1198 udp lport 0 verify-x509-name "pfenseRand" name pkcs12 pfSense-udp-1198-Sister.p12 ns-cert-type server comp-lzo adaptive auth-user-pass /boot/config/plugins/openvpnclient/password.txt status /tmp/openvpn/openvpn-status Interestingly, I have another remote VPNserver using different options without a password and the plugin never trys to change the .ovpn file and it works fine. Any bright ideas?

hmmm I rebooted a few times to see how /dev/sda was getting assigned, seems the RDM assigned cache drive gets assigned as /dev/sda everytime. Since it's being passed through as a Raw device mapping, it seems to be seen first in the boot sequence before all my drives attached to the passed through I/O card does seance it has to load the I/O card driver little later in the boot sequence. I'll have to keep this in mind.

So I thought about what Linix was doing here and the device-id's are nothing more then symbolic links to the /dev/sda file. the sda file is getting picked up every time as the cache drive is manually mountable this way. I looked at how ESXi was listing the disk at it's Linux device level, looked at the naming format in unRAID and created my own sym links via the go file. ln -s /dev/sda1 /dev/disk/by-id/ata-LITEONIT_LCS2D256M6S_2.5_7mm_256GB_TW0XFJWX5508534T1145-part1 ln -s /dev/sda /dev/disk/by-id/ata-LITEONIT_LCS2D256M6S_2.5_7mm_256GB_TW0XFJWX5508534T1145 After doing this, unRAID picked up my manually populated disk ID's and I was able to assign the cache drive. The drive is now being used as a cache and sharing the cache share properly with out having to manually mount it. Dirty work around but it works. I may try my original idea of assigning vmdk disks sourced from different VMware store SSD drives and see if this is feasible. I was going to create a cache pool this way. I don't know if anyone else out there is setup this way via ESXi however shall see how it goes.

Confirmed, all drives picked up and no driver load error.

Special thank you for sp1k3ster as well

I run ESXi 6.0 with Plop boot manager as well. Will give this a go for sure, thanks for sharing it here, much obliged.

I had this working before and don't know what happened. I decided to do a little experimenting with my Raw Device passthrough disk cache drive on my backup unRAID virtual machine first. The cache was directly attached to the VM at one point and I moved it off a SAS card attached to the VM and attached it to the motherboard to free up a SAS I/O card connection for a data drive. I RDM mapped it and all was well. I loose temp sensors and such this way but for a SSD cache disk, I was good with that. Then I thought about making that cache disk a VMware data store disk instead and pass a traditional vmdk disk as my caching disk just to see if it can be done. There was alot of wasted space on the disk the way I was using it that made this option attractive. I rsynced my RDM cache disk into the raid array and removed it. When assigning a typical vmdk, it failed to show up by disk-id and when assigning a device as a cache disk, unraid GUI just showed sda and the disk size with no other identifying characteristics. Assigning the cache disk wouldn't stick and the web page constantly showed the cache drive as unassigned after just assigning the disk. Hmmmm not desirable I say, so I back it all out. Removed the SSD drive from vmware's storage pool and setup a RDM disk again and the same thing happened again and it was just working before.... I manually formatted, mounted and restored my cache disk data which got my dockers up however unRAID does not see it as a cache disk via the GUI. See below, /dev/sda is there and mounted manually but with no identifiers. Any ideas? Is it possible to manually populate this data with a /proc file system hack? root@unRAIDbackup:/dev# ls -al /dev/disk/by-id | grep sda root@unRAIDbackup:/dev# ls -al /dev/disk/by-uuid | grep sda root@unRAIDbackup:/dev# cat /etc/mtab | grep sda /dev/sda1 /mnt/cache btrfs rw 0 0 root@unRAIDbackup:/dev# btrfs fi show /mnt/cache Label: none uuid: c3a77508-ff82-4fc5-a147-e0eb90812706 Total devices 1 FS bytes used 61.95GiB devid 1 size 238.47GiB used 88.04GiB path /dev/sda1 btrfs-progs v4.1.2 root@unRAIDbackup:/dev# fdisk -l /dev/sda WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted. Disk /dev/sda: 256.1 GB, 256060514304 bytes 168 heads, 63 sectors/track, 47252 cylinders, total 500118192 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sda1 2048 500118191 250058072 83 Linux root@unRAIDbackup:/dev# root@unRAIDbackup:/dev# ls -al /dev/disk/by-id | grep sda root@unRAIDbackup:/dev# ls -al /dev/disk/by-uuid | grep sda root@unRAIDbackup:/dev# cat /etc/mtab | grep sda /dev/sda1 /mnt/cache btrfs rw 0 0 root@unRAIDbackup:/dev# btrfs fi show /mnt/cache Label: none uuid: c3a77508-ff82-4fc5-a147-e0eb90812706 Total devices 1 FS bytes used 61.95GiB devid 1 size 238.47GiB used 88.04GiB path /dev/sda1 btrfs-progs v4.1.2 root@unRAIDbackup:/dev# fdisk -l /dev/sda WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted. Disk /dev/sda: 256.1 GB, 256060514304 bytes 168 heads, 63 sectors/track, 47252 cylinders, total 500118192 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sda1 2048 500118191 250058072 83 Linux root@unRAIDbackup:/dev# hdparm -I /dev/sda /dev/sda: SG_IO: bad/missing sense data, sb[]: 70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ATA device, with non-removable media Serial Number: <@??? Standards: Likely used: 1 Configuration: Logical max current cylinders 0 0 heads 0 0 sectors/track 510 0 -- Logical/Physical Sector size: 512 bytes device size with M = 1024*1024: 0 MBytes device size with M = 1000*1000: 0 MBytes cache/buffer size = unknown Capabilities: IORDY not likely Cannot perform double-word IO R/W multiple sector transfer: not supported DMA: not supported PIO: pio0

I got too many other VM's to go to bare unraid. I run unraid as 2 separate VM's to manage my primary and backup raid array. I'll be running 6.1.9 for a long time till this is resolved.

I have m1015 IBM SAS cards flashed to IT mode passed through via ESXi and I got a mpt2sas driver load failure in my boot up and no disks present. Resolved by backing out the change. Just an FYI

/mnt/disks/ST3000DM001-1CH166_Z1F64H2W <== before /mnt/disks/ST3000DM001_1CH166_Z1F64H2W <=== after, note the underscore is used instead of a hyphen. Interestingly the WebGUI still shows the hyphen. If you click and expand the device in the GUI, it shows you the mount which uses a underscore instead in the device name.

and yes rebooted like 12 times

son of a gun I think I found it. I assumed everything in mnt was a disk and I was wrong. Looks like an unassigned devices update changed the name of my disks that are mounted outside of the RAID array ever so slightly. For this setup I have a rsync job that does a disk to disk backup sync to disks mounted with unassigned devices. Looks like I am now syncing to the memory file system and the job is creating the disk directory rather then sync to a disk., That'll run the root file system out of space in a hurry.

I am working on a family members system and actually had this problem myself caused by the file integrity plugin which is not present. I've removed 90% of all plugins and all but 3 essential dockers and and I can't get this issue to stop. I even have docker disabled for now. By the time it gets full, many of my diagnostic commands error out with an I/O warning file system full message. To catch this while it was happening and before the file system got full, I had to put in a cron job. Running the below command in the cron job fails to stay within the root file system like it should and I see all sorts of results in like /mnt/cache. That made the results largely useless. find / -type d -xdev -exec du -a {} + | sort -n -r Anyone have any ideas on what I can do to isolate this? I manually did du's on all my root folders that I know are used by the memory based root file system and not a disk or flash drive but nothing shows up as taking space here. root@BernadettesSvr:/# ls bin/ boot/ dev/ etc/ home/ init@ lib/ lib64/ mnt/ proc/ root/ run/ sbin/ sys/ tmp/ usr/ var/ root@BernadettesSvr:/# du -sm bin 10 bin root@BernadettesSvr:/# du -sm dev 0 dev root@BernadettesSvr:/# du -sm etc 5 etc root@BernadettesSvr:/# du -sm home 0 home root@BernadettesSvr:/# du -sm lib 15 lib root@BernadettesSvr:/# du -sm lib64 20 lib64 root@BernadettesSvr:/# du -sm root 1 root root@BernadettesSvr:/# du -sm proc du: cannot access ‘proc/7972/task/7972/fd/3’: No such file or directory du: cannot access ‘proc/7972/task/7972/fdinfo/3’: No such file or directory du: cannot access ‘proc/7972/fd/4’: No such file or directory du: cannot access ‘proc/7972/fdinfo/4’: No such file or directory 0 proc root@BernadettesSvr:/# du -sm run 1 run root@BernadettesSvr:/# du -sm sbin 15 sbin root@BernadettesSvr:/# du -sm sys du: cannot access ‘sys/kernel/slab/unraid/md’: No such file or directory 0 sys root@BernadettesSvr:/# du -sm tmp 1 tmp root@BernadettesSvr:/# du -sm usr 190 usr root@BernadettesSvr:/# du -sm var 6 var root@BernadettesSvr:/# df -h / Filesystem Size Used Avail Use% Mounted on - 2.9G 2.8G 164M 95% / I can't see where the space is at, like it's invisible or something. I can't use lsof to see what processes are using the file system since to run it against the root file system produces far too big of a list. These are the only plugins left attached showing there versions. Docker remains disabled for problem isolation. I am working on the system over VPN which is why the openVPN plugin remains. Any bright ideas? The fact I disabled so many components and the problem remains is a little concerning.

well if you are referring to my post peter it appears the VPN resets when the VPN times out and resablishes. It starts backup with settings that do not match what is in the plugin GUI, namely the extended routing for exclusive IP's only.

d.bech, What's not as clear as it could be in other docs is essentially this. Create a openvpn folder on your flash. Drop your .ovpn config file in this directory. The plugin does not create this folder or a sample file to work with even though it probably should. In the GUI, select the .ovpn file you created that has the majority of your vpn settings in it. These settings are not in the web interface but edited in this config file. Download a sample file to get you started on what the file looks like and what it does. Once the file is in the right folder and named correctly, the web GUI can select it as your "VPN" profile per say. If you are using CA certs or client authentication certificates you must create on your own, they also should be dropped in this folder and referenced in your config file. Also if you want to use extended routing, you have to create a webaddress.txt file in the openvpn folder. it will read this file and create routing table entries for when the VPN starts. No where in the GUI will it create this file for you or edit it. Must be done manually.

So my VPN kept reasserting itself with a default route to send all traffic over the VPN when I have extended routing for specific IP's only selected. It would also clear my specific IP host routes from my routing table read from the Webbaddresses.txt file I had to adjust the metric of my normal default route to take priority over the VPN default route however when this happens the route entry for specific IP's would get cleared out too. I found the cause in the logs and when the default route I don't want gets inserted, Any ideas on how to make sure the extended routing of specific IP's only sticks when the VPN resets/connects? I might have to create a cron job that runs like every min to read the routing table and insert the routes manually if there is no other fix to this. Thanks Log Tue Mar 1 01:20:50 2016 [vpn] Inactivity timeout (--ping-restart), restarting Tue Mar 1 01:20:50 2016 SIGUSR1[soft,ping-restart] received, process restarting Tue Mar 1 01:20:50 2016 Restart pause, 2 second(s) Tue Mar 1 01:20:52 2016 Socket Buffers: R=[212992->425984] S=[212992->212992] Tue Mar 1 01:20:52 2016 TCP/UDP: Preserving recently used remote address: [AF_INET]209.197.20.207:1194 Tue Mar 1 01:20:52 2016 UDPv4 link local: [undef] Tue Mar 1 01:20:52 2016 UDPv4 link remote: [AF_INET]209.197.20.207:1194 Tue Mar 1 01:21:07 2016 TLS: Initial packet from [AF_INET]209.197.20.207:1194, sid=883df6eb a9137f82 Tue Mar 1 01:21:07 2016 VERIFY OK: depth=1, C=US, ST=VPN, L=VPN, O=VPN, OU=VPN, CN=VPN, name=VPN, emailAddress=VPN Tue Mar 1 01:21:07 2016 Validating certificate key usage Tue Mar 1 01:21:07 2016 ++ Certificate has key usage 00a0, expects 00a0 Tue Mar 1 01:21:07 2016 VERIFY KU OK Tue Mar 1 01:21:07 2016 Validating certificate extended key usage Tue Mar 1 01:21:07 2016 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication Tue Mar 1 01:21:07 2016 VERIFY EKU OK Tue Mar 1 01:21:07 2016 VERIFY OK: depth=0, C=US, ST=VPN, L=VPN, O=VPN, OU=VPN, CN=vpn, name=VPN Tue Mar 1 01:21:07 2016 Data Channel Encrypt: Cipher 'AES-256-CBC' initialized with 256 bit key Tue Mar 1 01:21:07 2016 Data Channel Encrypt: Using 256 bit message hash 'SHA256' for HMAC authentication Tue Mar 1 01:21:07 2016 Data Channel Decrypt: Cipher 'AES-256-CBC' initialized with 256 bit key Tue Mar 1 01:21:07 2016 Data Channel Decrypt: Using 256 bit message hash 'SHA256' for HMAC authentication Tue Mar 1 01:21:07 2016 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-GCM-SHA384, 2048 bit RSA Tue Mar 1 01:21:07 2016 [vpn] Peer Connection Initiated with [AF_INET]209.197.20.207:1194 Tue Mar 1 01:21:09 2016 SENT CONTROL [vpn]: 'PUSH_REQUEST' (status=1) Tue Mar 1 01:21:09 2016 PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1 bypass-dhcp,dhcp-option DNS 198.18.0.1,dhcp-option DNS 198.18.0.2,rcvbuf 262144,explicit-exit-notify 5,route-gateway 172.21.80.1,topology subnet,ping 20,ping-restart 40,ifconfig 172.21.80.5 255.255.254.0' Tue Mar 1 01:21:09 2016 OPTIONS IMPORT: timers and/or timeouts modified Tue Mar 1 01:21:09 2016 OPTIONS IMPORT: explicit notify parm(s) modified Tue Mar 1 01:21:09 2016 OPTIONS IMPORT: --sndbuf/--rcvbuf options modified Tue Mar 1 01:21:09 2016 Socket Buffers: R=[425984->425984] S=[212992->212992] Tue Mar 1 01:21:09 2016 OPTIONS IMPORT: --ifconfig/up options modified Tue Mar 1 01:21:09 2016 OPTIONS IMPORT: route options modified Tue Mar 1 01:21:09 2016 OPTIONS IMPORT: route-related options modified Tue Mar 1 01:21:09 2016 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified Tue Mar 1 01:21:09 2016 Preserving previous TUN/TAP instance: tun5 Tue Mar 1 01:21:09 2016 NOTE: Pulled options changed on restart, will need to close and reopen TUN/TAP device. Tue Mar 1 01:21:09 2016 /usr/sbin/ip route del 209.197.20.207/32 Tue Mar 1 01:21:09 2016 /usr/sbin/ip route del 0.0.0.0/1 RTNETLINK answers: No such process Tue Mar 1 01:21:09 2016 ERROR: Linux route delete command failed: external program exited with error status: 2 Tue Mar 1 01:21:09 2016 /usr/sbin/ip route del 128.0.0.0/1 RTNETLINK answers: No such process Tue Mar 1 01:21:09 2016 ERROR: Linux route delete command failed: external program exited with error status: 2 Tue Mar 1 01:21:09 2016 Closing TUN/TAP interface Tue Mar 1 01:21:09 2016 /usr/sbin/ip addr del dev tun5 172.20.20.18/22 Tue Mar 1 01:21:10 2016 ROUTE_GATEWAY 192.168.2.1/255.255.255.0 IFACE=eth0 HWADDR=00:0c:29:70:79:ac Tue Mar 1 01:21:10 2016 TUN/TAP device tun5 opened Tue Mar 1 01:21:10 2016 TUN/TAP TX queue length set to 100 Tue Mar 1 01:21:10 2016 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0 Tue Mar 1 01:21:10 2016 /usr/sbin/ip link set dev tun5 up mtu 1500 Tue Mar 1 01:21:10 2016 /usr/sbin/ip addr add dev tun5 172.21.80.5/23 broadcast 172.21.81.255 Tue Mar 1 01:21:10 2016 /usr/sbin/ip route add 209.197.20.207/32 via 192.168.2.1 Tue Mar 1 01:21:10 2016 /usr/sbin/ip route add 0.0.0.0/1 via 172.21.80.1 Tue Mar 1 01:21:10 2016 /usr/sbin/ip route add 128.0.0.0/1 via 172.21.80.1

Btw if anyone knows how to bounce a docker as easily as a plex service, I can update this to support version 6 however the issue I created this script for has never surfaced in the mature versions of Unraidv6 yet.

I was tracking why my unraid VM's both had constant 20-30% CPU showing up as kernel time and no active visible PID's driving it. This appears to have resolved that problem as well. I started using the integrity plugin instead I been meaning to setup anyway. I'm rather happy I knocked out both problems.

Ha my primary VM was impacted too, didn't realize it was growing the last few days since it has a much larger memory file system. Before root@unRAIDprime:~# df -h / Filesystem Size Used Avail Use% Mounted on - 7.4G 7.3G 57M 100% / After the uninstall root@unRAIDprime:~# df -h / Filesystem Size Used Avail Use% Mounted on - 7.4G 582M 6.8G 8% / Good deal, where's your tip jar? You deserve a dollar and a coke