I have updated to ESXi 6.5 U2 (May 2019 Patch) in the meantime and retried installing the compiled tools after deleting as you suggested.
The plugin install still fails the first time:
plugin: installing: https://raw.githubusercontent.com/StevenDTX/unRAID-open-vm-tools/master/openVMTools_compiled.plg
plugin: downloading https://raw.githubusercontent.com/StevenDTX/unRAID-open-vm-tools/master/openVMTools_compiled.plg
plugin: downloading: https://raw.githubusercontent.com/StevenDTX/unRAID-open-vm-tools/master/openVMTools_compiled.plg ... done
Kernel Version: 4.18.20-unRAID
plugin: run failed: /bin/bash retval: 8
And then succeeds on the second attempt:
plugin: installing: https://raw.githubusercontent.com/StevenDTX/unRAID-open-vm-tools/master/openVMTools_compiled.plg
plugin: downloading https://raw.githubusercontent.com/StevenDTX/unRAID-open-vm-tools/master/openVMTools_compiled.plg
plugin: downloading: https://raw.githubusercontent.com/StevenDTX/unRAID-open-vm-tools/master/openVMTools_compiled.plg ... done
Kernel Version: 4.18.20-unRAID
plugin: downloading: https://raw.githubusercontent.com/StevenDTX/unRAID-open-vm-tools/master/packages/libdnet-1.12-x86_64-6cf.txz ... done
+==============================================================================
| Installing new package /boot/config/plugins/OpenVMTools_compiled/packages/libdnet-1.12-x86_64-6cf.txz
+==============================================================================
Verifying package libdnet-1.12-x86_64-6cf.txz.
Installing package libdnet-1.12-x86_64-6cf.txz:
PACKAGE DESCRIPTION:
# libdnet
#
# libdnet provides a simplified, portable interface to several
# low-level networking routines, including
#
Executing install script for libdnet-1.12-x86_64-6cf.txz.
Package libdnet-1.12-x86_64-6cf.txz installed.
plugin: downloading: https://raw.githubusercontent.com/StevenDTX/unRAID-open-vm-tools/master/packages/libffi-3.2.1-x86_64-2.txz ... done
+==============================================================================
| Skipping package libffi-3.2.1-x86_64-2 (already installed)
+==============================================================================
Starting VMWare Tools Daemon.
-----------------------------------------------------------
Plugin OpenVMTools_compiled is installed.
Plugin Version: 2019.05.17
-----------------------------------------------------------
plugin: installed
ESXi still does not recognize any tools running though, inspecting processes no VMware Tools Daemon is running. Looking at the packages subfolder lists two libraries as from the installation log, but a 0 byte "open_vm_tools-10.3.5-4.18.20-Unraid-x86_64.tgz" which I assume is the problem.
Looking at your install script and repository, I have found the issue. The kernel of 6.6.7 is "4.18.20-unRAID", and your repository correctly contains "open_vm_tools-10.3.5-4.18.20-unRAID-x86_64.tgz". Your script on the other hand attempts to download and install "open_vm_tools-10.3.5-4.18.20-Unraid-x86_64.tgz", which gives an error 404 due to the case mismatch in the filename.
Manually downloading and installing with the correct filename yielded running tools, which also work after reboot. So it all comes down to that one typo of "Unraid" vs. "unRAID".