Python 3.11 Unraid TXZ Package/Plugin

kubed_zero · July 31, 2023

I've been trying to get Python 3.11 installed on Unraid 6.11 or 6.12. I looked around for pre-compiled TXZ packages, but only found 3.9.x packages from @dlandon in https://github.com/dlandon/python3 and @EUGENI_CAT in https://github.com/UnRAIDES/unRAID-NerdTools/blob/main/packages/6.11/python3-3.9.16-x86_64-1.txz

Usually for Slackware package compilation, a SlackBuild file is provided to configure the system before calling the package's make file. The newest SlackBuild I could find out there was for 3.9.5 http://www.slackware.com/~alien/slackbuilds/python3/build/python3.SlackBuild and it is not Unraid specific.

I was able to install a bunch of support packages on Unraid 6.12.3 and get the SlackBuild working:

Python-3.11.4.tar.xz             libmpc-1.3.1-x86_64-1.txz
binutils-2.40-x86_64-1.txz       libzip-1.10.0-x86_64-1.txz
expat-2.5.0-x86_64-1.txz         lzlib-1.13-x86_64-1.txz
gc-8.2.4-x86_64-1.txz            make-4.4.1-x86_64-1.txz
gcc-13.1.0-x86_64-2.txz          openssl-3.1.1-x86_64-1.txz
gcc-g++-13.1.0-x86_64-2.txz      openssl-solibs-3.1.1-x86_64-1.txz
git-2.41.0-x86_64-1.txz          openssl11-1.1.1u-x86_64-1.txz
glibc-2.37-x86_64-2.txz          openssl11-solibs-1.1.1u-x86_64-1.txz
guile-3.0.9-x86_64-1.txz         pkg-config-0.29.2-x86_64-4.txz
kernel-headers-6.1.35-x86-1.txz  python3.SlackBuild
libffi-3.4.4-x86_64-1.txz        zlib-1.2.13-x86_64-1.txz

However, I've had some issues after installing this custom-built Python. I bundled Pip into this install since that's supported with a makefile flag, but I started seeing "Can't connect to HTTPS URL because the SSL module is not available." after rebooting Unraid. I suspect this is because the SSL package and system libraries were installed during compilation, so Python doesn't reference its internal SSL library and instead uses the system library, which after an Unraid reboot no longer exists.

I've seen a couple other issues along this line as well.

I'm hoping someone here is aware of an Unraid-compatible Python 3.11 TXZ file, or a SlackBuild file that could get Python compiled and functional on Unraid. Or if someone here simply has more experience with compiling/SlackBuild/makefiles on Unraid, maybe that would also work.

Thanks!

kubed_zero · August 3, 2023

All right, I got things working.

Starting with Python 3.10, OpenSSL 1.1.1 or newer is required https://peps.python.org/pep-0644/ which means the OpenSSL 1.1 installed by default in Unraid 6.12.3 does not meet the requirements. In other words, using Python 3.10 or newer will require OpenSSL to also be installed as a prerequisite. I got OpenSSL 3.1.2 from https://slackware.uk/slackware/slackware64-current/slackware64/n/openssl-3.1.2-x86_64-1.txz and added it to the /boot/extra/ directory to install at boot.

I also spent a bunch of time working through the different settings and libraries needed to configure and build/compile Python 3.11.4 on Unraid. I eventually succeeded, and am happy to share the 3.11.4 TXZ file. I put this in the /boot/extra/ directory alongside OpenSSL and am happy to report that Python seems to be working for my needs. I included Pip in the TXZ as well, so Pip will install automatically next to Python and does not need a separate package to install

As far as I'm aware, I'm the only person to have successfully/publicly shared a Python 3.10 or 3.11 built for Slackware or Unraid. With that in mind, here are some notes I took along the way:

I read through http://www.slackware.com/~alien/slackbuilds/python3/build/python3.SlackBuild for inspiration for a SlackBuild file, but did not fully end up using it

My required package dependency for building steps
- gcc https://slackware.uk/slackware/slackware64-current/slackware64/d/gcc-13.2.0-x86_64-1.txz and `installpkg gcc-13.2.0-x86_64-1.txz ` to fix `configure: error: no acceptable C compiler found in $PATH`
- binutils https://slackware.uk/slackware/slackware64-current/slackware64/d/binutils-2.41-x86_64-1.txz to fix `configure: error: C compiler cannot create executables` which stems from `gcc: fatal error: cannot execute 'as': execvp: No such file or directory` https://stackoverflow.com/questions/56801179/fatal-error-cannot-execute-as-execvp-no-such-file-or-directory
- glibc https://slackware.uk/slackware/slackware64-current/slackware64/l/glibc-2.37-x86_64-2.txz to fix `configure: error: C compiler cannot create executables` with sub-error `/ld: cannot find crt1.o: No such file or directory`
- kernel-headers https://slackware.uk/slackware/slackware64-current/slackware64/d/kernel-headers-6.1.42-x86-1.txz to fix `C preprocessor "/lib/cpp" fails sanity check` with sub-error `linux/limits.h: No such file or directory`
- **Everything above here is the minimal set needed to get `./configure` running end to end**
- make https://slackware.uk/slackware/slackware64-current/slackware64/d/make-4.4.1-x86_64-1.txz obviously just to run make
- guile https://slackware.uk/slackware/slackware64-current/slackware64/d/guile-3.0.9-x86_64-1.txz to fix the make-time error `make: error while loading shared libraries: libguile-3.0.so.1: cannot open shared object file: No such file or directory`
- gc https://slackware.uk/slackware/slackware64-current/slackware64/l/gc-8.2.4-x86_64-1.txz to fix the make-time error `make: error while loading shared libraries: libgc.so.1: cannot open shared object file: No such file or directory`
- zlib https://slackware.uk/slackware/slackware64-current/slackware64/l/zlib-1.2.13-x86_64-1.txz RIP after 45m of compiling I saw an error `zipimport.ZipImportError: can't decompress data; zlib not available`
- libffi https://slackware.uk/slackware/slackware64-current/slackware64/l/libffi-3.4.4-x86_64-1.txz to fix compilation error `Failed to build these modules: _ctypes` and `_ctypes.c:118:10: fatal error: ffi.h: No such file or directory`
- openssl (not openssl11, which corresponds to 1.1 and not the 3.x of normal openssl) https://slackware.uk/slackware/slackware64-current/slackware64/n/openssl-3.1.1-x86_64-1.txz to let `./configure` know that SSL exists so that hopefully Python builds the SSL module `checking for openssl/ssl.h in /usr... yes / checking whether compiling and linking against OpenSSL works... yes`
- I see `/usr/bin/openssl` and `/usr/lib64/libevent_openssl.so` and `/usr/lib64/libssl.so` seem to exist on the test Unraid, so maybe it's safe to assume it's available by default? I suspect pkg-config will aid in detecting that it's available
- Nope, I see `checking whether compiling and linking against OpenSSL works... no` when running `/configure`
- Yeah, even when running make I see the same `Could not build the ssl module! Python requires a OpenSSL 1.1.1 or newer` error
- I see that `./configure` has a log line `checking for stdlib extension module _ssl... missing` which makes me suspect that SSL won't be built into this Python package installer, and thus pip is going to fail once I get to that point. I bet I could get around that by installing SSL using the instructions from earlier, but then I suspect it won't really be bundled and will instead just look to the system for an SSL implementation. That said, I wonder if I could install pkg-config AND SSL and set the https://docs.python.org/3/using/configure.html#cmdoption-with-openssl-rpath `--with-openssl-rpath=auto` flag during configuration and then it would just use whatever SSL is available? The alternative is to just build this, realize SSL is missing, install the SSL package as well, and call it a day. I think it should technically be fine if the python installer also depends on SSL
- pkg-config to resolve a general compiler warning https://slackware.uk/slackware/slackware64-current/slackware64/d/pkg-config-0.29.2-x86_64-4.txz that `WARNING: pkg-config is missing. Some dependencies may not be detected correctly.`
- bzip2 https://slackware.uk/slackware/slackware64-current/slackware64/a/bzip2-1.0.8-x86_64-3.txz to fix `checking for stdlib extension module _bz2... missing` in the `./configure`
- xz https://slackware.uk/slackware/slackware64-current/slackware64/a/xz-5.4.3-x86_64-1.txz to fix to fix `checking for stdlib extension module _lzma... missing` in the `./configure`
- gdbm https://slackware.uk/slackware/slackware64-current/slackware64/l/gdbm-1.23-x86_64-1.txz to fix `checking for stdlib extension module _gdbm... missing` in the `./configure`
- ncurses https://slackware.uk/slackware/slackware64-current/slackware64/l/ncurses-6.4_20230610-x86_64-1.txz to fix failed compilation of the `_ncurses` module

**My steps**
- Download python 3 source in xz format to `/boot/buildPython/Python-3.11.4.tar.xz`
- Create a temp directory to do work in, and move there: `rm -rf /tmp/buildPython | exit 0 && mkdir /tmp/buildPython && cd /tmp/buildPython`
- Install the dependencies needed to run `./configure` and `make` in `/boot/buildPython`
- Extract the Python source to the temp directory `tar xf /boot/buildPython/Python-3.11.4.tar.xz` which creates a subdirectory `/tmp/buildPython/Python-3.11.4/`
- Move into that subdirectory `cd /tmp/buildPython/Python-3.11.4/`
- Try to run `./configure --build=x86_64-slackware-linux --with-ensurepip=upgrade --prefix=/usr --libdir=/usr/lib64 --with-platlibdir=lib64 --enable-optimizations --with-lto --with-pkg-config=yes --disable-test-modules --without-static-libpython`
- Run make with multiple jobs (assuming multiple CPUs) to compile faster: `make -j6`
- Clear and create the directory that `make install` will output into `rm -rf /tmp/package-python-make-output | exit 0 && mkdir -p /tmp/package-python-make-output`
- Run make with `make install DESTDIR=/tmp/package-python-make-output`
- Now remove some of the files in `/tmp/package-python-make-output` that should not be packaged:
- `cd /tmp/package-python-make-output/`
- `find . $ -name '*.exe' -o -name '*.bat' $ -exec rm -f '{}' \+` to remove windows stuff
- `find . -type d -exec chmod 755 "{}" \+` to update permissions
- `find . -perm 640 -exec chmod 644 "{}" \+`
- `find . -perm 750 -exec chmod 755 "{}" \+`
- `find . -print0 | xargs -0 file | grep -e "executable" -e "shared object" | grep ELF | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null || true` I have no idea what this does but it seemed to find a couple files such as `./usr/lib64/python3.11/lib-dynload/termios.cpython-311-x86_64-linux-gnu.so` or `./usr/lib64/python3.11/lib-dynload/_codecs_tw.cpython-311-x86_64-linux-gnu.so` so maybe it's removing extended attributes? IDK
- `strip -s usr/lib/* usr/lib64/* usr/bin/*` as recommended by https://docs.slackware.com/howtos:slackware_admin:building_a_package
- `mkdir install && cd install && wget https://www.slackbuilds.org/slackbuilds/14.2/python/python3/slack-desc` to add a `slack-desc` file to the package
- edit the usr/bin/pip3 and other pip files to point to python3 instead of just python, as this package doesn't install a default python
- `cd /tmp/package-python-make-output && /sbin/makepkg -l y -c n /tmp/python3-3.11.4-x86_64-1.txz` to actually make the output Slackware package using all the files in the `/tmp/package-python-make-output` directory

python3-3.11.4-x86_64-20230802kubedwithpip.txz

kubed_zero · December 11, 2023

I just finished building Python 3.12.1 using similar instructions to above.

I now run "find usr/lib*/python* -name '*.so' | xargs strip --strip-unneeded" instead of "find . -print0 | xargs -0 file | grep -e "executable" -e "shared object" | grep ELF | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null || true"
I used a longer list of packages installed at compile time, some of these not needed for Python but needed for BorgBackup
I found that using newer versions of OpenSSL and glibc would allow for successful compilation, but failures during executions of Python with errors such as "python3: /lib64/libm.so.6: version GLIBC_2.38' not found (required by python3)" and sure enough, Unraid 6.12.6 has only "/lib64/libm-2.37.so"

Python-3.12.1.tar.xz             libffi-3.4.4-x86_64-1.txz
acl-2.3.1-x86_64-1.txz           libmpc-1.3.1-x86_64-1.txz
binutils-2.41-x86_64-1.txz       libzip-1.10.1-x86_64-1.txz
bzip2-1.0.8-x86_64-3.txz         lzlib-1.13-x86_64-1.txz
expat-2.5.0-x86_64-1.txz         make-4.4.1-x86_64-1.txz
fuse3-3.15.0-x86_64-1.txz        openssl-3.1.1-x86_64-1.txz
gc-8.2.4-x86_64-1.txz            openssl-solibs-3.1.1-x86_64-1.txz
gcc-13.2.0-x86_64-1.txz          openssl11-1.1.1w-x86_64-1.txz
gcc-g++-13.2.0-x86_64-1.txz      openssl11-solibs-1.1.1w-x86_64-1.txz
gdbm-1.23-x86_64-1.txz           pkg-config-0.29.2-x86_64-4.txz
git-2.43.0-x86_64-1.txz          readline-8.2.001-x86_64-1.txz
glibc-2.37-x86_64-2.txz          xz-5.4.3-x86_64-1.txz
guile-3.0.9-x86_64-1.txz         zlib-1.2.13-x86_64-1.txz
kernel-headers-6.1.64-x86-1.txz

I've attached the compiled python3-3.12.1 TXZ file, as well as the compiled pyfuse3-3.3.0 and borgbackup-1.2.7 WHL files that can be installed with Pip and are not otherwise available for Unraid. As far as I've tested in my use cases, they work as expected.

Below are my notes on the Python Wheel compilation for borgbackup and pyfuse3:

## 2023-12-10 Pyfuse3 Update
- Installing pyfuse3 before BorgBackup, as it's a dependency of Borg
- Installing some Slackware packages first:
```
binutils-2.41-x86_64-1.txz glibc-2.37-x86_64-2.txz
fuse3-3.16.2-x86_64-1.txz kernel-headers-6.1.64-x86-1.txz
gcc-13.2.0-x86_64-1.txz pkg-config-0.29.2-x86_64-4.txz
```

- Then run `pip3 install pyfuse3` which is building 3.3.0
- It installs a few pip3 packages as dependencies:
```
trio-0.23.1-py3-none-any.whl
attrs-23.1.0-py3-none-any.whl
sortedcontainers-2.4.0-py2.py3-none-any.whl
idna-3.6-py3-none-any.whl
outcome-1.3.0.post0-py2.py3-none-any.whl
sniffio-1.3.0-py3-none-any.whl

Successfully installed attrs-23.1.0 idna-3.6 outcome-1.3.0.post0 pyfuse3-3.3.0 sniffio-1.3.0 sortedcontainers-2.4.0 trio-0.23.1
```

- Permanently fetch all these packages (other than pyfuse3) with `pip3 download attrs==23.1.0 idna==3.6 outcome==1.3.0.post0 sniffio==1.3.0 sortedcontainers==2.4.0 trio==0.23.1` and put them in `/boot/python_wheels/` for offline installation at boot time
- Copy the `pyfuse3` wheel file from the directory stated in the build logs `cp /root/.cache/pip/wheels/b0/c1/b2/bd1f9969742c3b690d74ae13233287eec544c5a9135497443e/pyfuse3-3.3.0-cp312-cp312-linux_x86_64.whl /boot/python_wheels/`
- This would have appeared to be something similar to `Created wheel for pyfuse3: filename=pyfuse3-3.3.0-cp312-cp312-linux_x86_64.whl size=1283409 sha256=8471a14517ee73366b6b7fd7dc2921c57327830f0bb2d58a4569abf0e2ca50ad Stored in directory: /root/.cache/pip/wheels/b0/c1/b2/bd1f9969742c3b690d74ae13233287eec544c5a9135497443e`
- Note that renaming needs to follow some strict guidelines https://peps.python.org/pep-0491/#file-name-convention otherwise Pip will error out and give responses such as `ERROR: pyfuse3-3.3.0-cp312-cp312-linux_x86_64-kubed20231210.whl is not a valid wheel filename` or `ERROR: pyfuse3-3.3.0-cp312-cp312-linux_x86_64_kubed20231210.whl is not a supported wheel on this platform`
- Add a line to the `/boot/config/go` file to automatically install `pyfuse3` at boot, or just run it adhoc to confirm it works: `/usr/bin/pip3 install /boot/python_wheels/pyfuse3* --no-index --find-links file:///boot/python_wheels`
## 2023-12-10 Borg 1.2.7 Update
- OK, I'm now running Python 3.12.1 as compiled above
- I also have pyfuse3 installed in Pip in preparation
- (Sidenote) https://forums.unraid.net/topic/129200-plug-in-nerdtools/?do=findComment&comment=1291205 are my old instructions for compiling Borg
- (Sidenote) there's now a SlackBuild for installing Borg on Slackware! Updated in 2023Q3 which is more recent than the last time I updated this https://slackbuilds.org/slackbuilds/15.0/system/borgbackup/borgbackup.SlackBuild
- Following https://borgbackup.readthedocs.io/en/stable/installation.html#pip-installation generally
- First it notes to follow https://borgbackup.readthedocs.io/en/stable/installation.html#source-install and get some Slackware dependencies installed
- Intentionally skipping the installation of `libxxhash` and `libzstd` and `liblz4` as the desire is to use the bundled code instead of the system-provided libraries
- The instructions call for `acl` and `pkg-config` only, but the others are needed based on trial and error
```
acl-2.3.1-x86_64-1.txz glibc-2.37-x86_64-2.txz
binutils-2.41-x86_64-1.txz kernel-headers-6.1.64-x86-1.txz
gcc-13.2.0-x86_64-1.txz pkg-config-0.29.2-x86_64-4.txz
```
- Then it asks to install some Pip dependencies JUST for the build process: `wheel`, `setuptools`, `pkgconfig`
- This can be done with `pip3 download pkgconfig setuptools wheel && pip3 install *.whl`
```
pkgconfig-1.5.5-py3-none-any.whl wheel-0.42.0-py3-none-any.whl
setuptools-69.0.2-py3-none-any.whl
```
- Now run `pip3 install "borgbackup[pyfuse3]"` to install borgbackup and build the wheel. Note that this adds `pyfuse3` integration, but that can be skipped by running the simplified install command `pip3 install borgbackup`
- It installs a few pip3 packages as dependencies:
```
msgpack-1.0.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
packaging-23.2-py3-none-any.whl

Successfully installed borgbackup-1.2.7 msgpack-1.0.7 packaging-23.2
```

- Permanently fetch these packages (other than borgbackup) with `pip3 download msgpack==1.0.7 packaging==23.2` and put them in `/boot/python_wheels/` for offline installation at boot time
- Copy the `borgbackup` wheel file from the directory stated in the build logs `cp /root/.cache/pip/wheels/b0/c1/b2/bd1f9969742c3b690d74ae13233287eec544c5a9135497443e/pyfuse3-3.3.0-cp312-cp312-linux_x86_64.whl /boot/python_wheels/`
- This would have appeared to be something similar to `Created wheel for borgbackup: filename=borgbackup-1.2.7-cp312-cp312-linux_x86_64.whl size=6248226 sha256=830ec3fbcc5c74922f7e269e8378aa08052c8c9cec51f407c53166d989d62e7c Stored in directory: /root/.cache/pip/wheels/64/90/02/c60bc19558d5e1e7c5ed13041bc27d25d41518ddfdb2def852`
- Note that renaming needs to follow some strict guidelines https://peps.python.org/pep-0491/#file-name-convention otherwise Pip will error out
- Add a line to the `/boot/config/go` file to automatically install `pyfuse3` at boot, or just run it adhoc to confirm it works: `/usr/bin/pip3 install /boot/python_wheels/borgbackup* --no-index --find-links file:///boot/python_wheels`
- The final list of wheel files ready to install at boot time are:
```
attrs-23.1.0-py3-none-any.whl
borgbackup-1.2.7-cp312-cp312-linux_x86_64.whl
idna-3.6-py3-none-any.whl
msgpack-1.0.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
outcome-1.3.0.post0-py2.py3-none-any.whl
packaging-23.2-py3-none-any.whl
pyfuse3-3.3.0-cp312-cp312-linux_x86_64.whl
sniffio-1.3.0-py3-none-any.whl
sortedcontainers-2.4.0-py2.py3-none-any.whl
trio-0.23.1-py3-none-any.whl
```

pyfuse3-3.3.0-cp312-cp312-linux_x86_64.whl python3-3.12.1-x86_64-1-kubed20231210.txz borgbackup-1.2.7-cp312-cp312-linux_x86_64.whl

DjangoDurano · January 31

Hello.

I like your python builds for unraid, and I founded it at the right time.

So I had a question, is it possible that you provide your build packages?

As you mentioned it is necessary to have the correct packages to build, even not after build it will not run.

I tried to build some own python package, but the main problem for me is, to find older slackware packages, like openssl 3.1 or glibc 2.37 and so on.

If you know where I can find this older packages it's also fine.

Best regards

kubed_zero · January 31

14 minutes ago, DjangoDurano said:

If you know where I can find this older packages it's also fine

I think I was able to find everything I needed on https://slackware.uk/slackware/ but some general googling allowed me to find the older versions.

I've also uploaded some of the older packages as part of this post: glibc-2.37-x86_64-2.txz openssl-3.1.1-x86_64-1.txz

Let me know if you need anything else!

DjangoDurano · February 1

22 hours ago, kubed_zero said:

I think I was able to find everything I needed on https://slackware.uk/slackware/ but some general googling allowed me to find the older versions.

I've also uploaded some of the older packages as part of this post: glibc-2.37-x86_64-2.txz openssl-3.1.1-x86_64-1.txz

Let me know if you need anything else!

Thanks for your help.

I google for a long time but it's really hard to find older slackware packages. Older "stable" versions are can be found for example glibc 2.33 but other versions not.

I know https://slackware.uk/slackware/ but there are also only newer versions. The only thing is, that there are slackBuild scripts, maybe it is possible to build a previous version with that scripts.

kubed_zero · February 1

1 hour ago, DjangoDurano said:

Thanks for your help.

I google for a long time but it's really hard to find older slackware packages. Older "stable" versions are can be found for example glibc 2.33 but other versions not.

I know https://slackware.uk/slackware/ but there are also only newer versions. The only thing is, that there are slackBuild scripts, maybe it is possible to build a previous version with that scripts.

https://slackware.uk/search?p=%2F&q=glibc-2.37
https://slackware.uk/cumulative/slackware64-current/slackware64/l/glibc-2.37-x86_64-1.txz

https://slackware.uk/cumulative/slackware64-current/slackware64/l/glibc-2.37-x86_64-2.txz
https://slackware.uk/cumulative/slackware64-current/slackware64/l/glibc-2.37-x86_64-3.txz

Behold! With some digging through that website, old versions can be found

DjangoDurano · February 2

23 hours ago, kubed_zero said:

https://slackware.uk/search?p=%2F&q=glibc-2.37
https://slackware.uk/cumulative/slackware64-current/slackware64/l/glibc-2.37-x86_64-1.txz

https://slackware.uk/cumulative/slackware64-current/slackware64/l/glibc-2.37-x86_64-2.txz
https://slackware.uk/cumulative/slackware64-current/slackware64/l/glibc-2.37-x86_64-3.txz

Behold! With some digging through that website, old versions can be found

Oh. Didn't see the search field. Thanks for notice me.

Is there a problem, if I add your python packages as dependency to my python sleep script at github?

https://github.com/DjangoDurano/Unraid_Python_Sleep_Script

kubed_zero · February 2

36 minutes ago, DjangoDurano said:

Is there a problem, if I add your python packages as dependency to my python sleep script at github?

Thanks for checking! No problem, feel free to use it.

c0d3m0nk3y · August 5

Thanks for this. Finally let me upgrade a project I use across multiple systems now that I'm not limited to 3.9

It did take a little more than just installing the packs to get everything properly wired up. So if anyone runs into any issues, take a look at my install script

```

for p in $(find ./*.txz)
do
installpkg $p
done

if [[ -f /usr/bin/pip ]]
then
rm /usr/bin/pip
fi

ln -s /usr/bin/pip3 /usr/bin/pip

if [[ -f /usr/bin/python ]]
then
rm /usr/bin/python
fi

ln -s /usr/bin/python3.12 /usr/bin/python

pip install --upgrade pip
pip install --upgrade pipx

```

That loop installs the python 3.12 package I downloaded from here along with the glibc and openssl packages from

ich777 · August 6

You could also just try to install the iSCSI Target Plugin since it includes Python.

aspiziri · August 11

On 12/10/2023 at 8:54 PM, kubed_zero said:
I just finished building Python 3.12.1 using similar instructions to above.

I now run "find usr/lib*/python* -name '*.so' | xargs strip --strip-unneeded" instead of "find . -print0 | xargs -0 file | grep -e "executable" -e "shared object" | grep ELF | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null || true"

I used a longer list of packages installed at compile time, some of these not needed for Python but needed for BorgBackup

I found that using newer versions of OpenSSL and glibc would allow for successful compilation, but failures during executions of Python with errors such as "python3: /lib64/libm.so.6: version GLIBC_2.38' not found (required by python3)" and sure enough, Unraid 6.12.6 has only "/lib64/libm-2.37.so"
Python-3.12.1.tar.xz             libffi-3.4.4-x86_64-1.txz
acl-2.3.1-x86_64-1.txz           libmpc-1.3.1-x86_64-1.txz
binutils-2.41-x86_64-1.txz       libzip-1.10.1-x86_64-1.txz
bzip2-1.0.8-x86_64-3.txz         lzlib-1.13-x86_64-1.txz
expat-2.5.0-x86_64-1.txz         make-4.4.1-x86_64-1.txz
fuse3-3.15.0-x86_64-1.txz        openssl-3.1.1-x86_64-1.txz
gc-8.2.4-x86_64-1.txz            openssl-solibs-3.1.1-x86_64-1.txz
gcc-13.2.0-x86_64-1.txz          openssl11-1.1.1w-x86_64-1.txz
gcc-g++-13.2.0-x86_64-1.txz      openssl11-solibs-1.1.1w-x86_64-1.txz
gdbm-1.23-x86_64-1.txz           pkg-config-0.29.2-x86_64-4.txz
git-2.43.0-x86_64-1.txz          readline-8.2.001-x86_64-1.txz
glibc-2.37-x86_64-2.txz          xz-5.4.3-x86_64-1.txz
guile-3.0.9-x86_64-1.txz         zlib-1.2.13-x86_64-1.txz
kernel-headers-6.1.64-x86-1.txz
I've attached the compiled python3-3.12.1 TXZ file, as well as the compiled pyfuse3-3.3.0 and borgbackup-1.2.7 WHL files that can be installed with Pip and are not otherwise available for Unraid. As far as I've tested in my use cases, they work as expected.

Below are my notes on the Python Wheel compilation for borgbackup and pyfuse3:

## 2023-12-10 Pyfuse3 Update
- Installing pyfuse3 before BorgBackup, as it's a dependency of Borg
- Installing some Slackware packages first:
```
binutils-2.41-x86_64-1.txz glibc-2.37-x86_64-2.txz
fuse3-3.16.2-x86_64-1.txz kernel-headers-6.1.64-x86-1.txz
gcc-13.2.0-x86_64-1.txz pkg-config-0.29.2-x86_64-4.txz
```

- Then run `pip3 install pyfuse3` which is building 3.3.0
- It installs a few pip3 packages as dependencies:
```
trio-0.23.1-py3-none-any.whl
attrs-23.1.0-py3-none-any.whl
sortedcontainers-2.4.0-py2.py3-none-any.whl
idna-3.6-py3-none-any.whl
outcome-1.3.0.post0-py2.py3-none-any.whl
sniffio-1.3.0-py3-none-any.whl

Successfully installed attrs-23.1.0 idna-3.6 outcome-1.3.0.post0 pyfuse3-3.3.0 sniffio-1.3.0 sortedcontainers-2.4.0 trio-0.23.1
```

- Permanently fetch all these packages (other than pyfuse3) with `pip3 download attrs==23.1.0 idna==3.6 outcome==1.3.0.post0 sniffio==1.3.0 sortedcontainers==2.4.0 trio==0.23.1` and put them in `/boot/python_wheels/` for offline installation at boot time
- Copy the `pyfuse3` wheel file from the directory stated in the build logs `cp /root/.cache/pip/wheels/b0/c1/b2/bd1f9969742c3b690d74ae13233287eec544c5a9135497443e/pyfuse3-3.3.0-cp312-cp312-linux_x86_64.whl /boot/python_wheels/`
- This would have appeared to be something similar to `Created wheel for pyfuse3: filename=pyfuse3-3.3.0-cp312-cp312-linux_x86_64.whl size=1283409 sha256=8471a14517ee73366b6b7fd7dc2921c57327830f0bb2d58a4569abf0e2ca50ad Stored in directory: /root/.cache/pip/wheels/b0/c1/b2/bd1f9969742c3b690d74ae13233287eec544c5a9135497443e`
- Note that renaming needs to follow some strict guidelines https://peps.python.org/pep-0491/#file-name-convention otherwise Pip will error out and give responses such as `ERROR: pyfuse3-3.3.0-cp312-cp312-linux_x86_64-kubed20231210.whl is not a valid wheel filename` or `ERROR: pyfuse3-3.3.0-cp312-cp312-linux_x86_64_kubed20231210.whl is not a supported wheel on this platform`
- Add a line to the `/boot/config/go` file to automatically install `pyfuse3` at boot, or just run it adhoc to confirm it works: `/usr/bin/pip3 install /boot/python_wheels/pyfuse3* --no-index --find-links file:///boot/python_wheels`
## 2023-12-10 Borg 1.2.7 Update
- OK, I'm now running Python 3.12.1 as compiled above
- I also have pyfuse3 installed in Pip in preparation
- (Sidenote) https://forums.unraid.net/topic/129200-plug-in-nerdtools/?do=findComment&comment=1291205 are my old instructions for compiling Borg
- (Sidenote) there's now a SlackBuild for installing Borg on Slackware! Updated in 2023Q3 which is more recent than the last time I updated this https://slackbuilds.org/slackbuilds/15.0/system/borgbackup/borgbackup.SlackBuild
- Following https://borgbackup.readthedocs.io/en/stable/installation.html#pip-installation generally
- First it notes to follow https://borgbackup.readthedocs.io/en/stable/installation.html#source-install and get some Slackware dependencies installed
- Intentionally skipping the installation of `libxxhash` and `libzstd` and `liblz4` as the desire is to use the bundled code instead of the system-provided libraries
- The instructions call for `acl` and `pkg-config` only, but the others are needed based on trial and error
```
acl-2.3.1-x86_64-1.txz glibc-2.37-x86_64-2.txz
binutils-2.41-x86_64-1.txz kernel-headers-6.1.64-x86-1.txz
gcc-13.2.0-x86_64-1.txz pkg-config-0.29.2-x86_64-4.txz
```
- Then it asks to install some Pip dependencies JUST for the build process: `wheel`, `setuptools`, `pkgconfig`
- This can be done with `pip3 download pkgconfig setuptools wheel && pip3 install *.whl`
```
pkgconfig-1.5.5-py3-none-any.whl wheel-0.42.0-py3-none-any.whl
setuptools-69.0.2-py3-none-any.whl
```
- Now run `pip3 install "borgbackup[pyfuse3]"` to install borgbackup and build the wheel. Note that this adds `pyfuse3` integration, but that can be skipped by running the simplified install command `pip3 install borgbackup`
- It installs a few pip3 packages as dependencies:
```
msgpack-1.0.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
packaging-23.2-py3-none-any.whl

Successfully installed borgbackup-1.2.7 msgpack-1.0.7 packaging-23.2
```

- Permanently fetch these packages (other than borgbackup) with `pip3 download msgpack==1.0.7 packaging==23.2` and put them in `/boot/python_wheels/` for offline installation at boot time
- Copy the `borgbackup` wheel file from the directory stated in the build logs `cp /root/.cache/pip/wheels/b0/c1/b2/bd1f9969742c3b690d74ae13233287eec544c5a9135497443e/pyfuse3-3.3.0-cp312-cp312-linux_x86_64.whl /boot/python_wheels/`
- This would have appeared to be something similar to `Created wheel for borgbackup: filename=borgbackup-1.2.7-cp312-cp312-linux_x86_64.whl size=6248226 sha256=830ec3fbcc5c74922f7e269e8378aa08052c8c9cec51f407c53166d989d62e7c Stored in directory: /root/.cache/pip/wheels/64/90/02/c60bc19558d5e1e7c5ed13041bc27d25d41518ddfdb2def852`
- Note that renaming needs to follow some strict guidelines https://peps.python.org/pep-0491/#file-name-convention otherwise Pip will error out
- Add a line to the `/boot/config/go` file to automatically install `pyfuse3` at boot, or just run it adhoc to confirm it works: `/usr/bin/pip3 install /boot/python_wheels/borgbackup* --no-index --find-links file:///boot/python_wheels`
- The final list of wheel files ready to install at boot time are:
```
attrs-23.1.0-py3-none-any.whl
borgbackup-1.2.7-cp312-cp312-linux_x86_64.whl
idna-3.6-py3-none-any.whl
msgpack-1.0.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
outcome-1.3.0.post0-py2.py3-none-any.whl
packaging-23.2-py3-none-any.whl
pyfuse3-3.3.0-cp312-cp312-linux_x86_64.whl
sniffio-1.3.0-py3-none-any.whl
sortedcontainers-2.4.0-py2.py3-none-any.whl
trio-0.23.1-py3-none-any.whl
```

pyfuse3-3.3.0-cp312-cp312-linux_x86_64.whl 1.22 MB · 6 downloads python3-3.12.1-x86_64-1-kubed20231210.txz 12.25 MB · 23 downloads borgbackup-1.2.7-cp312-cp312-linux_x86_64.whl 5.96 MB · 7 downloads

Thank you for all of your investigations and work to pull this together! Clearly a lot of work went into it. Reading through this its also clear that I stood no chance of figuring it out on my own (previously leveraged NerdTools - and NerdPack before that - for python/borgbackup).

Am I reading this thread correctly that to migrate away from NerdTools for my borgbackup install to this "native" solution I simply download these three attached packages, I place the python file in my /boot/extra/ directory and reboot my box which will install python. From there I can use python's PIP to install the other two packages?

How are things like version bumps from borg handled? I assume an entirely new set of compilations need to be generated following the steps you did above? I ask as it seems a bit over my head and am worried I'll be installing a point-in-time version of python/borg and then stuck without the skillset to ever upgrade for bug fixes, etc.

Edited August 11 by aspiziri

fabricionaweb · August 13

The OP's post is old and I feel like some info can be added.

- Do not just install random packages on your machine

- After Unraid 6.12.0 the OpenSSL included is already 1.1.1

- Python is included in Slackware, dont need to look elsewhere for the .SlackBuild recipe. You can just get it from the official mirrors

- Slackware64 (amd64 or x86_64) is what you look for, and there are two branches: stable (15.0) and the current (testing)

At time I post this:

- "15.0" (Unraid 6) you get the Python 3.9.19 due security patches

- "current" (Unraid 7 afaik, I havent tested) you get the Python 3.11.9

Both has sources (the .SlackBuild) and .txz packages available there.

Problem is: it is not a static build, it share .so dependencies and if you just grab a build from "current" to install on "15.0" it will not always work... or may not worth try to fix the whole dependency tree... you can end broken something else.

Another observation is that pip3 (and setuptools) isnt include by default, you can just install both by simple running "python3 -m ensurepip" but it needs to run again after every install (or reboot) in case of Unraid (not a big deal actually). This is upstream decision due pip being a separated project.

Although the source is just there for you to do what is needed for each particular case... Like ours.

Grab the "current" .SlackBuild and build it against the "15.0" most of the dependencies will probably be correct.

I have build the 3.11.9 (and 3.12.5) from the official recipe (with few changes) on a Slackware-15 LXC (all my love to @ich777's plugin) and it seems to work just fine. glibc and openssl all linked correct within Unraid 6. You dont need (neither want to) be installing openssl3 glibc or anything else...

I have made a document within the steps I followed if someone wants to build it:

https://gist.github.com/fabricionaweb/95c84ba87710c781330b8842efafd0b1

Edited August 13 by fabricionaweb

kubed_zero · August 14

On 8/11/2024 at 2:44 PM, aspiziri said:

to migrate away from NerdTools for my borgbackup install to this "native" solution I simply download these three attached packages, I place the python file in my /boot/extra/ directory and reboot my box which will install python. From there I can use python's PIP to install the other two packages?

Yes, that's right.

9 hours ago, fabricionaweb said:

pip3 (and setuptools) isnt include by default, you can just install both by simple running "python3 -m ensurepip" but it needs to run again after every install (or reboot) in case of Unraid

I built pip into my package because I needed Unraid to get a later version of Python and Pip up and running *without internet* and therefore needed Pip and the Pip packages to install while the internet was still unavailable.

On 8/11/2024 at 2:44 PM, aspiziri said:

How are things like version bumps from borg handled?

I need to review my notes but either just running the pip update command may be possible, OR it might be necessary to install the build dependencies to recompile the Wheel file that Pip can then install.

On 8/11/2024 at 2:44 PM, aspiziri said:

stuck without the skillset to ever upgrade for bug fixes, etc.

Better than using an even older version that is generally available? Borg is doing its big 2.0 update *soon* so I'm waiting for that to redo all the compilation stuff. At that point I'll probably do the latest 1.x and the latest 2.x release and offer both packages

9 hours ago, fabricionaweb said:

official recipe (with few changes) on a Slackware-15 LXC

I opted to just compile on Unraid itself as it's then guaranteed to operate the same on the machine doing compliation and the machine running just the compiled packages 😃

kubed_zero · August 14

9 hours ago, fabricionaweb said:

Do not just install random packages on your machine

I agree, somewhat. If there are officially built versions of the *latest* Python (which as of writing is 3.12.5) and Borg (1.4.0) I'd happily skip the compliation steps and just use those. As far as I've seen that hasn't been available, so I've been stuck compiling them for my own use and sharing them in case others run into the same issue as me. I share the instructions though in case my files aren't trusted, so anyone can perform the same steps as me to get their own packages

Edited August 14 by kubed_zero
clarify

DjangoDurano · August 14

17 hours ago, fabricionaweb said:

The OP's post is old and I feel like some info can be added.

- Do not just install random packages on your machine

- After Unraid 6.12.0 the OpenSSL included is already 1.1.1

- Python is included in Slackware, dont need to look elsewhere for the .SlackBuild recipe. You can just get it from the official mirrors

- Slackware64 (amd64 or x86_64) is what you look for, and there are two branches: stable (15.0) and the current (testing)

At time I post this:

- "15.0" (Unraid 6) you get the Python 3.9.19 due security patches

- "current" (Unraid 7 afaik, I havent tested) you get the Python 3.11.9

Both has sources (the .SlackBuild) and .txz packages available there.

Problem is: it is not a static build, it share .so dependencies and if you just grab a build from "current" to install on "15.0" it will not always work... or may not worth try to fix the whole dependency tree... you can end broken something else.

Another observation is that pip3 (and setuptools) isnt include by default, you can just install both by simple running "python3 -m ensurepip" but it needs to run again after every install (or reboot) in case of Unraid (not a big deal actually). This is upstream decision due pip being a separated project.

Although the source is just there for you to do what is needed for each particular case... Like ours.

Grab the "current" .SlackBuild and build it against the "15.0" most of the dependencies will probably be correct.

I have build the 3.11.9 (and 3.12.5) from the official recipe (with few changes) on a Slackware-15 LXC (all my love to @ich777's plugin) and it seems to work just fine. glibc and openssl all linked correct within Unraid 6. You dont need (neither want to) be installing openssl3 glibc or anything else...

I have made a document within the steps I followed if someone wants to build it:

https://gist.github.com/fabricionaweb/95c84ba87710c781330b8842efafd0b1

It's correct that installing things on unraid is not the best way.

But installing packages from official isn't the best way in that case, that leads definitely trouble with some modules that are built in python.

Nerdtools by the way uses this "officials builds" and I used there python 3.9 packages. Normal things are working but for my python sleep script it got me errors, that come from not compatible packages from building vs. unraid.

I think it will be better if you clarify this in your post, that installing from official source isn't recommended.

Even with your build process that you are posting.

If you look at your unraid server at

/var/lib/pkgtools/packages

there should be all packages that are installed on unraid. If you look for maybe glibc and then search here (use the complete name that are listed in unraid) then you got the version that are installed within unraid.

I think the best way is, too look whats packages is installed within unraid, and then installing this packages into your slackware-lxc and then build python.

This should give us an fully working/compatible version of python for unraid.

Otherwise maybe some things are builded within python, with newer packages and this give us unexpected errors.

Edited August 14 by DjangoDurano

ich777 · August 14

On 8/13/2024 at 7:37 PM, fabricionaweb said:

The OP's post is old and I feel like some info can be added.

17 hours ago, kubed_zero said:

Yes, that's right.

~~I'm really unhappy about this thread because in my opinion Python shouldn't be installed on unRAID directly, at least not with the intent do general purpose stuff.~~

~~unRAID isn't a general purpose server, it's an application server and you should always install your applications in a Docker, LXC, VM~~

~~However if you really want to install Python why not take to official Slackware package? You'll get a full overview of Slackware current packages over here:~~

~~https://mirrors.slackware.com/slackware/slackware64-current/FILELIST.TXT~~

~~(as time of writing Python 3.11.9 is the latest Python 3 version)~~

~~Here are all possible packages for Unraid 7.0.0 beta series (which is based on Slackware current).~~

~~Again, I really don't think that it's a good idea to install Python on unRAID itself... I even think there are Docker containers out there for BorgBackup, Python and so on...~~

Please ignore my post, doesn't mean to upset anyone... I should learn to not post where I shouldn't post...

fabricionaweb · August 14

This thread is nice because seems no one is really reading each other messages hahahahah

But thanks everyone, it still have some nice insights

kubed_zero · August 14

6 hours ago, DjangoDurano said:

look whats packages is installed within unraid, and then installing this packages into your slackware-lxc and then build python

Or just use a fresh trial copy of the latest version of Unraid to build the packages in the first place! This is my technique at least. That way I'm also able to check against the latest stable versus beta version without too much effort.

5 hours ago, ich777 said:

in my opinion Python shouldn't be installed on unRAID directly, at least not with the intent do general purpose stuff.

I disagree strongly, but we're all welcome to our own opinions for each of our own use cases. Here is my hot take, in my opinion Unraid should be purely a NAS OS and not the hypervisor Docker flash-drive-backed extravaganza it's become. I want my NAS to focus purely on safely storing the files within.

In my use case I run Unraid as a VM and use Python only for NAS-related stuff, aka Borgbackup. Borg needs 3.10 or newer, and I'm not running Beta Unraid in prod just to get 3.11, so I need to update the system version of Python to a later version. At the time I started this off, 3.9.x was literally the latest slackware-compatible version I could find on the internet, so compiling itself was the only choice I had. I am happy to see there are more options available to folks now, but disappointed 3.12.x Slackware/Unraid builds are not more widely available.

5 hours ago, ich777 said:

why not take to official Slackware package?

This wasn't available at the time I needed it, and it's still not tracking the latest major version of Python, 3.12.x (released LAST YEAR). Especially given these big updates improve system-related features and performance, running a new version is very important to me.

All that said, I still think this self-compilation path is the best option I've got unless someone beats me to it and shares a TXZ file or the official files are kept more up to date. Given the recent thread activity, I might even be sparked into building the new versions in the next few weeks! 😃

4 hours ago, fabricionaweb said:

no one is really reading each other messages

This was a good prompt, I looked more closely at your GitHub Gist and the ensurepip flag is interesting, I might check that out in the next round of development. Same thing with the suggestion for /var/lib/pkgtools/packages, what a nice summary of everything that's installed! My current 6.12.8 Unraid installation seems to have glibc-2.37-x86_64-2 installed

primeval_god · August 14

3 hours ago, kubed_zero said:

Here is my hot take, in my opinion Unraid should be purely a NAS OS and not the hypervisor Docker flash-drive-backed extravaganza it's become. I want my NAS to focus purely on safely storing the files within.

The whole reason for adding Docker in the first place was in support of keeping the core unRAID OS focused on its NAS duties. The idea being that any app the user decides they have to install, that is not built into the core OS, should be isolated. Using docker, non-core functionality, whether backup, media serving, or whatever else, can be run with resource limits, restricted access to specific files/folders, and minimal dependence on the host os. No need to worry about what version of Slackware unRAID is based on and what packages are available/included in it if your application has all of its dependencies bundled in a convenient container image.

kubed_zero · August 18

8 minutes ago, primeval_god said:

keeping the core unRAID OS focused on its NAS duties

Well no, because now you have Docker is running on the NAS. Now Unraid has to deal with things such as VLANs and networking, persistent storage outside the array, resource sharing, and a whole other slew of factors. I don't mess with anything other than a parity drive and a few data drives, and Docker would add unnecessary complication.

That's fine though, we are all entitled to use Unraid how we want. Again, I'm sharing the work I did on my own system, everyone's free to use it or free to ignore.

Mxoio · August 20

Hi, there was a Python app you could download, but I had to do a fresh install unraid and couldn't find it, seems like it has been removed. @ich777 I've downloaded ISCSI target, which works perfectly for a simple plug-and-play situation. Thank you for mentioning this.

kubed_zero · August 27

All right friends, here are updated files for Python 3.12.5 (with bundled Pip 24.2), Borg 1.4.0, and PyFUSE 3.3.0 (same version as last time but I recompiled).

All were built and tested on Unraid 6.12.13, plus tested on 7.0.0-beta.2. I confirmed Python started up and ran some basic tests, I confirmed Pip could download and update packages, and I confirmed Borg could make new repos and mount previous backups with FUSE.

I ended up using the same package versions as last time, as many updated versions were giving me GLIBC-2.38 errors, as Unraid 6.x uses GLIBC-2.37 (and Linux 6.1.x). For example, trying to use OpenSSL-3.3.1 instead of 3.1.1 caused this error to crop up when trying to use wget. Thankfully Unraid 7.x seems to use GLIBC-2.39 (and Linux 6.8.x) so I suspect the issue will go away once 7.x is live.

Unraid 7.x also bundles OpenSSL-3.3.1 so I suspect up-to-date versions of all packages will be able to be used next time, at the expense of dropping Unraid 6.x compatibility.

Another update is that Borg 1.4.0 no longer bundles liblz4, libstd, or libxxhash https://github.com/borgbackup/borg/pull/8117 but the documentation said it did https://borgbackup.readthedocs.io/en/stable/installation.html#dependencies. There is now a bug report to get this fixed https://github.com/borgbackup/borg/issues/8342

I compiled Python 3.12.5 with the following packages (some not needed for Python compilation itself, but needed to exist on the system afterwards for the Pip compilation of pyfuse3 and borgbackup): acl-2.3.1-x86_64-1.txz attr-2.5.2-x86_64-1.txz binutils-2.41-x86_64-1.txz bzip2-1.0.8-x86_64-3.txz expat-2.5.0-x86_64-1.txz fuse3-3.15.0-x86_64-1.txz fuse3-3.16.2-x86_64-1.txz gc-8.2.4-x86_64-1.txz gcc-13.2.0-x86_64-1.txz gcc-g++-13.2.0-x86_64-1.txz gdbm-1.23-x86_64-1.txz git-2.43.0-x86_64-1.txz glibc-2.37-x86_64-2.txz guile-3.0.9-x86_64-1.txz kernel-headers-6.1.64-x86-1.txz libffi-3.4.4-x86_64-1.txz libmpc-1.3.1-x86_64-1.txz libzip-1.10.1-x86_64-1.txz lz4-1.10.0-x86_64-1.txz lzlib-1.13-x86_64-1.txz make-4.4.1-x86_64-1.txz ncurses-6.5-x86_64-1.txz openssl-3.1.1-x86_64-1.txz openssl-solibs-3.1.1-x86_64-1.txz openssl11-1.1.1w-x86_64-1.txz openssl11-solibs-1.1.1w-x86_64-1.txz pkg-config-0.29.2-x86_64-4.txz readline-8.2.001-x86_64-1.txz xxHash-0.8.2-x86_64-1.txz xz-5.4.3-x86_64-1.txz zlib-1.2.13-x86_64-1.txz zstd-1.5.6-x86_64-1.txz

The end of the compilation spit out the following summary:

The following modules are *disabled* in configure script:
_ctypes_test          _sqlite3              _testbuffer
_testcapi             _testclinic           _testimportmultiple
_testinternalcapi     _testmultiphase       _xxtestfuzz
xxsubtype

The necessary bits to build these optional modules were not found:
_dbm                  _tkinter              _uuid
nis
To find the necessary bits, look in configure.ac and config.log.

Checked 110 modules (31 built-in, 64 shared, 1 n/a on linux-x86_64, 10 disabled, 4 missing, 0 failed on import)

Then I worked on pyfuse3 and borgbackup compilation, a much simpler process than Python (one line each!):

I confirmed pyfuse3 worked with the following dependency versions: attrs-24.2.0 idna-3.8 outcome-1.3.0.post0 pyfuse3-3.3.0 sniffio-1.3.1 sortedcontainers-2.4.0 trio-0.26.2

I confirmed borgbackup worked with the following dependency versions: msgpack-1.0.8 packaging-24.1

All my Python use cases seem to be working, but let me know if any issues are encountered. Don't forget to add OpenSSL-3.1.1 to the /boot/extra/ directory alongside the Python package! It's attached, or you can download from https://slackware.uk/search?p=%2F&q=openssl-3.1.1-x86_64-1.txz

python3-3.12.5-x86_64-1-kubed20240825.txz borgbackup-1.4.0-cp312-cp312-linux_x86_64.whl pyfuse3-3.3.0-cp312-cp312-linux_x86_64.whl openssl-3.1.1-x86_64-1.txz

SDUGoten · Monday at 10:03 PM

On 8/27/2024 at 2:15 PM, kubed_zero said:
All right friends, here are updated files for Python 3.12.5 (with bundled Pip 24.2), Borg 1.4.0, and PyFUSE 3.3.0 (same version as last time but I recompiled).

All were built and tested on Unraid 6.12.13, plus tested on 7.0.0-beta.2. I confirmed Python started up and ran some basic tests, I confirmed Pip could download and update packages, and I confirmed Borg could make new repos and mount previous backups with FUSE.

I ended up using the same package versions as last time, as many updated versions were giving me GLIBC-2.38 errors, as Unraid 6.x uses GLIBC-2.37 (and Linux 6.1.x). For example, trying to use OpenSSL-3.3.1 instead of 3.1.1 caused this error to crop up when trying to use wget. Thankfully Unraid 7.x seems to use GLIBC-2.39 (and Linux 6.8.x) so I suspect the issue will go away once 7.x is live.

Unraid 7.x also bundles OpenSSL-3.3.1 so I suspect up-to-date versions of all packages will be able to be used next time, at the expense of dropping Unraid 6.x compatibility.

Another update is that Borg 1.4.0 no longer bundles liblz4, libstd, or libxxhash https://github.com/borgbackup/borg/pull/8117 but the documentation said it did https://borgbackup.readthedocs.io/en/stable/installation.html#dependencies. There is now a bug report to get this fixed https://github.com/borgbackup/borg/issues/8342

I compiled Python 3.12.5 with the following packages (some not needed for Python compilation itself, but needed to exist on the system afterwards for the Pip compilation of pyfuse3 and borgbackup): acl-2.3.1-x86_64-1.txz attr-2.5.2-x86_64-1.txz binutils-2.41-x86_64-1.txz bzip2-1.0.8-x86_64-3.txz expat-2.5.0-x86_64-1.txz fuse3-3.15.0-x86_64-1.txz fuse3-3.16.2-x86_64-1.txz gc-8.2.4-x86_64-1.txz gcc-13.2.0-x86_64-1.txz gcc-g++-13.2.0-x86_64-1.txz gdbm-1.23-x86_64-1.txz git-2.43.0-x86_64-1.txz glibc-2.37-x86_64-2.txz guile-3.0.9-x86_64-1.txz kernel-headers-6.1.64-x86-1.txz libffi-3.4.4-x86_64-1.txz libmpc-1.3.1-x86_64-1.txz libzip-1.10.1-x86_64-1.txz lz4-1.10.0-x86_64-1.txz lzlib-1.13-x86_64-1.txz make-4.4.1-x86_64-1.txz ncurses-6.5-x86_64-1.txz openssl-3.1.1-x86_64-1.txz openssl-solibs-3.1.1-x86_64-1.txz openssl11-1.1.1w-x86_64-1.txz openssl11-solibs-1.1.1w-x86_64-1.txz pkg-config-0.29.2-x86_64-4.txz readline-8.2.001-x86_64-1.txz xxHash-0.8.2-x86_64-1.txz xz-5.4.3-x86_64-1.txz zlib-1.2.13-x86_64-1.txz zstd-1.5.6-x86_64-1.txz

The end of the compilation spit out the following summary:
The following modules are *disabled* in configure script:
_ctypes_test          _sqlite3              _testbuffer
_testcapi             _testclinic           _testimportmultiple
_testinternalcapi     _testmultiphase       _xxtestfuzz
xxsubtype

The necessary bits to build these optional modules were not found:
_dbm                  _tkinter              _uuid
nis
To find the necessary bits, look in configure.ac and config.log.

Checked 110 modules (31 built-in, 64 shared, 1 n/a on linux-x86_64, 10 disabled, 4 missing, 0 failed on import)
Then I worked on pyfuse3 and borgbackup compilation, a much simpler process than Python (one line each!):

I confirmed pyfuse3 worked with the following dependency versions: attrs-24.2.0 idna-3.8 outcome-1.3.0.post0 pyfuse3-3.3.0 sniffio-1.3.1 sortedcontainers-2.4.0 trio-0.26.2

I confirmed borgbackup worked with the following dependency versions: msgpack-1.0.8 packaging-24.1

All my Python use cases seem to be working, but let me know if any issues are encountered. Don't forget to add OpenSSL-3.1.1 to the /boot/extra/ directory alongside the Python package! It's attached, or you can download from https://slackware.uk/search?p=%2F&q=openssl-3.1.1-x86_64-1.txz

python3-3.12.5-x86_64-1-kubed20240825.txz 11.7 MB · 6 downloads borgbackup-1.4.0-cp312-cp312-linux_x86_64.whl 3.81 MB · 5 downloads pyfuse3-3.3.0-cp312-cp312-linux_x86_64.whl 1.22 MB · 2 downloads openssl-3.1.1-x86_64-1.txz 5.11 MB · 3 downloads

I wonder what is the proper way to install urllib3 for this? Putting into /boot/config/go might be to early to execute before these binary is in place.

Thanks.

trurl · Monday at 11:32 PM

User Scripts plugin

Python 3.11 Unraid TXZ Package/Plugin

Recommended Posts

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment