[Plugin] LXC Plugin

[zh522130]

9 hours ago, ich777 said:

And what was your solution the / or the char device?

I'm very sorry, I had only tested 'file' before. I just tested 'char' and it does not work properly; only 'file' works correctly. rocminfo can correctly identify the GPU. finally, I added the following content for the iGPU of the 7840HS:

# For AMD GPU
lxc.cgroup2.devices.allow = c 226:0 rwm
lxc.cgroup2.devices.allow = c 226:128 rwm
lxc.cgroup2.devices.allow = c 242:0 rwm
lxc.mount.entry = /dev/dri/card0 dev/dri/card0 none bind,optional,create=file
lxc.mount.entry = /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file
lxc.mount.entry = /dev/kfd dev/kfd none bind,optional,create=file

it may also be applicable to other AMD GPUs.

[srirams]

Would it be possible to keep lxc containers running even if the array is stopped (all lxc data is on a unassigned devices drive)?

[ich777]

9 hours ago, srirams said:

Would it be possible to keep lxc containers running even if the array is stoppe

No, that‘s not how the plugin is designed and meant to be used.

 

Why do you need that or what is the exact use case?

[srirams]

11 hours ago, ich777 said:

Why do you need that or what is the exact use case?

I would like to use a container to run openwrt (or just plain alpine) as my router. I've tried it out and it works great (except for stopping when the array is stopped )

[ich777]

41 minutes ago, srirams said:

I would like to use a container to run openwrt (or just plain alpine) as my router. I've tried it out and it works great (except for stopping when the array is stopped )

You could create a user script that runs on stopping the array that does the following:

sleep 5

lxc-start CONTAINERNAME

 

But I really don‘t know if that would work.

The sleep is only there because to make sure the container is actually stopped before it is started again.

[srirams]

Thanks, but that will still kill the network for some time which would be undesirable...

 

I'm wondering if I can just disable the disks_mounted and unmounting_disks events with no side effects (provided that all data lives on drives not on the array)?

[ich777]

50 minutes ago, srirams said:

I'm wondering if I can just disable the disks_mounted and unmounting_disks events with no side effects (provided that all data lives on drives not on the array)?

No because then you run into various other issues.

 

The plugin was never designed to be used with the Array stopped, if you want to use it like that you have to come up with your own custom solution.

[jikerom]

I encountered the following error while creating the container. I noticed that there are two "/" characters in some parts of the download link for "rootfs. tar. xz". Is the error related to this?
How do I solve it?

root@Tower:/mnt/cache# lxc-create --name Debian_bullseye --template download -- --dist debian --release bullseye --arch amd64
Downloading the image index
Downloading the rootfs
ERROR: Failed to download https://images.linuxcontainers.org//images/debian/bullseye/amd64/default/20240511_05:24//rootfs.tar.xz
lxc-create: Debian_bullseye: ../src/lxc/lxccontainer.c: create_run_template: 1589 Failed to create container from template
lxc-create: Debian_bullseye: ../src/lxc/tools/lxc_create.c: lxc_create_main: 318 Failed to create container Debian_bullseye

 

[ich777]

39 minutes ago, jikerom said:

How do I solve it?

May I ask where are you located in the world?

 

39 minutes ago, jikerom said:

 

I encountered the following error while creating the container

 

Did you yet try to create a container in the GUI?

 

Please post your Diagnostics.

[jikerom]

15 hours ago, ich777 said:

May I ask where are you located in the world?

I am in China

15 hours ago, ich777 said:

Did you yet try to create a container in the GUI?

 

Please post your Diagnostics.

The same error message applies when installing using GUI

My current solution is to download the file from the link above using a browser, upload it to the "/mnt/cache/lxc/debian/rootfs" folder, unpack it, and then place the config file in the "/mnt/cache/lxc/debian/" folder. Then, I can run it in the LXC plugin

[ich777]

1 minute ago, jikerom said:

The same error message applies when installing using GUI

Can you please upload your Diagnostics?

I would really like to see your settings.

 

1 minute ago, jikerom said:

Then, I can run it in the LXC plugin

Is it possible that your server can't reach https://images.linuxcontainers.org?

The double / shouldn't cause any issues.

This is the first time that I see that issue? Are you using the proxy manager?

[jikerom]

1 hour ago, ich777 said:

Is it possible that your server can't reach https://images.linuxcontainers.org?

The double / shouldn't cause any issues.

This is the first time that I see that issue? Are you using the proxy manager?

Yes, I used the proxy,
Unraid seems to be accessible https://images.linuxcontainers.org Although it is relatively slow

I think it may be that the LXC plugin did not pass through the server's proxy

Here is my diagnosis

tower-diagnostics-20240513-1729.zip

[ich777]

48 minutes ago, jikerom said:

Unraid seems to be accessible https://images.linuxcontainers.org Although it is relatively slow

You have to specifically try if you can download for example the container list index with:

wget https://images.linuxcontainers.org/meta/simplestreams/v1/index.json

and make sure that the file is not empty afterwards.

 

I just tried it again from my machine and it seems to work just fine:

 

I also pushed an update just now to the LXC plugin to make sure that the LXC container image index is pulled through the proxy since this was not the case before, did you see drop downs on the add container page like in this screenshot:

 

56 minutes ago, jikerom said:

I think it may be that the LXC plugin did not pass through the server's proxy

This is a thing that I also tested if LXC is downloading everything through the proxy and it seems that it does:

PROXY   : 2024/05/13 12:31:28 handler.go:138: INFO     Request: 10.0.0.140:50800 => 172.19.0.2:8118 "" HTTP/1.1 CONNECT //images.linuxcontainers.org:443
PROXY   : 2024/05/13 12:31:29 handler.go:138: INFO     Request: 10.0.0.140:50808 => 172.19.0.2:8118 "" HTTP/1.1 CONNECT //images.linuxcontainers.org:443
PROXY   : 2024/05/13 12:31:29 handler.go:138: INFO     Request: 10.0.0.140:50816 => 172.19.0.2:8118 "" HTTP/1.1 CONNECT //images.linuxcontainers.org:443
PROXY   : 2024/05/13 12:31:29 handler.go:138: INFO     Request: 10.0.0.140:50826 => 172.19.0.2:8118 "" HTTP/1.1 CONNECT //images.linuxcontainers.org:443
PROXY   : 2024/05/13 12:31:30 handler.go:138: INFO     Request: 10.0.0.140:50830 => 172.19.0.2:8118 "" HTTP/1.1 CONNECT //images.linuxcontainers.org:443
PROXY   : 2024/05/13 12:31:30 handler.go:138: INFO     Request: 10.0.0.140:50842 => 172.19.0.2:8118 "" HTTP/1.1 CONNECT //fra1lxdmirror01.do.letsbuildthe.cloud:443

(this is the output from my proxy server when it downloads a image from LXC with a proxy specified)

 

How did you enable the proxy?

[jikerom]

53 minutes ago, ich777 said:

您必须专门尝试是否可以下载例如包含以下内容的容器列表索引：

wget https://images.linuxcontainers.org/meta/simplestreams/v1/index.json

并确保之后文件不为空。

Yes, it can be downloaded

55 minutes ago, ich777 said:

did you see drop downs on the add container page like in this screenshot:

Yes.

59 minutes ago, ich777 said:

How did you enable the proxy?

I set up clash on ikuai and then set the proxy to the unread server through the Proxy Editor plugin.

 

Thank you for your patient answer. I think it might be due to the internet connection on my end. I'm giving it a try

[kidab]

Not sure if this is the right place to ask but am at my wit's end. 

 

I've installed the plugin and am running all kinds of LXC containers. Everything works great!

 

However im currently facing an issue trying to run a k3's cluster. I am trying to install the NVIDA GPU operator which performs all kinds of bind mounts to view and confirm what GPU resources are available. Installation fails however with several errors complaining that the root directory (and others) are not mounted in shared or slave mode. 

 

I've done all kinds of reading and have also tried adding "lxc.rootfs.options = rshared" to the lxc conf. No change. I've also tried "slave" and see neither option in "/proc/mounts" for the root directory. (shown below). No change if I use an XFS or btrfs formatted disk.

 

/dev/md1p1 / xfs rw,noatime,nouuid,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0

 

Really hoping you could point me in the right direction, thanks!

[ich777]

6 hours ago, kidab said:

k3's cluster

Sorry I‘m not familiar with this, can you link some documentation please and how that you've set it up on Unraid with the LXC plugin?

 

6 hours ago, kidab said:

Really hoping you could point me in the right direction, thanks!

You mount a while block device or how do you do that exactly?

Can you maybe also share your Diagnostics so that I can see the whole configuration from your System and LXC?

 

EDIT: Is this correct? I just installed the operator like described here.

root@k3s:/# helm install --wait --generate-name \
    -n gpu-operator --create-namespace \
    nvidia/gpu-operator
NAME: gpu-operator-1715666542
LAST DEPLOYED: Tue May 14 06:02:25 2024
NAMESPACE: gpu-operator
STATUS: deployed
REVISION: 1
TEST SUITE: None

 

root@k3s:/# kubectl get nodes -o json | jq '.items[].metadata.labels | keys | any(startswith("feature.node.kubernetes.io"))'
true

 

I don't have Nvidia GPU installed in my main server but I assume the outputs are correct???

 

Did you add this to your container configuration:

lxc.cgroup2.devices.allow = a
lxc.cap.drop =
lxc.mount.auto = "proc:rw sys:rw"

(please note this is basically as if you run K3S on the host since this gives the LXC container almost full control from the host but I think that's intended)

 

Don't know if that's relevant for you or if you already know that but I had to also create /etc/rc.local file in the container the K3S installation with the contents:

#/bin/sh -e
if [ ! -e /dev/kmsg ]; then
  ln -s /dev/console /dev/kmsg
fi
mount --make-rshared /

and make it executable.

 

After that I installed K3S with:

curl -sfL https://get.k3s.io | sh -s - server --disable servicelb --disable traefik --write-kubeconfig-mode 644

 

ran:

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

 

and:

kubectl get nodes

to check if everything is working.

This was sourced from here and here (with slight modifications)

 

 

After that I installed the Nvidia GPU Operator like linked above.

 

I did use Debian Bookworm as the LXC container with the following packages:

curl wget nano jq git

 

Please excuse me if you knew that all already and your installation is failing at another point...

[kidab]

Awesome I will test this out and report back later. I believe the contents of /etc/r.local are going to be what solves my issues. Specifically the mounting of / with —make-rshared 

 

Everything else regarding k3s installation is pretty much exactly same for me. Thanks!

[kidab]

Ok reporting back that everything seems to work! My container is running Ubuntu 22 so I modified the rc.local to this:

#!/bin/bash
mount --make-rshared /

 

At first it wasn't being applied. To solve that I followed an online tutorial to create a rc-local.service to actually run the contents of that file^. After a reboot the errors about shared mounts went away.

 

However I did proceed to get the following error from the Nvidia gpu operator (formatting modified slightly for clarity):

 level=info msg="creating symlinks under /dev/char that correspond to NVIDIA character devices"
time="2024-05-14T21:31:51Z" level=info msg="Error: error validating driver installation: error creating symlink creator: failed to load NVIDIA kernel modules: failed to load module nvidia: exit status 1; output=modprobe: 

FATAL: Module nvidia not found in directory /lib/modules/6.1.79-Unraid

Failed to create symlinks under /dev/char that point to all possible NVIDIA character devices.

The existence of these symlinks is required to address the following bug:
      https://github.com/NVIDIA/gpu-operator/issues/430\n\nThis bug impacts container runtimes configured with systemd cgroup management enabled.

To disable the symlink creation, set the following envvar in 

ClusterPolicy:
  validator:
    driver:
      env:
      - name: DISABLE_DEV_CHAR_SYMLINK_CREATION
        value: "true"

I followed the instructions in the error and update the helm install options. As of now, my node was properly tagged as having two Nvidia GPUs and all seems well! I have yet to schedule any actual workflows but am sure they will run. Thanks!!!

[Beo]

Hi,

 

Very nice !

 

Do you know if it's possible to add turnkey lxc templates ?

 

Best regards

[ich777]

1 hour ago, Beo said:

Do you know if it's possible to add turnkey lxc templates ?

I don't know anything about Turnkey, can you explain that a bit in detail, is this a Distribution or something that gets installed or is Turnkey installed in a container?

[bubbadk]

how do i create an extra disk for debian. i would like to play with cockpit and i need to create a disk for it

[ich777]

25 minutes ago, bubbadk said:

how do i create an extra disk for debian.

Can you describe a bit in detail what you want to do?

 

Do you want to create a image or what do you mean with disk exactly? Do you want to mount something from the host? Do you want to bind a physical block device to the container?

 

27 minutes ago, bubbadk said:

i would like to play with cockpit

What is Cockpit? I assume when I Google for "Cockpit Software" I get a lot of results.

[bubbadk]

2 minutes ago, ich777 said:

Can you describe a bit in detail what you want to do?

 

Do you want to create a image or what do you mean with disk exactly? Do you want to mount something from the host? Do you want to bind a physical block device to the container?

 

i just want to make an extra virtual disk that can be mounted in debian LXC

 

What is Cockpit? I assume when I Google for "Cockpit Software" I get a lot of results.

https://cockpit-project.org/

 

 

[ich777]

1 hour ago, bubbadk said:

i just want to make an extra virtual disk that can be mounted in debian LXC

I don't recommend doing it that way because if you create a container on Unraid it doesn't create a virtual disk it uses a directory (or BTRFS/ZFS subvolume depending on how you configured the plugin).

However you can create a image and mount it but you have to do that manually.

 

Why not create a second container and destroy it if you don't need it anymore?

That's the beauty of LXC, if you don't need it anymore or you are done with testing simply destroy it and after that you can install it to an existing container or create a new container for production use.

 

Just deploy a new container like here:

 

...and when you are done with testing destroy it:

 

 

Sorry if I completely miss your use case but this would be the way I would do it if I just want to test something <- that's actually what I do sometimes if I want to test something on Linux because it is really easy to set up and destroy if it's not what I was searching or not suitable for the use case.

[wy586]

Hello, plugin author. May I ask why CentOS 7 cannot retrieve the IPv4 address while CentOS 8 can? I need CentOS 7 to use certain applications.