[Plugin] LXC Plugin

[stamina77]

8 hours ago, ich777 said:

These entries are wrong.

You can't use : you have to use =

: where the old format and are not supported anymore.

 

Yes, because of the formatting error.

 

May I ask why you are using LXC for Plex in the first place? A Docker container would be much easier to set up or did I get this wrong and you have a different use case for the container?

 

thanks ich777，i successful pass through nvidia card and installed plex under instruction of this thread

https://www.geekbitzone.com/posts/2022/proxmox/plex-lxc/install-plex-in-proxmox-lxc/

 

It totally work as I expected，I can use nv card to encode video from unraid folder and port forward plex service on dedicated ethernet（not like orignal unraid docker failed to fulfill that purpose ) to pub connection

 

now I have lots of interest to dive into lxc learning.

 

Thanks ich777, you are the pro

[ich777]

8 hours ago, buxel said:

I have tried rebooting and disabling/enabling the plugin already. The folders on `/mnt/cache/lxc` are owned by `root`. Is that correct?

Yes.

 

8 hours ago, buxel said:

migrate some existing containers from my previous servers.

What was your previous server and how do you migrate?

 

8 hours ago, buxel said:

I tried your plugin on a fresh installation of Unraid 6.12.8 and LXC plugin verison 2024.03.14. ´Creating containers worked fine but destroying them always results in this error:

Please post your Diagnostics.

Are these migrated containers or are these newly set up containers through the plugin?

I assume that the path /mnt/cache is on a ZFS pool, did you delete by any chance the dataset zfs_lxccontainers?

Or did you maybe create a container, then migrated the data from the old system over?

 

Please describe exactly what you are doing.

 

EDIT: Just as a sanity check I created and felted a container, using also ZFS as Backing Storage type.

[buxel]

Hello @ich777, thank you for the quick response.

 

Migration is something i want to tackle later. For now, all containers are newly created through the plugin.

I have not deleted (or any other way messed with) anything. I first want to see how the system works, before i break it 😉. However, i have not created any dataset myself, i just followed the instructions in the plugin and (i think) it mentioned that the non-existing paths will be created.

 

Please find the logs attached. Before exporting, i have tried again with "test-container", which gave the same error as before.

 

 

hermes-diagnostics-20240319-1929.zip

Edited March 19 by buxel

[ich777]

1 hour ago, buxel said:

Please find the logs attached. Before exporting, i have tried again with "test-container", which gave the same error as before.

Just out of curiosity I tried also a NIX container 23.11:

 

...let it run for a bit:

 

...and then deleted it:

 

Seems to work fine here...

 

I then changed the plugin to Directory as Backing Storage type:

 

...and there it is when I delete the container:

 

 

However it seems to be related specific to NIXOS since it is not happening with a Debian container.

 

I just noticed that there the immutable bit is set. You have to remove that first and then delete it.

To remove that bit open up a Unraid terminal and type in:

chattr -i <PATHTOCONTAINER>/rootfs/var/empty

(you have to replace the path to container with the absolute path to /rootfs/var/empty like in my screenshot /mnt/nvme/lxc/nix/rootfs/var/empty)

 

After that you can delete the folder for the container from the lxc directory and it will disappear (refresh the LXC page once) or you could also go to the LXC page and delete the container since after that the removal will work as intended.

 

Hope that helps for now and I will look into if I can work around that in the plugin.

 

I'll look into that and report back.

 

EDIT: BTW, please set the Storage Backing Type in the LXC settings to BTRFS in your case since then LXC will use BTRFS nativesnapshots.

 

EDIT2: @buxel please update the LXC plugin to version 2024.03.19a where I implemented a workaround to remove the immutable bit first and then delete the container.

[buxel]

I can confirm everything you just said. The immutable bit seems to date back quite a while: https://github.com/NixOS/nixpkgs/commit/3877ec5b2ff7436f4962ac0fe3200833cf78cb8b#commitcomment-19100105

 

I assume, NixOS being special in it's ways about immutability, this was implemented to keep some declarative guarantees and prevent apps from writing where they shouldn't.

I have (ab)used this bit myself to make sure no app is accidently writing to a not yet mounted share.

 

Oh, and thanks about the note on the backing storage. 👍

[ich777]

2 minutes ago, buxel said:

Oh, and thanks about the note on the backing storage. 👍

I don't know if you have seen it already but I've made a workaround for that, please update the plugin to version 2024.03.19a

 

Container removal now works fine, even when you are using Directory as the backing storage type.

[buxel]

Great job! I can confirm the fix works on my end as well

Thank you!

[buxel]

Hello again,

 

i've been tinkering with the container and noticed some difference to my other server.

I'm trying to run Tailscale inside the container, following these instructions: https://tailscale.com/kb/1130/lxc-unprivileged

 

tailscaled refuses to work:

is CONFIG_TUN enabled in your kernel? `modprobe tun` failed with: 
wgengine.NewUserspaceEngine(tun "tailscale0") error: tstun.New("tailscale0"): operation not permitted
flushing log.
logger closing down
getLocalBackend error: createEngine: tstun.New("tailscale0"): operation not permitted

 

`/dev/net/tun` is available inside the container but i suspect some permissions to be off.

The Proxmox wiki mentions permissions, but i don't know if the mappings are the same in Unraid: https://pve.proxmox.com/wiki/OpenVPN_in_LXC

 

Here is the container's config:

# Template used to create this container: /usr/share/lxc/templates/lxc-download
# Parameters passed to the template: --dist nixos --release 23.11 --arch amd64
# Template script checksum (SHA-1): 78b012f582aaa2d12f0c70cc47e910e9ad9be619
# For additional config options, please look at lxc.container.conf(5)

# Uncomment the following line to support nesting containers:
#lxc.include = /usr/share/lxc/config/nesting.conf
# (Be aware this has security implications)
lxc.mount.entry = proc dev/.lxc/proc proc create=dir,optional 0 0
lxc.mount.entry = sys dev/.lxc/sys sysfs create=dir,optional 0 0

# Allow Tailscale to work
lxc.cgroup2.devices.allow = c 10:200 rwm
#lxc.mount.entry = /dev/net/tun dev/net/tun none bind,create=file
lxc.mount.entry = /dev/net dev/net none bind,create=dir

# Distribution configuration
lxc.include = /usr/share/lxc/config/common.conf
lxc.arch = x86_64
# According to NixOS Wiki
lxc.init.cmd = /sbin/init


# Container specific configuration
lxc.rootfs.path = btrfs:/mnt/cache/lxc/morbo/rootfs
lxc.uts.name = morbo

# Network configuration
lxc.net.0.type = veth
lxc.net.0.flags = up
lxc.net.0.link = br0
lxc.net.0.name = eth0

lxc.net.0.hwaddr=52:54:00:72:04:38
lxc.start.auto=1

 

Any pointers would be appreciated 😊

Edited March 24 by buxel

[ich777]

3 hours ago, buxel said:

Any pointers would be appreciated 😊

As far as I can tell in the Proxmox documentation is nesting enabled:

features: nesting=1

and that means basically uncomment this line:

#lxc.include = /usr/share/lxc/config/nesting.conf

so that it looks like that:

lxc.include = /usr/share/lxc/config/nesting.conf

 

However it seems a bit much only for the tun device to be working, IIRC you don't need to uncomment this but I could be also wrong about that. Sorry but I really can't help with Tailscale since I don't use it.

 

 

3 hours ago, buxel said:

# Allow Tailscale to work

It seems that you've found my post from here:

[buxel]

57 minutes ago, ich777 said:

lxc.include = /usr/share/lxc/config/nesting.conf

 

 

I have tried this but it results in an invalid configuration. I'm quite a noob to how LXC and apparmor/selinux relate, but from this thread i gathered that Unraid does not use apparmor -> hence the invalid configuration error when i uncomment the "lxc.include" line. 
 

# <content of /usr/share/lxc/config/nesting.conf>
# Use a profile which allows nesting
lxc.apparmor.profile = lxc-container-default-with-nesting

# Add uncovered mounts of proc and sys, else unprivileged users
# cannot remount those

lxc.mount.entry = proc dev/.lxc/proc proc create=dir,optional 0 0
lxc.mount.entry = sys dev/.lxc/sys sysfs create=dir,optional 0 0

 

This is why i copied over the two other 'lxc.mount' lines over to the container's config. Thinking about it, i may just be missing the equivalent setting for SELinux... 🤔

 

My hunch says the /dev/net/tun device is mapped propery but the tailscale process is lacking permissions to modify it. 

[ich777]

Just now, buxel said:

This is why i copied over the two other 'lxc.mount' lines over to the container's config. Thinking about it, i may just be missing the equivalent setting for SELinux... 🤔

No, please forget about that, this is the wrong direction.

 

Stop the container, comment the line from above again (this was the wrong file that I pointed you to), append this to your config:

lxc.cap.drop =

and start the container again (no, I didn't forgot anything, this is meant to be like in my example).

[buxel]

Thank you for looking into this with me.

I realized that i have mixed up two different problems: nesting was enabled for NixOS. The two mounts allow it to work properly, but are unrelated to tailscale or '/dev/net/tun'

 

I have updated the config but there is no change in behavior. The error messages are the same. 

For completeness sake, here is the current config:

 

# Uncomment the following line to support nesting containers:
#lxc.include = /usr/share/lxc/config/nesting.conf
# (Be aware this has security implications)
lxc.mount.entry = proc dev/.lxc/proc proc create=dir,optional 0 0
lxc.mount.entry = sys dev/.lxc/sys sysfs create=dir,optional 0 0

# Allow Tailscale to work
lxc.cap.drop =
lxc.cgroup2.devices.allow = c 10:200 rwm
lxc.mount.entry = /dev/net/tun dev/net/tun none bind,create=file

# Distribution configuration
lxc.include = /usr/share/lxc/config/common.conf
lxc.arch = x86_64
# According to NixOS Wiki
lxc.init.cmd = /sbin/init

# Container specific configuration
lxc.rootfs.path = btrfs:/mnt/cache/lxc/morbo/rootfs
lxc.uts.name = morbo

# Network configuration
lxc.net.0.type = veth
lxc.net.0.flags = up
lxc.net.0.link = br0
lxc.net.0.name = eth0

lxc.net.0.hwaddr=52:54:00:72:04:38
lxc.start.auto=1

 

[ich777]

6 minutes ago, buxel said:

# Allow Tailscale to work
lxc.cap.drop =

This is not what I've wrote you should do, I wrote:

15 minutes ago, ich777 said:

append this to your config

to be more precise put this at the end from your config.

 

I would always recommend to append your changes at the end because you make your life harder than it is if you put it in different spots in the config.

[ich777]

7 minutes ago, buxel said:

# According to NixOS Wiki
lxc.init.cmd = /sbin/init

BTW what is this? The container should do that on it's own and this is not necessary.

[buxel]

Oh, i was not aware of that the order in the config matters. That is a pretty big gotcha!

 

With the _appended_ line it actually works as expected👍. Does this mean the container inherits all capabilities from the host? If i figure out the right one, could i just add it via "lxc.cap.add"? i assume it is "net_admin"

 

The init command is there because I just folloewed the NixOS wiki. But you are right, it works fine with

Edited March 24 by buxel

[ich777]

54 minutes ago, buxel said:

Does this mean the container inherits all capabilities from the host?

Yes.

 

54 minutes ago, buxel said:

could i just add it via "lxc.cap.add"?

There is no cap add.

 

You can allow all caps with lxc.cap.drop = and then forbid individual caps like:

lxc.cap.drop =
lxc.cap.drop = mac_admin mac_override sys_time sys_module sys_rawio

(the listed caps above in the second line are all caps that are dropped at container start - you have to experiment what you have to drop and what not)

[stayupthetree]

How will this affect LXC in Unraid? 
https://discuss.linuxcontainers.org/t/important-notice-for-lxd-users-image-server/18479

[ich777]

7 hours ago, stayupthetree said:

How will this affect LXC in Unraid? 

This post is from December 2023…

 

This won‘t affect the plugin at all because the plugin is not using LXD it is pure LXC and only LXD users will loose access to the image repository.

 

Question:

Quote

Will incus still maintain access to all prebuilt distributions?

Answer:

Quote

Yes, Incus and LXC both have access to all the images on the image server.