[Plugin] LXC Plugin

[bmartino1]

Thank you ich777
While I have a github account and can share and release code.
https://github.com/bmartino1?tab=repositories

ATM this git and github is fairly new to me and alot of my code stuff is in alpha or test as i was testing and reviewing other's code.

I'm more looking to export the entire rootFS folder. With config. to transfer competed edits, gpup keys and other via GNU GENERAL PUBLIC LICENSE. Some things can get weird and copyright tricky fast...

But exposing the root FS to GitHub opens it up to potential bad actors as the secrets and others are exposed form the debain os...

Which is why I could see sharing a snapshot instead.(Maybe...)

Quick Review of your lxc template. Its an interesting way to share the lxc and use pi hole... Incidentally, I have recently set up a similar LXC using Pi hole with Cloudflare /stubby /unbound in a debian lxc container. Its setup and working properly as well. Seeing your template, I may be looking to add lan cache to it.

ATM I think it's easier to explain why I did xyz and share the commands I ran then it is to try to rebuild and generate a full template. ATM this is Not as easily exported or setup to use for a client/user to point click...

So I would have 2 lxc container template Ideas I would like share to the community when I/We/ETC are done and capable. I will dig into it more tomorrow when i have more time.

 

I will need to see if I can transfer a snapshot to restore to a friend's lxc as a test and then explore what in the snapshot. Not sure if a snapshot is as exposing of the root system. or what I should pull to keep a general backup of instulation. As i have used other apt key programs to install all apps and copy all configs I have had and used before in the past to move systems form PC to PC.... 

This way I could guide a client/User to make a XYZ lxc container named the same in their general lxc path and download/git clone the source. Then restore snap and be done. Some small lxc conf edit may be required.

IDK if it more work to make a lxc image. to IDK use proxmox to make a image and export it to github. Then again potential use the wget/ lxc template download command. Similar to adding a lxc template repository you posted earlier. I have options regardless. Just not sure what path is best to take and better for security/client use.

I'm not sure what " Incus." but I understand not wanting to implement and use Canonical lxc/lxd new codes and system.

I need to do some more research.

[ich777]

2 hours ago, bmartino1 said:

I'm more looking to export the entire rootFS folder. With config. to transfer competed edits, gpup keys and other via GNU GENERAL PUBLIC LICENSE. Some things can get weird and copyright tricky fast...

Oh wait, we are talking about completely different use cases here.

You basically want a system that allows you to transfer your images to other machines correct?

My use case would be just to make pre-made container archives available for users to download and deploy through the CA App.

 

3 hours ago, bmartino1 said:

Which is why I could see sharing a snapshot instead.(Maybe...)

This is already possible but I would rather recommend doing that through the included Backup function which I ship with the Unraid plugin.

This will basically compress the rootfs including the config (or not compress it, just how you set it up) and you easily can transfer it over to another or even the same system and deploy it as a new container, similar how Snapshots work but also different since this can be any location on your server.

 

The backup function is both available in the WebGUI (when you configure it in the LXC settings) or from the command line, open up a Unraid terminal and issue:

lxc-autobackup --help

or in the WebUI go to the LXC settings and click Backup:

 

3 hours ago, bmartino1 said:

But exposing the root FS to GitHub opens it up to potential bad actors as the secrets and others are exposed form the debain os...

This is not really an issue for my use case since as said above my use case would be only to deploy premade container images for easy deployment to end users, similar like Docker works.

In my PiHole container the SSH keys for example are deleted before the container archive is created and the keys will be created on first start from the container by this script, so to speak even the keys don't leak out to GitHub.

 

Again, this is a completely different use case from what you are looking for.

 

3 hours ago, bmartino1 said:

Quick Review of your lxc template. Its an interesting way to share the lxc and use pi hole... Incidentally, I have recently set up a similar LXC using Pi hole with Cloudflare /stubby /unbound in a debian lxc container. Its setup and working properly as well. Seeing your template, I may be looking to add lan cache to it.

I would recommend that you just deploy it and try it out how my solution works and LANCache is completely optional.

 

3 hours ago, bmartino1 said:

ATM I think it's easier to explain why I did xyz and share the commands I ran then it is to try to rebuild and generate a full template. ATM this is Not as easily exported or setup to use for a client/user to point click...

My take on that was for you to be able to create a Unifi container archive for others for easy deployment.

 

3 hours ago, bmartino1 said:

This way I could guide a client/User to make a XYZ lxc container named the same in their general lxc path and download/git clone the source. Then restore snap and be done. Some small lxc conf edit may be required.

But why would you do that?

I built the template function because it is way easier to deploy for users (just a wget and the WebGUI part) currently and it will be even be easier when the templates are supported by CA.

 

Basically what my approach (for containers that you want to share with the community) is similar to the Backup function it creates a highly compressed container archive from a temporary container that users can easily deploy with just the necessary packages and scripts to run the application(s) inside the container). However I don't recommend use the Backup function to share containers with other users because it is not designed for that.

 

3 hours ago, bmartino1 said:

IDK if it more work to make a lxc image. to IDK use proxmox to make a image and export it to github.

I really don't like the idea of the image because it would be way, way easier to leak out some of what you've mentioned before, is way less flexible and of course don't forget how much increase this means in terms for filesize.

 

3 hours ago, bmartino1 said:

I'm not sure what " Incus." but I understand not wanting to implement and use Canonical lxc/lxd new codes and system.

Incus is basically a fork from LXD because LXD was ripped out of the hands from the community from Cononical without a real warning, so to speak Incus is the new LXD, but again I won't include Incus or LXD in the Unraid plugin.

 

 

I would really recommend that you try some of the images for example this image:

https://github.com/ich777/unraid_lxc_debian_novnc

which containers a fully fledged XFCE desktop environment that you can access through noVNC.

You just have to always download the RAW template file to: /tmp/lxc_container_template.xml and then visit the site: http://<YOURSERVERIP>/LXCAddTemplate to install it.

 

The templates also fully support custom container icons and also a WebGUI button like it is the case for Docker, this is how everything looks on my server:

[bmartino1]

Quote

Oh wait, we are talking about completely different use cases here.

You basically want a system that allows you to transfer your images to other machines correct?

My use case would be just to make pre-made container archives available for users to download and deploy through the CA App.

 

Correct, I was looking to share my created image to another unraid system as a group of friends and myself run similar services, dockers, lxc apps... and are in a ring for backups between each other. I was looking to copy or move my working lxc container to their system...

Incidently In a way this could also be done with the share CA premade container option. I'm still digging into how the containerize and code - run xyz and to make image for Unifi per your github and examples. as it gets trickey with the changing the Repository and how unifi is still using a bionic repo for the package and with ubuntu/debian specially jammy reaching potential EOL. with the recent change on secure repository and the apt-key sytem being deprecated. So i need to find the coands to add the repository and make a script that runs to make the CA container

 

Or export the already done container. In either case, its easiest on me to just type up a how to guide with each step pictures and commands. But as apps, repositories and version of Ubuntu change over time, that mileage may vary.

My other concerns ar, by the time I get to the lxc template.
Mongol db 4.4 will be os is about to hit EOL in April so that will be the next hurdle to jump which may change how I would want to have a client go to CA and download a premade image. Since Pete Asking has a docker version and it works well, but dev is slow I went LXC route.

I'm currently in dev on this and have not posted stuff to GitHub for a test of your wget lxc template and template add. ATM with the beta 7 release this has more of my attention and testing. so while i plan to do this dev on it has slowed/stopped.

 

This was why I was looking at the lxd commands to backup and move the image.

[ich777]

12 minutes ago, bmartino1 said:

This was why I was looking at the lxd commands to backup and move the image.

Again, lxc-autobackup is what you are looking for on Unraid.

 

12 minutes ago, bmartino1 said:

Correct, I was looking to share my created image to another unraid system as a group of friends and myself run similar services, dockers, lxc apps...

Then you should use the Backup function.

 

12 minutes ago, bmartino1 said:

my working lxc container to their system...

There is already a plugin out there that I look for feedback and I have currently stopped development because of real life but I will continue when everything calms a bit down:

 

12 minutes ago, bmartino1 said:

as it gets trickey with the changing the Repository

Why, you can always pass over a variable or set it in a file or even grab the new location automatically, there are so many ways to do it.

 

12 minutes ago, bmartino1 said:

Or export the already done container. In either case, its easiest on me to just type up a how to guide with each step pictures and commands. But as apps, repositories and version of Ubuntu change over time, that mileage may vary.

Again, what you find on my GitHub and as I explained above this is just a automated way of creating a container with all packages installed and the best thing is that you don't have to write tutorials, you do that right in the script that creates and packages the container.

 

12 minutes ago, bmartino1 said:

My other concerns ar, by the time I get to the lxc template.
Mongol db 4.4 will be os is about to hit EOL in April so that will be the next hurdle to jump which may change how I would want to have a client go to CA and download a premade image. Since Pete Asking has a docker version and it works well, but dev is slow I went LXC route.

LXC containers are static and similar to a VM, if you want control over what is installed then I would always choose a LXC container over a Docker container because you never know what breaking changes are introduced when making an update or the container even auto updating. That's why I have my Homeassistant and PiHole container in a LXC container because PiHole is way easier to manage in a LXC container than Docker and you don't have to deal with any port trickery or similar.

 

A user that installs a LXC container must be well aware that he is responsible for updates...

 

12 minutes ago, bmartino1 said:

I'm currently in dev on this and have not posted stuff to GitHub for a test of your wget lxc template and template add. ATM with the beta 7 release this has more of my attention and testing. so while i plan to do this dev on it has slowed/stopped.

I think you will find time for that because, the container is set up in 1 minute.

 

 

However again, just try it out and then we can talk, it is always a bad idea to talk about stuff when you didn't tried it and if you don't want to use my approach I'm also fine with that.

[Mainfrezzer]

Not sure if you noticed but it seems like ipv6 broke with Unraid 7. Absolutely no functionality apart from a local-link which is unusable.

[ich777]

39 minutes ago, Mainfrezzer said:

Not sure if you noticed but it seems like ipv6 broke with Unraid 7. Absolutely no functionality apart from a local-link which is unusable.

Everything is working over here.

 

TBH it is a little hard for me to test IPv6 since I only habe it internally but I can reach my containers through IPv6

 

What network type are you using? Bridge and IPVLAN or no bridge and MACVLAN?

 

In terms of network nothing changed.

[Mainfrezzer]

10 minutes ago, ich777 said:

 

 

What network type are you using? Bridge and IPVLAN or no bridge and MACVLAN?

 

I noticed while running on a macvlan bridge. Then i swapped back to ipvlan bridge, dummy-saved the change in lxc and still nothing. Only a downgrade to 6.12.10 brought the ipv6 functionality back. ( i didnt bother checking beta1)

[ich777]

3 minutes ago, Mainfrezzer said:

I noticed while running on a macvlan bridge. Then i swapped back to ipvlan bridge, dummy-saved the change in lxc and still nothing. Only a downgrade to 6.12.10 brought the ipv6 functionality back. ( i didnt bother checking beta1)

Sorry but I really can‘t test what‘s different since I don‘t have IPv6.

 

Can you be a bit more specific what‘s not working?

Does the container not get a IPv6 or can you not reach it?

 

Nothing changed in Unraid 7 in terms for IPv6 AFAIK.

[Mainfrezzer]

On 7/14/2024 at 9:47 AM, ich777 said:

 

 

Can you be a bit more specific what‘s not working?

Does the container not get a IPv6 or can you not reach it?

 

The container just start with ipv4 and the fe80:: address. They do not get a GUA or ULA Address. 


Edit: i upgraded straight back to 7 and this is how it looks:





Edit: Found a fix for the issue. Change your LXC config from

 

lxc.net.0.type = veth
lxc.net.0.flags = up
lxc.net.0.link = br0
lxc.net.0.name = eth0

to

 

lxc.net.0.type = macvlan
lxc.net.0.flags = up
lxc.net.0.link = br0
lxc.net.0.name = eth0

and its working again under unraid 7.

Edit-Edit: drawback of course is that the veth enabled communication between the host and container, that doesnt work with macvlan and requires the host access enabled


Edit-Edit-Edit. 

 

Quote

ip6tables -P FORWARD ACCEPT

Will resolve the issue with veth. The default policy for FORWARD changed from ACCEPT to DROP
 

Edited July 15 by Mainfrezzer
found a "fix" for unraid 7

[Grobalt]

is it possible to create / install lxc containers with the proxmox VE helper scripts ? on promox they run flawless but here on a "naked new lxc container" it seems they are not working:

 

https://tteck.github.io/Proxmox/#unifi-network-server-lxc

 

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/ct/unifi.sh)"
   __  __      _ _____
  / / / /__   (_) __(_)
 / / / / __ \/ / /_/ /
/ /_/ / / / / / __/ /
\____/_/ /_/_/_/ /_/

Loading...
/dev/fd/63: line 516: whiptail: command not found
⚠  User exited script
 

 

 

Edited July 30 by Grobalt

[bmartino1]

1 hour ago, Grobalt said:

is it possible to create / install lxc containers with the proxmox VE helper scripts ? on promox they run flawless but here on a "naked new lxc container" it seems they are not working:

 

https://tteck.github.io/Proxmox/#unifi-network-server-lxc

 

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/ct/unifi.sh)"
   __  __      _ _____
  / / / /__   (_) __(_)
 / / / / __ \/ / /_/ /
/ /_/ / / / / / __/ /
\____/_/ /_/_/_/ /_/

Loading...
/dev/fd/63: line 516: whiptail: command not found
⚠  User exited script
 

 

 

Install instruction for unif lxc here:

I'm working on a unraid template for this atm. Proxmox maybe...

https://github.com/bmartino1/Unraid-LXC-Unifi

[bmartino1]

5 hours ago, Grobalt said:

is it possible to create / install lxc containers with the proxmox VE helper scripts ? on promox they run flawless but here on a "naked new lxc container" it seems they are not working:

 

https://tteck.github.io/Proxmox/#unifi-network-server-lxc

 

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/ct/unifi.sh)"
   __  __      _ _____
  / / / /__   (_) __(_)
 / / / / __ \/ / /_/ /
/ /_/ / / / / / __/ /
\____/_/ /_/_/_/ /_/

Loading...
/dev/fd/63: line 516: whiptail: command not found
⚠  User exited script
 

 

 

I have setup a crude way to acomplish the template...

Create debain lxc, restore form a bakcup I made....

Otherwise install fresh:
https://community.ui.com/questions/UniFi-Installation-Scripts-or-UniFi-Easy-Update-Script-or-UniFi-Lets-Encrypt-or-UniFi-Easy-Encrypt-/ccbc7530-dd61-40a7-82ec-22b17f027776

[ich777]

8 hours ago, Grobalt said:

is it possible to create / install lxc containers with the proxmox VE helper scripts ? on promox they run flawless but here on a "naked new lxc container" it seems they are not working:

Nope, but I've created such a container archive for testing purposes, do the following:

1. Open up a Unraid terminal and execute:

   wget -O /tmp/lxc_container_template.xml https://raw.githubusercontent.com/ich777/unraid-lxc-unifi/main/lxc_container_template.xml

2. In your Browser navigate to: http://<YourunRAIDIP>/LXCAddTemplate
3. Make your changes if necessary
4. Click Apply
5. Wait for the Done button

 

After that you should be greeted with that:

 

[ich777]

2 hours ago, bmartino1 said:

I have setup a crude way to acomplish the template...

Please ask before publishing such a complicated thing and also this complicated tutorial here:

(btw, LXC is not a VM and there is still no official support for LXC templates so you can't post it easily in a section here on the forums)

 

I've now already created a PR on your repository, this gives you (the LXC archive maintainer) basically the ability to do the following:

Open up a Unraid terminal and do the following:

1. git clone https://github.com/ich777/unraid_lxc_unifi
2. cd unraid_lxc_unifi
3. bash createLXCarchive.sh

This will basically do 1. pull the repository from GitHub, 2. change directory into the cloned repository, 3. call the createLXCarchive.sh which will create a temporary LXC container, execute the scripts one by one in the build directory output the archive, build.log and a md5 sum into your LXC base directory in the sub folder "/cache/build_cache".

 

After that you (as the maintainer) have to go to the GitHub repository, click on Releases, click on Draft a new Release, Choose or create a Tag, put in a Title and maybe short description and upload the three files from the build_cache directory and finally Publish the Release.

(this could be all automated but that's up to you and always depends on your environment)

(please keep in mind I didn't changed the repository links since you haven't accepted the PR)

 

If you want to give a user access to your archive just post something like that:

These are only five (or better speaking really only three) easy steps for a user to do until official support for container templates is introduced.

 

 

The huge benefit that a user gets from this is that he gets a similar experience to a Docker container and he doesn't have to fiddle around with any scripts:

 

 

Please always upload the build.log for full transparency and that users can review what was done when executing the createLXCarchive.sh

 

You can also see my other build scripts/examples for other containers here.

[Grobalt]

Thanks for your collaboration, that would be awesome if all promox helper scripts can be used an easy way soon!

[ich777]

23 minutes ago, Grobalt said:

Thanks for your collaboration, that would be awesome if all promox helper scripts can be used an easy way soon!

The Proxmox helper scripts won't work on Unraid since they are working completely different and not all necessary tools are available on Unraid however you should be able to install the Unifi Network Server as pointed out here (I really try to make this as simple as possible for the end user) :

 

[Grobalt]

ok ... would have been great in combination with the lxc updater etc

[ich777]

7 minutes ago, Grobalt said:

ok ... would have been great in combination with the lxc updater etc

What is the LXC updater?

 

The above linked container image has a script installed that check on every Sunday at 0:00 for Unifi updates and installs them if any are found.

[Grobalt]

it is one of the "base" scripts on the proxmox  helper scrips side:

 

https://tteck.github.io/Proxmox/

https://tteck.github.io/Proxmox/#proxmox-ve-cron-lxc-updater

 

This script will add/remove a crontab schedule that updates all LXCs every Sunday at midnight.

To exclude LXCs from updating, edit crontab (crontab -e) and add CTID as shown in the example (-s 103 111)

 

[ich777]

16 minutes ago, Grobalt said:

it is one of the "base" scripts on the proxmox  helper scrips side:

But that does the script in my container too.

Evey Sunday at 0:00

 

It always depends on how you set it up, I would not like to update all of my containers automatically too much risk involved there (Homeassistant, PiHole,...), actually that was one of the reasons why I switched Homeassistant over to a LXC container...

[Grobalt]

sure, but it is a "central" script and you have like a config file where you exclude the "not wanted" LXCs  all good 

[ich777]

Just now, Grobalt said:

sure, but it is a "central" script and you have like a config file where you exclude the "not wanted" LXCs  all good 

The main issue is that this thing is for Proxmox and as said above dependencies for these scripts are missing and the scripts are for Proxmox, that's why I came up with the container archive solution which should work more reliable (at least there will be always be a image available) and everything is self contained where you also should be able to transfer your containers over to other systems too.

 

But please be aware that the container that I made is just a proof of concept for @bmartino1 and I didn't actually tested it because I don't have any Unifi hardware and I'm not familiar with Unifi.

I can tell you that it works and you can reach the WebUI

[bmartino1]

11 hours ago, ich777 said:

Please ask before publishing such a complicated thing and also this complicated tutorial here:

(btw, LXC is not a VM and there is still no official support for LXC templates so you can't post it easily in a section here on the forums)

 

I've now already created a PR on your repository, this gives you (the LXC archive maintainer) basically the ability to do the following:

Open up a Unraid terminal and do the following:

1. git clone https://github.com/ich777/unraid_lxc_unifi
2. cd unraid_lxc_unifi
3. bash createLXCarchive.sh

This will basically do 1. pull the repository from GitHub, 2. change directory into the cloned repository, 3. call the createLXCarchive.sh which will create a temporary LXC container, execute the scripts one by one in the build directory output the archive, build.log and a md5 sum into your LXC base directory in the sub folder "/cache/build_cache".

 

After that you (as the maintainer) have to go to the GitHub repository, click on Releases, click on Draft a new Release, Choose or create a Tag, put in a Title and maybe short description and upload the three files from the build_cache directory and finally Publish the Release.

(this could be all automated but that's up to you and always depends on your environment)

(please keep in mind I didn't changed the repository links since you haven't accepted the PR)

 

If you want to give a user access to your archive just post something like that:

These are only five (or better speaking really only three) easy steps for a user to do until official support for container templates is introduced.

 

 

The huge benefit that a user gets from this is that he gets a similar experience to a Docker container and he doesn't have to fiddle around with any scripts:

 

 

Please always upload the build.log for full transparency and that users can review what was done when executing the createLXCarchive.sh

 

You can also see my other build scripts/examples for other containers here.

Thank You Very Much For this! I now understand how this works and why the repository is set up the way it is and how the codes / line would execute to build/make the LXC on unraid.

[ich777]

Just now, bmartino1 said:

Thank You Very Much For this! I now understand how this works and why the repository is set up the way it is and how the codes / line would execute to build/make the LXC on unraid.

I've also left a comment on GitHub, please don't change the temporary container name, this is a safety feature.

Please read the comment here.

[bmartino1]

Hey @ich777

How would one custom set a icon?



If i want to change the icon in the lxc tab where is that data loading from?