Allow template repositories to be hosted from other sources


LTM

Recommended Posts

I would like to propose a feature that modifies the "Template Repositories" section on the Docker page. I have containers that are modified to my needs that I would like to host on a private git server (using Gitea). I have tried adding a template repository to this section just to find it not working. I searched the forums and have found a few (rather old) posts saying that you can only use GitHub to host the template repositories and that it has to be public. 

 

The problem I am seeing and why I want to use my own server is because if you post all of your repository templates on GitHub without going through and scrubbing all of your passwords, application keys, hashes, etc., you will be posting all of that stuff publicly. 

 

In my case, I am hosting a Gitea instance on a separate device that is only accessible on my local network. I currently have all my XMLs saved to a repository on that server. It would be really nice to be able to just add the repo url and restore all of my containers in case of a catastrophic failure.

 

GitLab support would be another integration that I can see a lot of other people wanting.

Link to comment

Beginning with Unraid 6.10.0-rc1, the entire Template Repositories section has been removed (Thanks be to JoBu)

 

2 Choices:

  1. save the xml's within /config/plugins/dockerMan/templates-user on the flash drive and use the Add Container Button
  2. save the xml's within /config/plugins/community.applications/private/LTM on the flash drive and use CA to manage them (If not categorized, they will appear within the Private Category)

 

  • Like 2
Link to comment
On 8/5/2021 at 3:53 PM, Squid said:

Beginning with Unraid 6.10.0-rc1, the entire Template Repositories section has been removed (Thanks be to JoBu)

 

2 Choices:

  1. save the xml's within /config/plugins/dockerMan/templates-user on the flash drive and use the Add Container Button
  2. save the xml's within /config/plugins/community.applications/private/LTM on the flash drive and use CA to manage them (If not categorized, they will appear within the Private Category)

 

So in the future, we just need to edit these files manually versus having it on the web page? What is the reason for removing this?

Link to comment

Template Repositories has effectively been deprecated since 2015 when Community Applications was first introduced - Even CA's predecessor Community Repositories (which managed repositories) only lasted for 2 months.  (And the coding of the entire template system when Unraid 6 was introduced was designed around the ability to be able to have at one some a "Store" like experience.  The use of repositories was a stop-gap and easy solution vs the massive investment in both time and resources that CA required.  It was never meant to be a permanent fixture IMO)  Repositories which are not within CA which you would maintain yourself are exceedingly rare, and 2 options exist for management of them without having the Repositories section.

 

CA hides the underlying complexity from using Repositories from the user and is a far better UI than Repositories ever offered (or could offer)

 

There are also some issues with the coding that isn't worth the time or effort to fix (due to the above).

 

The only fundamental difference is that instead of storing the xml's within a GitHub repository you maintain you are storing them locally on the flash drive

Link to comment
On 8/8/2021 at 7:36 AM, Squid said:

The only fundamental difference is that instead of storing the xml's within a GitHub repository you maintain you are storing them locally on the flash drive

So what happens if the flash drive fails? Are the templates backed up with the rest of the flash drive when you create a backup? Are they uploaded to the new UPC? Because if this is the case, then I don't want them uploaded. I would much rather keep doing what I am doing and making periodic backups and storing them on other devices on my network and a backup abroad. I'm sure I can automate this somehow, but I have not had the time to look in to it.

 

I honestly don't even like how CA templates work. I would much rather just make a compose file and be able to do it that way instead of having to use the GUI or code it in XML. I would be able to get things working much faster to be able to add a line of code instead of going through the steps of having to edit an environment variable through the GUI. But I digress.

 

Again, some of my containers are modified to my needs. Settings, paths, and variables which includes passwords and sensitive info that I don't want getting out. Every time I edit a container, I have to start it, make sure things are working, go back in to the edit screen, save the xml file, copy and paste it in to VS Code, and git psuh it to my Gitea server. I do this in case my flash drive fails. It would be really nice to just be able to enter in the repo URL and have all of my modified templates just there to hit start and be back up and running when the time comes.

 

Since the devs have decided to remove this feature already, I guess anything I say is just going to be discarded. 

Edited by LTM
Link to comment
9 hours ago, LTM said:

I honestly don't even like how CA templates work. I would much rather just make a compose file and be able to do it that way instead of having to use the GUI or code it in XML. I would be able to get things working much faster to be able to add a line of code instead of going through the steps of having to edit an environment variable through the GUI. But I digress.

You're not alone in this, lots of others would prefer this, compared to the template mess...I have no idea though on how to convince them to go with the more commonly used compose way...would make life so much easier for so many. 

Link to comment

I meant to say "going to be discarded" in my last post... Was wording it differently and forgot to delete the "not".

 

3 hours ago, hotio said:

I have no idea though on how to convince them to go with the more commonly used compose way...would make life so much easier for so many. 

I guess just rally the troops?

 

The only reasons why I can think is that they chose XML is because it would be easier to import data from an XML file than to import a compose file. But a compose file would still be decently easy enough to import from since there is structured formatting. Plus, it would make it a lot easier for people that are new to unraid to get going faster. The other reason why I think they went for XML is because of the extra data that would be unraid specific like the description, icon url, categories, etc.

 

I can see the current way for those that are not savvy enough to learn how to create a compose file or just like doing things through a UI. But then again, I feel like it would be easy enough to let everyone have the option to choose between a UI that converts the fields to a compose file or just have a text field that lets the users post a compose file directly.

 

But again, all moot points since the template section is going away.

Link to comment

I'm super confused with this. 

So, basically now you download an app and that's it? How are you going to modify paths, add variables, etc. Like now with the templates from the GUI. I think all of my apps have stuff modified when it comes to variables and paths, hell some are even in different folders for the config file. 

 

 

Link to comment

 

On 8/8/2021 at 7:36 AM, Squid said:

Template Repositories has effectively been deprecated since 2015 when Community Applications was first introduced

 

On 8/9/2021 at 12:16 PM, trurl said:

The template section is going away but templates are not.

 

1 hour ago, gacpac said:

I'm super confused with this. 

Probably you never even used Template Repositories. It was another way to get Templates for the Add Container page before the Apps page was developed.

 

And selecting a template from the dropdown at the top of the Add Container page isn't needed even to reinstall a container from your saved templates because that is taken care of by Previous Apps on the Apps page. Nobody even talks about the Select a template dropdown anymore.

Link to comment
30 minutes ago, trurl said:

 

 

 

Probably you never even used Template Repositories. It was another way to get Templates for the Add Container page before the Apps page was developed.

 

And selecting a template from the dropdown at the top of the Add Container page isn't needed even to reinstall a container from your saved templates because that is taken care of by Previous Apps on the Apps page. Nobody even talks about the Select a template dropdown anymore.

 

So in removing the template tab, i'm forced to go to the community applications and restore apps so to speak. I can work with that, and If i understand what you are saying then, i can still click edit app and modify the template's host paths, ports, variables, etc. 

 

I'm going through the scenarios in my head in case my usb breaks or something. I'm also using the appdatabackup plugin, that it's literally saving the templates to be restored from the community apps

Link to comment

Oh yes! Agree, 

 

It's that I need to understand the process beforehand. Normally I never go to the previous apps at all. 

 

Quote ---- Appdatabackup plugin

General Procedure On How To Restore Your Appdata

Replace your cache drive if necessary

Recreate your appdata share (and set it to be cache only)

Recreate your docker.img file if necessary (ideally, this should be performed regardless)

Select a backup set (if using dated backup sets) and hit 'Restore'

Sit back and have a coffee

Ideally restart your server

Re-add your previously installed docker applications via CA's Previous Apps section. No adjustment of the templates should be necessary

Edited by gacpac
Link to comment
  • 1 month later...
On 8/5/2021 at 11:53 PM, Squid said:

save the xml's within /config/plugins/community.applications/private/LTM on the flash drive and use CA to manage them (If not categorized, they will appear within the Private Category)

Is there a way to fully emulate, and test how a template would be shown if it was a "true" CA template? mostly wanting to test Branches, and the extra links showing in the CA card

Link to comment

Using the xml within a private repository (as detailed) is the easiest way (although it bypasses some of the modifications / checks that the feed does)

 

Alternatively, you could publish the template via the normal channels but add in to the template <MaxVer>6.0.0</MaxVer>  It'll appear within CA if you turn on Display Incompatible Apps (which most users don't have turned on)

Link to comment
  • 3 months later...

I'm unable to find my private XML file in Community Applications (v2022.01.07) in Unraid (v6.10-rc2); could someone help me understand what I'm missing?

 

I've put my template in the folder on the flash drive, as mentioned at the top of this thread:

```bash

$ pwd
/boot/config/plugins/community.applications/private/LTM
$ ls
jan-poka.xml

```

 

But in Community Applications there's no "Private" category, and searching for anything related to this template doesn't show it — even after uninstalling/reinstalling Community Applications. 

Edited by jphastings
Added extra steps I've tried
Link to comment

Thanks! It's linked in the post above, but you helped me realise validation might prevent it from appearing! Either the malformed `Icon` url (missing the 'h' of 'https') or the use of html escaping ('&ldquo;') clearly weren't considered valid — changing them made it appear in Private Apps. Thank you 😊

Link to comment
  • 2 weeks later...
  • 3 months later...

Since this just came up again on a different thread and we are now on the cusp of 6.10.0 stable being released soon(tm), let me just re-iterate the following:

 

The Template Repositories section of the OS is now removed in 6.10.0+  It is not coming back.

 

It is not a "feature" being removed, but rather a reflection of reality of how the system was originally designed (to support an App Store type interface) and where we have been since 2015 (an App Store type interface released).  Template Repositories was always intended to be a short term solution.

 

Since 2015 and the introduction of Community Applications, a reasonable assumption is that 99.999% of users have never used the Template Repositories section.

 

If you are one of the 0.001% of users that is still currently using it, there are 2 completely viable alternatives listed in this thread (put the xml's into either /config/plugins/dockerMan/templates-user or /config/plugins/community.applications/private/myNameHere)

 

NOTE: another reasonable assumption is that of the 0.001% of the users who think that they are using it, 90% of them actually aren't.

 

The reason why it's being removed in 2022 instead of being removed in 2015 is because prior to 6.10.0 Community Applications was basically an optional plugin.  In 6.10.0, while CA is still technically optional, the Apps Tab will always appear in the GUI and if CA is not installed then there is a link to automatically install it.

 

Community Applications does not force you to only use templates which are present within it, nor does it hinder you creating your own templates via "Add Container".

 

  • Like 2
Link to comment
  • 2 weeks later...

Quick question @Squid

/config/plugins/dockerMan/templates-user
/config/plugins/community.applications/private/myNameHere

Is there any functional difference between the 2 folders?

For example, with the latter option, do templates get modified in this folder, or do they copy to template-users and retain the original template.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.