Has anyone managed to configure this docker Lite LLM?

 

I didn't understand the manual and I couldn't configure the .yaml file

Did you ever get this figured out? Having similar issues

 

I updated this variable's container path AND value.  

I believe something is wrong with the template mounting the volume after some of the variables because I'm getting a oci runtime create failed  --config executable file not found in $Path: unknown 

Getting the same error.  Anyone figured out the solution?

Having the same issue. 

If I do change the mapping directly to the file, not the directory (like @TD2779 mentioned) :

at least I am getting a step further. But docker then is complaining it would be a directory?

 

Edited March 15, 20251 yr by seppo
clarification

8 hours ago, seppo said:

Having the same issue. 

If I do change the mapping directly to the file, not the directory (like @TD2779 mentioned) :

at least I am getting a step further. But docker then is complaining it would be a directory?

 

Ok, one step closer!

You need to directly point it to the config.yaml BUT it has to be in a subfolder!

Putting it in "configDir" solved this for me.
so the full path now is: "/mnt/user/appdata/LiteLLM/configDir/config.yaml" see screenshot.

Only now I am stuck with getting the config right 🤪

This unraid template is the least straight forward. At least for me.  

Ya I came to the same conclusion and got it working by mapping the config directly into the container in a subfolder then got stuck with the same database_url error. Apparently litellm needs a database so you will need to add a connection string to tell litellm where the database is. I configured it by using the docs and connecting it to a Postgres16 container. 

 

Connection string should look like this and be added as a line into the config.yaml where the commented out database url is. 

 

database_url: "postgresql://<user>:<password>@<host>:<port>/<dbname>" 
 

Example

database_url: "postgresql://admin:[email protected]:5432/litellm"

Came here after watching NetworkChucks Video
https://www.youtube.com/watch?v=nQCOTzS5oU0

The Unraid app template is certianly broken. But only needs a few tweaks.
My steps to get it working

Step1 : 
Create postgresql16 docker
- Pick a dir that does not conflict with any other postgres16 you may have (important if you do)
- Create = user / pass / database name



Step2:
Create LiteLLM config in appdata folder to refrence in step 3
- /mnt/user/appdata/LiteLLM/configDir/litellm_config.yaml
- create your own key
- enter your postgres details as above in Step1
 

Quote

 

general_settings:

  master_key: sk-XXXXXXXXX

  store_model_in_db: True

  database_url: "postgresql://<username>:<password>@<IP>:<port>/<database>"

 

Step3:

Create LiteLLM Docker
- I had to use different tag due to user issue when loding URL. You can try latest, but if you fail copy as below (ghcr.io/berriai/litellm:main-v1.63.7-nightly)

- Fix dir structure for config: /mnt/user/appdata/LiteLLM/configDir/litellm_config.yaml > /app/config.yaml

Step4:
Log in to web GUI with admin / <master key set in config>
The rest you can set up in gui

Edited March 16, 20251 yr by Raider_M

Thank you Raider_M, your post helped me to achieve my goal, very detailed and informative. People like you and all the other supportive guys in this forum make the Unraid community great!

Thank you for the help! It at least installed. But the webui won't load.

@Raider_M 

@JamieS

 

Hello, did either of you find this to be worthwhile in the end? I don’t mind putting in the work to get one of these working, but I’d love for that time to go towards a good solution. I’m struggling to figure out which software is the most useful as I see fairly equal votes for this, Librechat, and open webui plus some others. I primarily use Claude and gpt4o if that’s relevant. Thanks for any insight you can provide!!

I have followed the above guide from @Raider_M and had some issue that seems be solved now, but then another problem came after the docker runs for about 20-30 seconds. Anyone know a fix for that?

 

Thx this solved it for me :

1. Manual make a directory (liteconfigdir)
   

2. make yamle fith name config.yaml not another name... didnt work at my Unraid
   

3. make a new Variable Field and give it the $LITELLM_DATABASE_PASSWORD and your real PW so use it

yaml file:

GNU nano 8.4 config.yaml

general_settings:

master_key: sk-111222333444 #your set masterkey

store_model_in_db: True

database:

type: "postgres" #i used postgres if you use mysql than change this to it

host: "localhost"

port: 5555 #the port my db is listening

username: "admin" #Username of DB

password: "$LITELLM_DATABASE_PASSWORD" #Password is saved in variable in Unraid Screen

database_name: "LiteLLM" #the set DB name

Edited May 22, 20251 yr by Olivilo

Okay so after following the steps, I have gotten LiteLLM to run, but when I go to the ui page my browser shows a blank page.... what gives?

When I look at the logs when trying to launch the ui I get this...

LiteLLM: Proxy initialized with Config, Set models:

gpt-3.5-turbo-end-user-test

gpt-3.5-turbo-end-user-test

gpt-3.5-turbo

gpt-3.5-turbo-large

gpt-4

sagemaker-completion-model

text-embedding-ada-002

dall-e-2

openai-dall-e-3

fake-openai-endpoint

fake-openai-endpoint-2

fake-openai-endpoint-3

fake-openai-endpoint-4

fake-openai-endpoint-3

bad-model

good-model

*

anthropic/*

bedrock/*

groq/*

mistral-embed

gpt-instruct

fake-openai-endpoint-5

badly-configured-openai-endpoint

INFO: 10.0.0.15:61841 - "GET /ui HTTP/1.1" 307 Temporary Redirect

INFO: 10.0.0.15:61841 - "GET /ui/ HTTP/1.1" 200 OK

INFO: 10.0.0.15:61841 - "GET /litellm2/_next/static/chunks/webpack-a426aae3231a8df1.js HTTP/1.1" 404 Not Found

INFO: 10.0.0.15:61843 - "GET /litellm2/_next/static/chunks/fd9d1056-205af899b895cbac.js HTTP/1.1" 404 Not Found

INFO: 10.0.0.15:61844 - "GET /litellm2/_next/static/chunks/117-c4922b1dd81b62ce.js HTTP/1.1" 404 Not Found

INFO: 10.0.0.15:61845 - "GET /litellm2/_next/static/chunks/main-app-4f7318ae681a6d94.js HTTP/1.1" 404 Not Found

INFO: 10.0.0.15:61841 - "GET /litellm2/_next/static/chunks/fd9d1056-205af899b895cbac.js HTTP/1.1" 404 Not Found

INFO: 10.0.0.15:61843 - "GET /litellm2/_next/static/chunks/117-c4922b1dd81b62ce.js HTTP/1.1" 404 Not Found

INFO: 10.0.0.15:61844 - "GET /litellm2/_next/static/chunks/main-app-4f7318ae681a6d94.js HTTP/1.1" 404 Not Found

INFO: 10.0.0.15:61841 - "GET /litellm2/_next/static/chunks/webpack-a426aae3231a8df1.js HTTP/1.1" 404 Not Found

17:59:02 - LiteLLM Proxy:DEBUG: hanging_request_check.py:148 - Checking for hanging requests....

18:01:32 - LiteLLM Proxy:DEBUG: hanging_request_check.py:148 - Checking for hanging requests....

It looks like there are some .js files (javascript files maybe?) that is not finding... I am not sure what I have done wrong. Any help would be appreciated.

Thanks in advance.

On 4/22/2025 at 3:37 PM, jss said:

@Raider_M 

@JamieS

 

Hello, did either of you find this to be worthwhile in the end? I don’t mind putting in the work to get one of these working, but I’d love for that time to go towards a good solution. I’m struggling to figure out which software is the most useful as I see fairly equal votes for this, Librechat, and open webui plus some others. I primarily use Claude and gpt4o if that’s relevant. Thanks for any insight you can provide!!

I'm also in the same boat.... any opinions?

I got it to work.

The docker container is weird because it creates config.yaml as a directory instead of a file. The problem is the path creation.

It is very easy, Here is the complete guide on how to fix it.

Install Postgresql 16

Nothing fancy here, stock standard.

(optional) if you don't know how to use postgresql, install pgadmin. Standard install, login, create a database, create/assign whatever user you want or leave it the default postgres with the password postgres and be done with it. You don't need to do anything crazy

Install LiteLLM - Guide

1 Install the docker container as it is (it will fail) - that's ok

Let the container fail, that's fine, you will need to tweak a couple of things.

2 Remove the config directory and replace it with litellm_config.yaml

3 Edit litellm_config.yaml

Add the following to litellm_config.yaml

general_settings:
 master_key: sk-112233(whatever-this-will-be-your-login-password)
 store_model_in_db: True
 database_url: "postgresql://postgres:[email protected]:5432/litellm" 

The guide to follow for the last line (database_url) is this:  database_url: "postgresql://<username>:<password>@<IP>:<port>/<database>"

My server doesn't want to accept localhost as the IP address so I wrote down the IP address of the server.

4 Update the LitleLLM container with the new variable (This is where it fails)

Edit the config path.

Enjoy LiteLLM.

Edited June 15, 20251 yr by Jules
wording

Thank you!!! I got it working, thanks to your awesome instructions!!!

On 6/14/2025 at 4:34 PM, Jules said:

I got it to work.

The docker container is weird because it creates config.yaml as a directory instead of a file. The problem is the path creation.

It is very easy, Here is the complete guide on how to fix it.

Install Postgresql 16

Nothing fancy here, stock standard.

(optional) if you don't know how to use postgresql, install pgadmin. Standard install, login, create a database, create/assign whatever user you want or leave it the default postgres with the password postgres and be done with it. You don't need to do anything crazy

Install LiteLLM - Guide

1 Install the docker container as it is (it will fail) - that's ok

Let the container fail, that's fine, you will need to tweak a couple of things.

2 Remove the config directory and replace it with litellm_config.yaml

3 Edit litellm_config.yaml

Add the following to litellm_config.yaml

general_settings:
 master_key: sk-112233(whatever-this-will-be-your-login-password)
 store_model_in_db: True
 database_url: "postgresql://postgres:[email protected]:5432/litellm" 

The guide to follow for the last line (database_url) is this:  database_url: "postgresql://<username>:<password>@<IP>:<port>/<database>"

My server doesn't want to accept localhost as the IP address so I wrote down the IP address of the server.

4 Update the LitleLLM container with the new variable (This is where it fails)

Edit the config path.

Enjoy LiteLLM.

Thanks !
This worked for me too

Side note: a database is not strictly needed for LiteLLM to work. You do need it if you want to use the GUI, though.