Jump to content

Anyone got Oobabooga/text-generator-webui running in a container?


Go to solution Solved by Fredrick,

Recommended Posts

Hi guys!

I've actually spent two full nights now and am still very much unsuccessful in launching a container based on this github-repo

I can get it built using docker-compose in ssh on my server - the image is huge but I suspect that has something to do with it actually downloading a ubuntu-distro and huge CUDA libraries (?) into the docker.img. No matter I increased the size and tried it out. 

The web-server needs to find a language model that I'm downloading and storing in /mnt/user/appdata/oobabooga/models, but for some reason the web-server cant find my models. Think I'm messing up my mounts somehow. Here's the Docker-compose, i.e. it's fairly untouched apart from removing the GPU part and changing the /app/models mount. 

 

version: "3.3"
services:
  text-generation-webui:
    build:
      context: .
      args:
        # specify which cuda version your card supports: https://developer.nvidia.com/cuda-gpus
        TORCH_CUDA_ARCH_LIST: ${TORCH_CUDA_ARCH_LIST}
        WEBUI_VERSION: ${WEBUI_VERSION}
    env_file: .env
    ports:
      - "${HOST_PORT}:${CONTAINER_PORT}"
      - "${HOST_API_PORT}:${CONTAINER_API_PORT}"
    stdin_open: true
    tty: true
    volumes:
      - ./characters:/app/characters
      - ./extensions:/app/extensions
      - ./loras:/app/loras
      - /mnt/user/appdata/oobabooga/models:/app/models
      - ./presets:/app/presets
      - ./prompts:/app/prompts
      - ./softprompts:/app/softprompts
      - ./training:/app/training

 

Ideally I'd just have this entire image set up using Unraid frontend completely bypassing docker-compose, but my attempts at that solution was even more of a failure so I'm not sure there's any point in writing it up.

 

Anyone got any ideas where I'm going wrong or how I can get this set up in the simplest way? Anyone running text-generator-webui on Unraid? 

 

Thanks!

Link to comment
  • 8 months later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...