Anyone got Oobabooga/text-generator-webui running in a container?

[Fredrick]

Hi guys!

I've actually spent two full nights now and am still very much unsuccessful in launching a container based on this github-repo. 

I can get it built using docker-compose in ssh on my server - the image is huge but I suspect that has something to do with it actually downloading a ubuntu-distro and huge CUDA libraries (?) into the docker.img. No matter I increased the size and tried it out. 

The web-server needs to find a language model that I'm downloading and storing in /mnt/user/appdata/oobabooga/models, but for some reason the web-server cant find my models. Think I'm messing up my mounts somehow. Here's the Docker-compose, i.e. it's fairly untouched apart from removing the GPU part and changing the /app/models mount. 

 

version: "3.3"
services:
  text-generation-webui:
    build:
      context: .
      args:
        # specify which cuda version your card supports: https://developer.nvidia.com/cuda-gpus
        TORCH_CUDA_ARCH_LIST: ${TORCH_CUDA_ARCH_LIST}
        WEBUI_VERSION: ${WEBUI_VERSION}
    env_file: .env
    ports:
      - "${HOST_PORT}:${CONTAINER_PORT}"
      - "${HOST_API_PORT}:${CONTAINER_API_PORT}"
    stdin_open: true
    tty: true
    volumes:
      - ./characters:/app/characters
      - ./extensions:/app/extensions
      - ./loras:/app/loras
      - /mnt/user/appdata/oobabooga/models:/app/models
      - ./presets:/app/presets
      - ./prompts:/app/prompts
      - ./softprompts:/app/softprompts
      - ./training:/app/training

 

Ideally I'd just have this entire image set up using Unraid frontend completely bypassing docker-compose, but my attempts at that solution was even more of a failure so I'm not sure there's any point in writing it up.

 

Anyone got any ideas where I'm going wrong or how I can get this set up in the simplest way? Anyone running text-generator-webui on Unraid? 

 

Thanks!

[Fredrick]

I ended up going with a VM instead of Docker for this use case

[tigga69]

[tigga69]

thanks to Kixsume for the starting points.

 

https://github.com/oobabooga/text-generation-webui/issues/4850