April 4, 20251 yr Has anyone gotten this to work? I know the kernel now supports this but I’m still having trouble getting the you recognized I have the intel top on the latest version
April 5, 20251 yr Community Expert 9 hours ago, ceverson said: Has anyone gotten this to work? I know the kernel now supports this but I’m still having trouble getting the you recognized I have the intel top on the latest version A770 was supported before 7.1 what does container need have you added device path to container /dev/dri
April 7, 20251 yr Community Expert You should have added a device instead of a path. There are two easy ways of doing it. Option 1: Option 2:
April 11, 20251 yr Author sorry just getting back to this but these are the logs still seems to use my apu which is also intel is there something i need to do to say dont use apu and use the 2 gpus text error warn system array login 2025/04/11 00:18:58 routes.go:999: INFO server config env="map[OLLAMA_DEBUG:false OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:2 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[* http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:*] OLLAMA_RUNNERS_DIR: OLLAMA_TMPDIR:]" time=2025-04-11T00:18:58.298Z level=INFO source=images.go:697 msg="total blobs: 10" time=2025-04-11T00:18:58.298Z level=INFO source=images.go:704 msg="total unused blobs removed: 0" time=2025-04-11T00:18:58.298Z level=INFO source=routes.go:1044 msg="Listening on [::]:11434 (version 0.0.0)" time=2025-04-11T00:18:58.298Z level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama1531105704/runners time=2025-04-11T00:18:58.343Z level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2]" time=2025-04-11T00:18:58.343Z level=INFO source=gpu.go:122 msg="Detecting GPUs" time=2025-04-11T00:18:58.344Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2" :: initializing oneAPI environment ... entrypoint.sh: BASH_VERSION = 5.1.16(1)-release args: Using "$@" for setvars.sh arguments: serve :: ccl -- latest :: compiler -- latest :: dal -- latest :: debugger -- latest :: dev-utilities -- latest :: dnnl -- latest :: dpcpp-ct -- latest :: dpl -- latest :: ipp -- latest :: ippcp -- latest :: mkl -- latest :: mpi -- latest :: tbb -- latest :: oneAPI environment initialized :: [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached. [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production. - using env: export GIN_MODE=release - using code: gin.SetMode(gin.ReleaseMode) [GIN-debug] POST /api/pull --> github.com/ollama/ollama/server.(*Server).PullModelHandler-fm (5 handlers) [GIN-debug] POST /api/generate --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers) [GIN-debug] POST /api/chat --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers) [GIN-debug] POST /api/embeddings --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers) [GIN-debug] POST /api/create --> github.com/ollama/ollama/server.(*Server).CreateModelHandler-fm (5 handlers) [GIN-debug] POST /api/push --> github.com/ollama/ollama/server.(*Server).PushModelHandler-fm (5 handlers) [GIN-debug] POST /api/copy --> github.com/ollama/ollama/server.(*Server).CopyModelHandler-fm (5 handlers) [GIN-debug] DELETE /api/delete --> github.com/ollama/ollama/server.(*Server).DeleteModelHandler-fm (5 handlers) [GIN-debug] POST /api/show --> github.com/ollama/ollama/server.(*Server).ShowModelHandler-fm (5 handlers) [GIN-debug] POST /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers) [GIN-debug] HEAD /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers) [GIN-debug] POST /v1/chat/completions --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers) [GIN-debug] GET / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] GET /api/tags --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers) [GIN-debug] GET /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) [GIN-debug] HEAD / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] HEAD /api/tags --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers) [GIN-debug] HEAD /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) text error warn system array login 2025/04/11 00:18:58 routes.go:999: INFO server config env="map[OLLAMA_DEBUG:false OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:2 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[* http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:*] OLLAMA_RUNNERS_DIR: OLLAMA_TMPDIR:]" time=2025-04-11T00:18:58.298Z level=INFO source=images.go:697 msg="total blobs: 10" time=2025-04-11T00:18:58.298Z level=INFO source=images.go:704 msg="total unused blobs removed: 0" time=2025-04-11T00:18:58.298Z level=INFO source=routes.go:1044 msg="Listening on [::]:11434 (version 0.0.0)" time=2025-04-11T00:18:58.298Z level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama1531105704/runners time=2025-04-11T00:18:58.343Z level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2]" time=2025-04-11T00:18:58.343Z level=INFO source=gpu.go:122 msg="Detecting GPUs" time=2025-04-11T00:18:58.344Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2" :: initializing oneAPI environment ... entrypoint.sh: BASH_VERSION = 5.1.16(1)-release args: Using "$@" for setvars.sh arguments: serve :: ccl -- latest :: compiler -- latest :: dal -- latest :: debugger -- latest :: dev-utilities -- latest :: dnnl -- latest :: dpcpp-ct -- latest :: dpl -- latest :: ipp -- latest :: ippcp -- latest :: mkl -- latest :: mpi -- latest :: tbb -- latest :: oneAPI environment initialized :: [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached. [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production. - using env: export GIN_MODE=release - using code: gin.SetMode(gin.ReleaseMode) [GIN-debug] POST /api/pull --> github.com/ollama/ollama/server.(*Server).PullModelHandler-fm (5 handlers) [GIN-debug] POST /api/generate --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers) [GIN-debug] POST /api/chat --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers) [GIN-debug] POST /api/embeddings --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers) [GIN-debug] POST /api/create --> github.com/ollama/ollama/server.(*Server).CreateModelHandler-fm (5 handlers) [GIN-debug] POST /api/push --> github.com/ollama/ollama/server.(*Server).PushModelHandler-fm (5 handlers) [GIN-debug] POST /api/copy --> github.com/ollama/ollama/server.(*Server).CopyModelHandler-fm (5 handlers) [GIN-debug] DELETE /api/delete --> github.com/ollama/ollama/server.(*Server).DeleteModelHandler-fm (5 handlers) [GIN-debug] POST /api/show --> github.com/ollama/ollama/server.(*Server).ShowModelHandler-fm (5 handlers) [GIN-debug] POST /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers) [GIN-debug] HEAD /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers) [GIN-debug] POST /v1/chat/completions --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers) [GIN-debug] GET / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] GET /api/tags --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers) [GIN-debug] GET /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) [GIN-debug] HEAD / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] HEAD /api/tags --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers) [GIN-debug] HEAD /api/version --> githu
April 11, 20251 yr Author text error warn system array login 2025/04/11 00:48:45 routes.go:999: INFO server config env="map[OLLAMA_DEBUG:false OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:2 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[* http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:*] OLLAMA_RUNNERS_DIR: OLLAMA_TMPDIR:]" time=2025-04-11T00:48:45.725Z level=INFO source=images.go:697 msg="total blobs: 10" time=2025-04-11T00:48:45.725Z level=INFO source=images.go:704 msg="total unused blobs removed: 0" time=2025-04-11T00:48:45.725Z level=INFO source=routes.go:1044 msg="Listening on [::]:11434 (version 0.0.0)" time=2025-04-11T00:48:45.725Z level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama418901386/runners time=2025-04-11T00:48:45.769Z level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2]" time=2025-04-11T00:48:45.769Z level=INFO source=gpu.go:122 msg="Detecting GPUs" time=2025-04-11T00:48:45.771Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2" :: initializing oneAPI environment ... entrypoint.sh: BASH_VERSION = 5.1.16(1)-release args: Using "$@" for setvars.sh arguments: serve :: ccl -- latest :: compiler -- latest :: dal -- latest :: debugger -- latest :: dev-utilities -- latest :: dnnl -- latest :: dpcpp-ct -- latest :: dpl -- latest :: ipp -- latest :: ippcp -- latest :: mkl -- latest :: mpi -- latest :: tbb -- latest :: oneAPI environment initialized :: [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached. [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production. - using env: export GIN_MODE=release - using code: gin.SetMode(gin.ReleaseMode) [GIN-debug] POST /api/pull --> github.com/ollama/ollama/server.(*Server).PullModelHandler-fm (5 handlers) [GIN-debug] POST /api/generate --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers) [GIN-debug] POST /api/chat --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers) [GIN-debug] POST /api/embeddings --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers) [GIN-debug] POST /api/create --> github.com/ollama/ollama/server.(*Server).CreateModelHandler-fm (5 handlers) [GIN-debug] POST /api/push --> github.com/ollama/ollama/server.(*Server).PushModelHandler-fm (5 handlers) [GIN-debug] POST /api/copy --> github.com/ollama/ollama/server.(*Server).CopyModelHandler-fm (5 handlers) [GIN-debug] DELETE /api/delete --> github.com/ollama/ollama/server.(*Server).DeleteModelHandler-fm (5 handlers) [GIN-debug] POST /api/show --> github.com/ollama/ollama/server.(*Server).ShowModelHandler-fm (5 handlers) [GIN-debug] POST /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers) [GIN-debug] HEAD /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers) [GIN-debug] POST /v1/chat/completions --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers) [GIN-debug] GET / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] GET /api/tags --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers) [GIN-debug] GET /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) [GIN-debug] HEAD / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] HEAD /api/tags --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers) [GIN-debug] HEAD /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) [GIN] 2025/04/11 - 00:56:05 | 200 | 23.105µs | 127.0.0.1 | HEAD "/" [GIN] 2025/04/11 - 00:56:05 | 200 | 1.577391ms | 127.0.0.1 | POST "/api/show" [GIN] 2025/04/11 - 00:56:05 | 200 | 289.963µs | 127.0.0.1 | POST "/api/show" time=2025-04-11T00:56:05.135Z level=INFO source=gpu.go:122 msg="Detecting GPUs" time=2025-04-11T00:56:05.136Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2025-04-11T00:56:05.673Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2025-04-11T00:56:05.673Z level=INFO source=server.go:327 msg="starting llama server" cmd="/tmp/ollama418901386/runners/cpu_avx2/ollama_llama_server --model /root/.ollama/models/blobs/sha256-102a747c137683e81d431dab05d8f2158df4ab6f162f8f9019425a43d51e0e9f --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 999 --parallel 1 --port 45475" time=2025-04-11T00:56:05.674Z level=INFO source=sched.go:326 msg="loaded runners" count=1 time=2025-04-11T00:56:05.674Z level=INFO source=server.go:495 msg="waiting for llama runner to start responding" {"function":"server_params_parse","level":"INFO","line":2604,"msg":"logging to file is disabled.","tid":"22744593221632","timestamp":1744332965} {"build":1,"commit":"baa5868","function":"main","level":"INFO","line":2821,"msg":"build info","tid":"22744593221632","timestamp":1744332965} {"function":"main","level":"INFO","line":2828,"msg":"system info","n_threads":8,"n_threads_batch":-1,"system_info":"AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | ","tid":"22744593221632","timestamp":1744332965,"total_threads":20} llama_model_loader: loaded meta data with 40 key-value pairs and 363 tensors from /root/.ollama/models/blobs/sha256-102a747c137683e81d431dab05d8f2158df4ab6f162f8f9019425a43d51e0e9f (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = llama llama_model_loader: - kv 1: general.type str = model llama_model_loader: - kv 2: general.name str = Mistral Small 24B Instruct 2501 llama_model_loader: - kv 3: general.version str = 2501 llama_model_loader: - kv 4: general.finetune str = Instruct llama_model_loader: - kv 5: general.basename str = Mistral-Small llama_model_loader: - kv 6: general.size_label str = 24B llama_model_loader: - kv 7: general.license str = apache-2.0 llama_model_loader: - kv 8: general.base_model.count u32 = 1 llama_model_loader: - kv 9: general.base_model.0.name str = Mistral Small Base 2501 llama_model_loader: - kv 10: general.base_model.0.version str = 2501 llama_model_loader: - kv 11: general.base_model.0.organization str = Mistralai llama_model_loader: - kv 12: general.base_model.0.repo_url str = https://huggingface.co/mistralai/Mist... llama_model_loader: - kv 13: general.languages arr[str,10] = ["en", "fr", "de", "es", "it", "pt", ... llama_model_loader: - kv 14: llama.block_count u32 = 40 llama_model_loader: - kv 15: llama.context_length u32 = 32768 llama_model_loader: - kv 16: llama.embedding_length u32 = 5120 llama_model_loader: - kv 17: llama.feed_forward_length u32 = 32768 llama_model_loader: - kv 18: llama.attention.head_count u32 = 32 llama_model_loader: - kv 19: llama.attention.head_count_kv u32 = 8 llama_model_loader: - kv 20: llama.rope.freq_base f32 = 100000000.000000 llama_model_loader: - kv 21: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 llama_model_loader: - kv 22: llama.attention.key_length u32 = 128 llama_model_loader: - kv 23: llama.attention.value_length u32 = 128 llama_model_loader: - kv 24: general.file_type u32 = 15 llama_model_loader: - kv 25: llama.vocab_size u32 = 131072 llama_model_loader: - kv 26: llama.rope.dimension_count u32 = 128 llama_model_loader: - kv 27: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 28: tokenizer.ggml.pre str = qwen2 llama_model_loader: - kv 29: tokenizer.ggml.tokens arr[str,131072] = ["<unk>", "<s>", "</s>", "[INST]", "[... llama_model_loader: - kv 30: tokenizer.ggml.token_type arr[i32,131072] = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... llama_model_loader: - kv 31: tokenizer.ggml.merges arr[str,269443] = ["Ġ Ġ", "Ġ t", "e r", "i n", "Ġ �... llama_model_loader: - kv 32: tokenizer.ggml.bos_token_id u32 = 1 llama_model_loader: - kv 33: tokenizer.ggml.eos_token_id u32 = 2 llama_model_loader: - kv 34: tokenizer.ggml.unknown_token_id u32 = 0 llama_model_loader: - kv 35: tokenizer.ggml.add_bos_token bool = true llama_model_loader: - kv 36: tokenizer.ggml.add_eos_token bool = false llama_model_loader: - kv 37: tokenizer.chat_template str = {{ bos_token }}{% for message in mess... llama_model_loader: - kv 38: tokenizer.ggml.add_space_prefix bool = false llama_model_loader: - kv 39: general.quantization_version u32 = 2 llama_model_loader: - type f32: 81 tensors llama_model_loader: - type q4_K: 241 tensors llama_model_loader: - type q6_K: 41 tensors llama_model_load: error loading model: error loading model hyperparameters: invalid n_rot: 128, expected 160 llama_load_model_from_file: failed to load model llama_init_from_gpt_params: error: failed to load model '/root/.ollama/models/blobs/sha256-102a747c137683e81d431dab05d8f2158df4ab6f162f8f9019425a43d51e0e9f' {"function":"load_model","level":"ERR","line":410,"model":"/root/.ollama/models/blobs/sha256-102a747c137683e81d431dab05d8f2158df4ab6f162f8f9019425a43d51e0e9f","msg":"unable to load model","tid":"22744593221632","timestamp":1744332965} time=2025-04-11T00:58:24.611Z level=INFO source=server.go:500 msg="context expired before server started" time=2025-04-11T00:58:24.611Z level=ERROR source=sched.go:332 msg="error loading llama server" error="timed out waiting for llama runner to start: context canceled" [GIN] 2025/04/11 - 00:58:24 | 499 | 2m19s | 127.0.0.1 | POST "/api/chat" [GIN] 2025/04/11 - 00:58:45 | 200 | 14.414µs | 127.0.0.1 | HEAD "/" [GIN] 2025/04/11 - 00:58:45 | 404 | 57.778µs | 127.0.0.1 | POST "/api/show" time=2025-04-11T00:58:45.658Z level=ERROR source=images.go:962 msg="jwt token does not contain 3 parts" time=2025-04-11T00:58:45.882Z level=INFO source=download.go:136 msg="downloading aeda25e63ebd in 34 100 MB part(s)" time=2025-04-11T00:59:15.039Z level=INFO source=download.go:136 msg="downloading e0a42594d802 in 1 358 B part(s)" time=2025-04-11T00:59:16.321Z level=INFO source=download.go:136 msg="downloading dd084c7d92a3 in 1 8.4 KB part(s)" time=2025-04-11T00:59:17.441Z level=INFO source=download.go:136 msg="downloading 3116c5225075 in 1 77 B part(s)" time=2025-04-11T00:59:18.578Z level=INFO source=download.go:136 msg="downloading b6ae5839783f in 1 489 B part(s)" [GIN] 2025/04/11 - 00:59:22 | 200 | 36.553994129s | 127.0.0.1 | POST "/api/pull" [GIN] 2025/04/11 - 00:59:22 | 200 | 286.098µs | 127.0.0.1 | POST "/api/show" [GIN] 2025/04/11 - 00:59:22 | 200 | 216.223µs | 127.0.0.1 | POST "/api/show" time=2025-04-11T00:59:22.002Z level=INFO source=gpu.go:122 msg="Detecting GPUs" time=2025-04-11T00:59:22.004Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2025-04-11T00:59:23.133Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2025-04-11T00:59:23.133Z level=INFO source=server.go:327 msg="starting llama server" cmd="/tmp/ollama418901386/runners/cpu_avx2/ollama_llama_server --model /root/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 999 --parallel 1 --port 35683" time=2025-04-11T00:59:23.133Z level=INFO source=sched.go:326 msg="loaded runners" count=1 time=2025-04-11T00:59:23.133Z level=INFO source=server.go:495 msg="waiting for llama runner to start responding" {"function":"server_params_parse","level":"INFO","line":2604,"msg":"logging to file is disabled.","tid":"22525903984640","timestamp":1744333163} {"build":1,"commit":"baa5868","function":"main","level":"INFO","line":2821,"msg":"build info","tid":"22525903984640","timestamp":1744333163} {"function":"main","level":"INFO","line":2828,"msg":"system info","n_threads":8,"n_threads_batch":-1,"system_info":"AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | ","tid":"22525903984640","timestamp":1744333163,"total_threads":20} llama_model_loader: loaded meta data with 35 key-value pairs and 883 tensors from /root/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: gemma3.attention.head_count u32 = 8 llama_model_loader: - kv 1: gemma3.attention.head_count_kv u32 = 4 llama_model_loader: - kv 2: gemma3.attention.key_length u32 = 256 llama_model_loader: - kv 3: gemma3.attention.sliding_window u32 = 1024 llama_model_loader: - kv 4: gemma3.attention.value_length u32 = 256 llama_model_loader: - kv 5: gemma3.block_count u32 = 34 llama_model_loader: - kv 6: gemma3.context_length u32 = 131072 llama_model_loader: - kv 7: gemma3.embedding_length u32 = 2560 llama_model_loader: - kv 8: gemma3.feed_forward_length u32 = 10240 llama_model_loader: - kv 9: gemma3.mm.tokens_per_image u32 = 256 llama_model_loader: - kv 10: gemma3.vision.attention.head_count u32 = 16 llama_model_loader: - kv 11: gemma3.vision.attention.layer_norm_epsilon f32 = 0.000001 llama_model_loader: - kv 12: gemma3.vision.block_count u32 = 27 llama_model_loader: - kv 13: gemma3.vision.embedding_length u32 = 1152 llama_model_loader: - kv 14: gemma3.vision.feed_forward_length u32 = 4304 llama_model_loader: - kv 15: gemma3.vision.image_size u32 = 896 llama_model_loader: - kv 16: gemma3.vision.num_channels u32 = 3 llama_model_loader: - kv 17: gemma3.vision.patch_size u32 = 14 llama_model_loader: - kv 18: general.architecture str = gemma3 llama_model_loader: - kv 19: tokenizer.ggml.add_bos_token bool = true llama_model_loader: - kv 20: tokenizer.ggml.add_eos_token bool = false llama_model_loader: - kv 21: tokenizer.ggml.add_padding_token bool = false llama_model_loader: - kv 22: tokenizer.ggml.add_unknown_token bool = false llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 2 llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 1 llama_model_loader: - kv 25: tokenizer.ggml.merges arr[str,514906] = ["\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n", ... llama_model_loader: - kv 26: tokenizer.ggml.model str = llama llama_model_loader: - kv 27: tokenizer.ggml.padding_token_id u32 = 0 llama_model_loader: - kv 28: tokenizer.ggml.pre str = default llama_model_loader: - kv 29: tokenizer.ggml.scores arr[f32,262145] = [0.000000, 0.000000, 0.000000, 0.0000... llama_model_loader: - kv 30: tokenizer.ggml.token_type arr[i32,262145] = [3, 3, 3, 2, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 31: tokenizer.ggml.tokens arr[str,262145] = ["<pad>", "<eos>", "<bos>", "<unk>", ... llama_model_loader: - kv 32: tokenizer.ggml.unknown_token_id u32 = 3 llama_model_loader: - kv 33: general.quantization_version u32 = 2 llama_model_loader: - kv 34: general.file_type u32 = 15 llama_model_loader: - type f32: 479 tensors llama_model_loader: - type f16: 165 tensors llama_model_loader: - type q4_K: 205 tensors llama_model_loader: - type q6_K: 34 tensors llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma3' llama_load_model_from_file: failed to load model llama_init_from_gpt_params: error: failed to load model '/root/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25' {"function":"load_model","level":"ERR","line":410,"model":"/root/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25","msg":"unable to load model","tid":"22525903984640","timestamp":1744333163} this is what happened with ollama
July 1, 20251 yr I am looking to do the same thing. Any updates or guides on how to get this working?
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.