unsloth

/

Qwen3-VL-8B-Instruct-GGUF

like 42

Follow

Unsloth AI 19.4k

As the title says, I tried many methods but couldn't get it to work properly on ollama

Hi,
I opened a ticket about that.
https://github.com/ollama/ollama/issues/12896

it's because the ollama engine seems not ready yet.

C:\Users\appuser>ollama show qwen3-vl:8b-instruct-q8_0
Model
architecture qwen3vl
parameters 8.8B
context length 262144
embedding length 4096
quantization Q8_0

Capabilities
completion
vision
tools

Parameters
temperature 1
top_k 20
top_p 0.95

License
Apache License
Version 2.0, January 2004
...

C:\Users\appuser>ollama show hf.co/unsloth/Qwen3-VL-8B-Instruct-GGUF:IQ4_NL
Model
architecture qwen3vl
parameters 8.19B
context length 262144
embedding length 4096
quantization unknown

Capabilities
completion
vision

Projector
architecture clip
parameters 576.39M
embedding length 1152
dimensions 4096

Parameters
stop "<|im_start|>"
stop "<|im_end|>"
stop "<|im_start|>user"

C:\Users\appuser>ollama show hf.co/unsloth/Qwen3-VL-8B-Instruct-GGUF:Q8_0
Model
architecture qwen3vl
parameters 8.19B
context length 262144
embedding length 4096
quantization unknown

Capabilities
completion
vision

Projector
architecture clip
parameters 576.39M
embedding length 1152
dimensions 4096

Parameters
stop "<|im_start|>"
stop "<|im_end|>"
stop "<|im_start|>user"

C:\Users\appuser>ollama show hf.co/unsloth/Qwen3-VL-30B-A3B-Instruct-1M-GGUF:IQ4_NL
Model
architecture qwen3vlmoe
parameters 30.5B
context length 1000000
embedding length 2048
quantization unknown

Capabilities
completion
vision

Projector
architecture clip
parameters 538.63M
embedding length 1152
dimensions 2048

Parameters
stop "<|im_start|>"
stop "<|im_end|>"
stop "<|im_start|>user"

non VL :

C:\Users\appuser>ollama show hf.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF:IQ4_NL
Model
architecture qwen3moe
parameters 30.5B
context length 262144
embedding length 2048
quantization unknown

Capabilities
completion
tools

Parameters
stop "<|im_start|>"
stop "<|im_end|>"
temperature 0.7
min_p 0
repeat_penalty 1.05
top_k 20
top_p 0.8

C:\Users\appuser>

did you downloaded mmproj-BF16.gguf or mmproj-F16.gguf you must download mmproj file along with model and add them into modelfile if you are going to use ollama
edit after doing that i still get
Error: 500 Internal Server Error: model runner has unexpectedly stopped, this may be due to resource limitations or an internal error, check ollama server logs for details
even with lowering PARAMETER num_ctx 4096
PARAMETER num_gpu 11