mistralai

/

Devstral-Small-2-24B-Instruct-2512

like 321

Follow

Mistral AI_ 13.9k

I'm getting the following error when trying to call the model with tool calling enabled:

Here is my start command for vLLM:

#!/bin/bash

VLLM_NAME="vllm"
VLLM_BASE_IMAGE="docker.io/mistralllm/vllm_devstral"
VERSION="latest"

podman run -d \
                -p 3000:3000 \
                --device nvidia.com/gpu=all --security-opt=label=disable \
                -v /data/mistralai-Devstral-Small-2-24B-Instruct-2512:/inference_model \
                --name "${VLLM_NAME}" \
                --restart always \
                "${VLLM_BASE_IMAGE}:${VERSION}" \
                --model /inference_model \
                --host 0.0.0.0 \
                --port 3000 \
                --served_model_name /inference_model \
                --tool-call-parser mistral \
                --enable-auto-tool-choice \
                --max-model-len 12000 \
                --limit-mm-per-prompt '{"image":2}' \
                --max_num_seqs 24 \
                --dtype half \
                --enforce-eager