CPU is forbidden ?

#24
by edmond - opened

No matter what I try to use cpu, I have :
Cell In[17], line 6
result = moondream.query(image=image, question="What's in this image?")

File ~/.cache/huggingface/modules/transformers_modules/moondream/moondream3-preview/e86382f00368618bfbbef8026cb606e9c0e3cd0e/moondream.py:635 in query
image = self.encode_image(image, settings)

File ~/.cache/huggingface/modules/transformers_modules/moondream/moondream3-preview/e86382f00368618bfbbef8026cb606e9c0e3cd0e/moondream.py:315 in encode_image
img_emb = self._run_vision_encoder(image)

File ~/.cache/huggingface/modules/transformers_modules/moondream/moondream3-preview/e86382f00368618bfbbef8026cb606e9c0e3cd0e/moondream.py:294 in _run_vision_encoder
return self._vis_proj(global_features, reconstructed)

File ~/.cache/huggingface/modules/transformers_modules/moondream/moondream3-preview/e86382f00368618bfbbef8026cb606e9c0e3cd0e/moondream.py:192 in _vis_proj
return vision_projection(g, r, self.vision, self.config.vision)

File ~/.cache/huggingface/modules/transformers_modules/moondream/moondream3-preview/e86382f00368618bfbbef8026cb606e9c0e3cd0e/vision.py:88 in vision_projection
final_features = torch.cat([global_features, reconstructed], dim=-1)

RuntimeError: torch.cat(): all input tensors must be on the same device. Received cpu and mps:0

moondream org

Currently MD3 only runs on CUDA as it uses Flex Attn, but we have a MLX implementation in the pipeline.

Sign up or log in to comment