Andrew's picture

82 2

Andrew

sealad886

·

sealad886

AI & ML interests

None yet

Organizations

New activity in sealad886/OpenCodeReasoning_messages 4 months ago

Adding a `text` column

#11 opened 4 months ago by

Fixing an oopsie

#10 opened 4 months ago by

Adding a `text` column

#9 opened 4 months ago by

A smaller subset of validation data. First 10%.

#7 opened 4 months ago by

A smaller subset of training data. First 10%.

#6 opened 4 months ago by

A smaller subset of training data. First 10%.

#5 opened 4 months ago by

Upload dataset

#4 opened 4 months ago by

Upload dataset

#3 opened 4 months ago by

Upload dataset

#2 opened 4 months ago by

Upload dataset

#1 opened 4 months ago by

New activity in OpenPipe/Deductive-Reasoning-Qwen-14B 8 months ago

Rename files so format converting tools can find it.

#3 opened 8 months ago by

New activity in OpenPipe/Deductive-Reasoning-Qwen-32B 8 months ago

Rename files so format converting tools can find it.

#6 opened 8 months ago by

Not a 'thinking' model if anyone is wondering

#3 opened 9 months ago by

New activity in mlx-community/DeepSeek-R1-Distill-Qwen-32B-MLX 8 months ago

How to get these new quantinized version of model in LMStudio?

#24 opened 9 months ago by

New activity in mlx-community/DeepSeek-R1-Distill-Qwen-32B-MLX 9 months ago

Multiple quants for MLX framework

#25 opened 9 months ago by

New activity in mlx-community/DeepSeek-R1-Distill-Qwen-14B-MLX 9 months ago

Multiple quants for MLX framework

#1 opened 9 months ago by

New activity in mlx-community/DeepSeek-R1-Distill-Qwen-32B-MLX 9 months ago

Multiple quants for MLX framework

#23 opened 9 months ago by

Multiple quants for MLX framework

#17 opened 9 months ago by

Parent PR: Add Multi-Quantization Support for DeepSeek-R1-Distill-Qwen-32B via MLX_LM This PR introduces a new conversion pipeline that generates multiple quantized variants of DeepSeek-R1-Distill-Qwen-32B using the MLX_LM tool. Unlike previous methods based on llama.cpp, this implementation leverages MLX_LM’s unique `quant_predicate` configuration to produce high‑quality mixed‑bit quantizations optimized specifically for MLX runs. Key Changes and Features: - MLX_LM-Based Conversion: All model conversions are performed using MLX_LM, which uses parameters like `q_bits`, `q_group_size`, and the distinctive `quant_predicate` (e.g., `"mixed_3_6"`, `"mixed_2_6"`) to create finely tuned quantized models. This provides a superior balance between quality and performance tailored for MLX inference. - Asynchronous Workflow: The new pipeline supports asynchronous conversion and upload tasks. Each quantized variant is generated concurrently and then uploaded to the designated Hugging Face repository, streamlining the overall process. - Updated Documentation: The repository’s README has been fully updated to reflect the MLX_LM conversion process, providing clear instructions on prompt formatting, downloading individual variants, and running the models with MLX. The documentation emphasizes that these quantizations are for MLX runs only and are not intended for general GGUF deployments. - Enhanced User Flexibility: With multiple quantization options (including bf16, Q8_0, Q6_K, Q5_K_M, Q4_K_M, IQ4_NL, etc.), users can select the variant that best meets their hardware and performance requirements. Detailed usage and download instructions facilitate easy deployment. Benefits: - Optimized for MLX Runs: The generated quantized models are designed specifically for MLX inference, ensuring optimal performance and compatibility with MLX’s specialized runtime. - Scalability and Future-Proofing: This modular pipeline allows for easy integration of additional quantization recipes and future enhancements while keeping the conversion process aligned with MLX_LM’s capabilities. - Comprehensive Documentation: The updated README and model card provide thorough guidance on model usage, including prompt format, download instructions, and hardware-specific recommendations. This PR represents a significant advancement in making DeepSeek-R1-Distill-Qwen-32B more accessible and versatile for MLX users. It establishes a parent PR that will be referenced by every subsequent quantized model upload, ensuring consistency and traceability across all releases. Feedback and suggestions are welcome.

#22 opened 9 months ago by

Parent PR: Add Multi-Quantization Support for DeepSeek-R1-Distill-Qwen-32B via MLX_LM This PR introduces a new conversion pipeline that generates multiple quantized variants of DeepSeek-R1-Distill-Qwen-32B using the MLX_LM tool. Unlike previous methods based on llama.cpp, this implementation leverages MLX_LM’s unique `quant_predicate` configuration to produce high‑quality mixed‑bit quantizations optimized specifically for MLX runs. Key Changes and Features: - MLX_LM-Based Conversion: All model conversions are performed using MLX_LM, which uses parameters like `q_bits`, `q_group_size`, and the distinctive `quant_predicate` (e.g., `"mixed_3_6"`, `"mixed_2_6"`) to create finely tuned quantized models. This provides a superior balance between quality and performance tailored for MLX inference. - Asynchronous Workflow: The new pipeline supports asynchronous conversion and upload tasks. Each quantized variant is generated concurrently and then uploaded to the designated Hugging Face repository, streamlining the overall process. - Updated Documentation: The repository’s README has been fully updated to reflect the MLX_LM conversion process, providing clear instructions on prompt formatting, downloading individual variants, and running the models with MLX. The documentation emphasizes that these quantizations are for MLX runs only and are not intended for general GGUF deployments. - Enhanced User Flexibility: With multiple quantization options (including bf16, Q8_0, Q6_K, Q5_K_M, Q4_K_M, IQ4_NL, etc.), users can select the variant that best meets their hardware and performance requirements. Detailed usage and download instructions facilitate easy deployment. Benefits: - Optimized for MLX Runs: The generated quantized models are designed specifically for MLX inference, ensuring optimal performance and compatibility with MLX’s specialized runtime. - Scalability and Future-Proofing: This modular pipeline allows for easy integration of additional quantization recipes and future enhancements while keeping the conversion process aligned with MLX_LM’s capabilities. - Comprehensive Documentation: The updated README and model card provide thorough guidance on model usage, including prompt format, download instructions, and hardware-specific recommendations. This PR represents a significant advancement in making DeepSeek-R1-Distill-Qwen-32B more accessible and versatile for MLX users. It establishes a parent PR that will be referenced by every subsequent quantized model upload, ensuring consistency and traceability across all releases. Feedback and suggestions are welcome.

#21 opened 9 months ago by