DeepSeek-R1-Distill-Qwen-1.5B Q4_K_M
DeepSeek-R1-Distill-Qwen-1.5B text generation optimized as a Q4_K_M GGUF model for the llama.cpp runtime on Arm-based Premium Smartphone systems.
Loading example…
LLM chat application with Arm CPU inference provided by llama.cpp. This project demonstrates running large language models on CPU with inference provided by the llama.cpp server. The upstream Linux Arm64 image includes architecture-specific CPU backend variants for Armv8.0 baseline, Armv8.2 dot product/FP16/SVE, Armv8.6 int8 matrix multiply/SVE2, and Armv9.2 SME-capable CPUs. The stack includes: - llama.cpp - Qwen3-30B-A3B-Instruct-2507 with target-aware precision selection - Built-in web chat interface - No GPU required - pure CPU inference Topo supplies target memory; whichllm selects a precision during the build. Ideal for testing LLM workloads on Arm hardware without GPU dependencies while avoiding a source build during Topo Project deployment.
View on GitHubModels supported by this example, which include the pre- and post-processing needed to prepare inputs and turn model outputs into usable results.