Dhir-2B-4bit

Dhīr-2B-4bit is the 4-bit quantised release of Dhīr 2B , ZenteiQ's 2.03-billion-parameter foundation model, pretrained from scratch in India. At just 1.3 GB , it is designed for private, offline and memory-efficient edge AI deployment on mobiles, embedded chips and laptops without requiring a GPU. The model supports English and ten Indian languages. Architecture and Quantisation Dhīr 2B uses a dense decoder-only Transformer architecture. The model is distributed in GGUF Q4_K_M format for efficient local inference using llama.cpp. The weights were quantised directly from the released BF16 model, without additional training or fine-tuning. Model specifications: Base model: Dhīr 2B Parameters: 2.03 billion Quantisation: Q4_K_M (4-bit) Model format: GGUF File: Dhir-2B-Q4_K_M.gguf Model size: 1.3 GB Context length: 16,384 tokens Languages: English and ten Indian languages License: Apache 2.0 Quantisation Fidelity ZenteiQ evaluated Dhīr-2B-4bit against the original BF16 model using WikiText-2 perplexity, KL divergence of next-token prediction distributions, and standard benchmarks. The evaluation covered approximately 1.5 million tokens across 11 languages, code and mathematics . Key evaluation results: Within 3.4% relative perplexity difference compared with the original BF16 model on WikiText-2. 91% next-token prediction agreement with the BF16 model. Evaluation across multilingual text, code and mathematics using KL divergence and standard benchmarks. No additional training following quantisation. These results demonstrate substantial preservation of the original model's predictive behaviour while reducing its storage requirements for resource-constrained devices. The 91% agreement measures matching next-token predictions, not identical complete responses or overall task accuracy. For the full evaluation, refer to ZenteiQ's technical paper and research updates: https://zenteiq.ai/research/blog/ Choose a Dhīr Model ModelSizeRecommended Deployment Dhīr-2B-4bit 1.3 GB Edge devices, mobiles and embedded chips with limited memory Dhīr-2B-8bit 2.2 GB Laptops and mobiles; closest to the original 2B model Dhīr-5B-A1.7B-4bit 3.4 GB Laptops and desktops; stronger and faster than Dhīr 2B on a CPU Dhīr-5B-A1.7B-8bit 5.4 GB Laptops, desktops and small GPUs; closest to the original 5B model Deployment with ZenteiQ's llama.cpp ZenteiQ's llama.cpp repository is a Dhīr-compatible fork of the open-source llama.cpp inference engine. The repository provides the Dhīr-specific architecture and tokenizer support required to load and run Dhīr models in GGUF format. It enables: Offline local inference on compatible CPUs and GPUs. Command-line execution using llama-cli . OpenAI-compatible API serving using llama-server . Application integration through frameworks such as LangChain. Edge AI deployment on supported laptops, desktops, mobile devices and embedded systems. Until Dhīr support is merged into upstream llama.cpp, users should build the dhir-support branch of ZenteiQ's repository. Repository: https://github.com/zenteiq-research/llama.cpp The README provides the installation, build and inference commands. License Apache License 2.0 , the same as the original Dhīr 2B model published on AIKosh. Developed by ZenteiQ AiTech Innovations Pvt. Ltd. https://zenteiq.ai