Indian Flag
Government Of India
A-
A
A+
/Ola Krutrim ORGANISATION
UpvoteUpvote
0

Dhwani - Multilingual Speech LLM

Dhwani is India's first end-to-end trained speech Large Language Model (LLM), capable of directly understanding speech without a separate ASR (Automatic Speech Recognition) model, avoiding cascading ASR errors. It supports speech-to-text translation across multiple Indic languages and English.
  • Upvotes0upvotes
  • Downloads164downloads
  • Views4,324views
  • File size0
  • Shares0Shares
  • Last updatedupdated 06 Nov 2026
UpvoteUpvote
0
LICENSE Krutrim Community License Agreement Version 1.0
SECTOR Sector Agnostic

About Model

Dhwani is an end-to-end trained speech LLM designed for Indic speech-to-text and multilingual speech translation. Developed by Krutrim AI Labs, Dhwani is powered by Krutrim-1 LLM, enabling direct speech understanding without the need for ASR models. It features a dual encoder structure, utilizing Whisper's speech encoder for processing speech inputs and BEATs audio encoder for non-speech audio signals. The model employs a Window-Level Query Transformer (Q-Former) as a bridge between audio and text processing. Using Low-Rank Adaptation (LoRA) fine-tuning, Dhwani aligns audio-derived inputs with textual output, ensuring accurate speech recognition and translation. It supports English, Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Tamil, and Telugu and excels in use cases like multilingual communication, media translation, education, healthcare, customer support, business, and legal applications. Evaluation results show high BLEU scores for English-to-Indic and Indic-to-English translations, demonstrating its efficiency in real-world scenarios.

Metadata Metadata

Krutrim Community License Agreement Version 1.0

Sector Agnostic

Ola Krutrim

N.A.

Ola Krutrim

Hareesh Kumar G

N.A.

N.A.

N.A.

N.A.

N.A.

N.A.

N.A.

Open

28/02/25 07:00:47

N.A.

0

Tags

  • Translation
  • Indic Languages
  • Multilingual AI
  • Deep Learning
  • Speech-to-Text
  • conversational-AI
  • Krutrim AI
  • BharatBench
  • speech LLM
  • ASR-free speech recognition

Related Models Related Models

Dhir-5B-4bit
Dhīr 5B MoE Q4_K_M GGUF (3.4 GB) for offline llama.cpp in English + 10 Indian languages. 5.09B total/1.79B active parameters; 91% token agreement (1.5M tokens), ≤1.8% perplexity vs BF16.
Large Language Models
ZENTEIQ
  • See Upvoters3
  • Downloads14
  • Views186
  • File Size3.13 GB

microsoft/Phi-3-vision-128k-instruct
Phi-3-vision-128k-instruct is a state-of-the-art multimodal model by Microsoft, designed to process both text and visual inputs with a context length of up to 128,000 tokens.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads12
  • Views281
  • File Size0

Phi-3-Medium-4K-Instruct - Scalable AI Model for Reasoning and Code Generation
A 14B parameter AI model optimized for 4K and 128K token context lengths, designed for advanced reasoning, math, and coding applications, supporting both research and commercial use.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads0
  • Views262
  • File Size0

Phi-3-Medium-128K-Instruct ONNX-CUDA - Optimized AI Model for NVIDIA GPU Inference
An ONNX-optimized version of Phi-3-Medium-128K-Instruct, designed for high-speed, efficient inference on NVIDIA GPUs, supporting FP16 and INT4 quantization for enhanced performance.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads2
  • Views164
  • File Size0

Phi-3-Small-8K-Instruct ONNX-CUDA - Optimized AI Model for NVIDIA GPU Inference
An ONNX-optimized version of Phi-3-Small-8K-Instruct, designed for efficient inference on NVIDIA GPUs, supporting FP16 and INT4 quantization for fast, scalable AI applications.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads1
  • Views173
  • File Size0

Phi-3-Mini-4K-Instruct GGUF - Optimized AI Model for GGUF Inference
A GGUF-optimized version of Phi-3-Mini-4K-Instruct, designed for efficient low-memory AI inference, supporting quantized and full-precision formats for balanced performance and quality.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters1
  • Downloads29
  • Views1,559
  • File Size0

Phi-3-Medium-128K-Instruct - Advanced Long-Context AI for Code and Logical Reasoning
A 14B parameter AI model optimized for 128K token context length, designed for complex reasoning, mathematical problem-solving, and efficient AI applications in commercial and research environments.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads3
  • Views132
  • File Size0

Phi-3-Medium-128K-Instruct ONNX-CPU - Optimized AI Model for CPU Inference
An ONNX-optimized version of Phi-3-Medium-128K-Instruct, designed for efficient long-context inference on CPUs, supporting structured reasoning, text generation, and code processing.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads2
  • Views208
  • File Size0

Phi-3-Mini-128K-Instruct ONNX - Optimized AI Model for Long-Context ONNX Inference
An ONNX-optimized version of Phi-3-Mini-128K-Instruct, quantized for fast, scalable inference on CPUs, GPUs, and mobile devices, supporting long-context AI applications.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads1
  • Views121
  • File Size0

Phi-3-Medium-4K-Instruct ONNX-DirectML - Optimized AI Model for Windows GPU Inference
An ONNX-optimized version of Phi-3-Medium-4K-Instruct, designed for efficient AI inference on Windows machines using DirectML, supporting INT4 quantization for high-performance execution on AMD, Intel, and NVIDIA GPUs.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads0
  • Views211
  • File Size0


Related Models by Organization Related Models by Organization

Chitrarth - Multilingual Vision-Language Model
Chitrarth is a multilingual vision-language model (VLM) integrating a Large Language Model (LLM) with a vision module. It is trained on multilingual image-text data and supports 10 Indic languages along with English.
OLA KRUTRIM
  • See Upvoters1
  • Downloads145
  • Views2,373
  • File Size0

Krutrim Translate - Indic Language Translation Model
Krutrim Translate is a multilingual machine translation model optimized for Indic languages, supporting English-to-Indic and Indic-to-English translations. It extends IndicTrans2 with a longer context length (4096 tokens) and leverages the Bharat Parallel Corpus Collection (BPCC) for training.
OLA KRUTRIM
  • See Upvoters0
  • Downloads61
  • Views1,991
  • File Size0

Krutrim-1 Instruct Large Language Model
Krutrim-1 is a 7.3B parameter multilingual foundation model trained on a 2 trillion token dataset, designed for Indian linguistic and demographic needs. It supports 11 Indic languages and matches or exceeds comparable state-of-the-art models in multilingual tasks.
OLA KRUTRIM
  • See Upvoters0
  • Downloads28
  • Views1,368
  • File Size0

Vyakyarth - Multilingual Sentence Embedding Model
Vyakyarth is a sentence-transformers-based model fine-tuned for Indic languages, capable of mapping text to a 768-dimensional dense vector space for semantic search, similarity, classification, and clustering tasks.
OLA KRUTRIM
  • See Upvoters1
  • Downloads53
  • Views1,235
  • File Size0

Krutrim-2 Instruct
Krutrim-2 is a 12B parameter multilingual large language model built on the Mistral-NeMo 12B architecture, optimized for Indic languages and Indian cultural context. It supports long-form conversations, reasoning, coding, and translation tasks.
OLA KRUTRIM
  • See Upvoters0
  • Downloads33
  • Views1,101
  • File Size0


Related Models by Sector Related Models by Sector

Dhir-5B-4bit
Dhīr 5B MoE Q4_K_M GGUF (3.4 GB) for offline llama.cpp in English + 10 Indian languages. 5.09B total/1.79B active parameters; 91% token agreement (1.5M tokens), ≤1.8% perplexity vs BF16.
Large Language Models
ZENTEIQ
  • See Upvoters3
  • Downloads14
  • Views186
  • File Size3.13 GB

microsoft/Phi-3-vision-128k-instruct
Phi-3-vision-128k-instruct is a state-of-the-art multimodal model by Microsoft, designed to process both text and visual inputs with a context length of up to 128,000 tokens.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads12
  • Views281
  • File Size0

Phi-3-Medium-4K-Instruct - Scalable AI Model for Reasoning and Code Generation
A 14B parameter AI model optimized for 4K and 128K token context lengths, designed for advanced reasoning, math, and coding applications, supporting both research and commercial use.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads0
  • Views262
  • File Size0

Phi-3-Medium-128K-Instruct ONNX-CUDA - Optimized AI Model for NVIDIA GPU Inference
An ONNX-optimized version of Phi-3-Medium-128K-Instruct, designed for high-speed, efficient inference on NVIDIA GPUs, supporting FP16 and INT4 quantization for enhanced performance.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads2
  • Views164
  • File Size0

Phi-3-Small-8K-Instruct ONNX-CUDA - Optimized AI Model for NVIDIA GPU Inference
An ONNX-optimized version of Phi-3-Small-8K-Instruct, designed for efficient inference on NVIDIA GPUs, supporting FP16 and INT4 quantization for fast, scalable AI applications.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads1
  • Views173
  • File Size0

Phi-3-Mini-4K-Instruct GGUF - Optimized AI Model for GGUF Inference
A GGUF-optimized version of Phi-3-Mini-4K-Instruct, designed for efficient low-memory AI inference, supporting quantized and full-precision formats for balanced performance and quality.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters1
  • Downloads29
  • Views1,559
  • File Size0

Phi-3-Medium-128K-Instruct - Advanced Long-Context AI for Code and Logical Reasoning
A 14B parameter AI model optimized for 128K token context length, designed for complex reasoning, mathematical problem-solving, and efficient AI applications in commercial and research environments.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads3
  • Views132
  • File Size0

Phi-3-Medium-128K-Instruct ONNX-CPU - Optimized AI Model for CPU Inference
An ONNX-optimized version of Phi-3-Medium-128K-Instruct, designed for efficient long-context inference on CPUs, supporting structured reasoning, text generation, and code processing.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads2
  • Views208
  • File Size0

Phi-3-Mini-128K-Instruct ONNX - Optimized AI Model for Long-Context ONNX Inference
An ONNX-optimized version of Phi-3-Mini-128K-Instruct, quantized for fast, scalable inference on CPUs, GPUs, and mobile devices, supporting long-context AI applications.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads1
  • Views121
  • File Size0

Phi-3-Medium-4K-Instruct ONNX-DirectML - Optimized AI Model for Windows GPU Inference
An ONNX-optimized version of Phi-3-Medium-4K-Instruct, designed for efficient AI inference on Windows machines using DirectML, supporting INT4 quantization for high-performance execution on AMD, Intel, and NVIDIA GPUs.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads0
  • Views211
  • File Size0


Related Models by Task Type Related Models by Task Type

AI4Bharat- IndicConformerAutomatic Speech Recognition (ASR) Model for Dogri
The Dogri ASR (Automatic Speech Recognition) model transcribes 16kHz mono-channel audio into text. It is based on a Conformer-Large architecture with 120M parameters and a hybrid CTC-RNNT decoder, ensuring accurate and efficient speech-to-text conversion
AI4BHARAT
  • See Upvoters0
  • Downloads14
  • Views622
  • File Size0

AI4Bharat- Maithili - IndicConformer Automatic Speech Recognition (ASR) Model
This model takes in mono-channel audio files at a 16,000 Hz sampling rate (WAV format) and outputs the transcribed text of the speech contained in the audio.
AI4BHARAT
  • See Upvoters0
  • Downloads29
  • Views906
  • File Size0

AI4Bharat- Konkani - IndicConformer Automatic Speech Recognition (ASR) Model
Automatic Speech Recognition (ASR) model for Konkani speech recognition, processing 16,000 KHz mono WAV audio and transcribing spoken content into text
AI4BHARAT
  • See Upvoters0
  • Downloads38
  • Views947
  • File Size0

AI4Bharat- Kashmiri - IndicConformer Automatic Speech Recognition (ASR) Model
This Automatic Speech Recognition (ASR) model transcribes Kashmiri speech from 16,000 KHz mono WAV audio files into text
AI4BHARAT
  • See Upvoters0
  • Downloads29
  • Views1,045
  • File Size0

AI4Bharat- Kannada - IndicConformer Automatic Speech Recognition (ASR) Model
This Kannada Automatic Speech Recognition (ASR) model transcribes 16kHz mono-channel audio into text. It utilizes a Conformer-Large architecture with 120M parameters and a hybrid CTC-RNNT decoder for high-accuracy speech recognition.
AI4BHARAT
  • See Upvoters0
  • Downloads27
  • Views1,053
  • File Size0

AI4Bharat- Hindi - IndicConformer Automatic Speech Recognition (ASR) Model
This Hindi Automatic Speech Recognition (ASR) model converts 16kHz mono-channel audio into text. It is based on a Conformer-Large architecture with 120M parameters and a hybrid CTC-RNNT decoder, ensuring high-accuracy speech transcription.
AI4BHARAT
  • See Upvoters0
  • Downloads105
  • Views2,782
  • File Size0

AI4Bharat- Gujarati - IndicConformer Automatic Speech Recognition (ASR) Model
This Gujarati Automatic Speech Recognition (ASR) model converts 16kHz mono-channel audio into text. Built on a Conformer-Large architecture with 120M parameters and a hybrid CTC-RNNT decoder, it delivers high-accuracy speech-to-text transcription.
AI4BHARAT
  • See Upvoters0
  • Downloads42
  • Views631
  • File Size0

AI4Bharat - Telugu IndicWav2Vec Speech Model
IndicWav2Vec is a multilingual speech model pre-trained on 40 Indian languages and fine-tuned for Automatic Speech Recognition (ASR) in 9 languages, achieving state-of-the-art results on multiple benchmarks.
AI4BHARAT
  • See Upvoters0
  • Downloads51
  • Views1,385
  • File Size0

AI4Bharat- Bodo - IndicConformer Automatic Speech Recognition (ASR) Model
The Bodo ASR (Automatic Speech Recognition) model converts 16kHz mono-channel audio into text. Built on a Conformer-Large architecture with 120M parameters and a hybrid CTC-RNNT decoder, it ensures high-accuracy speech-to-text transcription.
AI4BHARAT
  • See Upvoters0
  • Downloads36
  • Views1,012
  • File Size0

AI4Bharat - Tamil IndicWav2Vec Speech Model
IndicWav2Vec is a multilingual speech model pre-trained on 40 Indian languages and fine-tuned for Automatic Speech Recognition (ASR) in 9 languages, achieving state-of-the-art results on multiple benchmarks.
AI4BHARAT
  • See Upvoters0
  • Downloads53
  • Views1,124
  • File Size0