Indian Flag
Government Of India
A-
A
A+
UpvoteUpvote
0

Phi-4-Multimodal-Instruct - Multimodal Foundation Model

A lightweight multimodal AI model that processes text, image, and audio inputs, optimized for multilingual reasoning, speech recognition, vision-language tasks, and generative AI applications.
  • Upvotes0upvotes
  • Downloads9downloads
  • Views492views
  • File size0
  • Shares0Shares
  • Last updatedupdated 12 Nov 2026
UpvoteUpvote
0
LICENSE MIT
SECTOR Sector Agnostic

About Model

Phi-4-Multimodal-Instruct is an advanced multimodal foundation model developed by Microsoft, designed to integrate language, vision, and speech for research and commercial applications. It builds upon the Phi-3.5 and Phi-4 models, supporting 128K token context length and incorporating supervised fine-tuning, direct preference optimization, and reinforcement learning from human feedback (RLHF) to enhance performance and safety. Key Features: Supports multiple modalities: Text: 24 languages, including Arabic, Chinese, English, French, Spanish, and more. Vision: Optimized for English image understanding. Audio: Supports English, Chinese, German, French, Italian, Japanese, Spanish, and Portuguese speech processing. Enhanced capabilities: Speech recognition and speech translation (outperforms WhisperV3 and SeamlessM4T). Strong reasoning in math, logic, and general knowledge. Vision-language understanding (chart/table comprehension, optical character recognition). Multi-image comparison and summarization. Speech summarization and QA. Function and tool calling for AI agents. State-of-the-art performance: Ranked #1 on the HuggingFace OpenASR leaderboard for speech recognition (March 2025). Vision processing benchmarks surpass models like Gemini-1.5-Pro and InternOmni-7B. Optimized for real-world applications: Works in memory-constrained environments and low-latency scenarios. Trained on 5 trillion text tokens, 2.3 million speech hours, and 1.1 trillion image-text tokens. Intended Uses: Phi-4-Multimodal-Instruct is designed for broad multilingual and multimodal research and commercial applications, including: 1. General AI assistants for reasoning and knowledge retrieval. 2. Speech AI models for transcription, translation, and summarization. 3. Computer vision AI for image-text comprehension and optical character recognition (OCR). 4. Medical AI research for language-vision understanding. 5. Education and coding AI for knowledge-based tasks.

Metadata Metadata

MIT

Sector Agnostic

Microsoft

N.A.

Vikram Malhotra

N.A.

N.A.

N.A.

N.A.

N.A.

N.A.

N.A.

Open

12/03/25 06:35:15

N.A.

0

Tags

  • Transformers
  • Multilingual
  • Microsoft
  • Audio Processing
  • NLP
  • Speech Recognition
  • Text Generation
  • Multimodal
  • Visual Question Answering

Related Models Related Models

spk cond tts pflow Hindi
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Hindi.
BHARATGEN
  • See Upvoters0
  • Downloads3
  • Views126
  • File Size3.46 GB

spk cond tts pflow Marathi
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Marathi.
BHARATGEN
  • See Upvoters0
  • Downloads0
  • Views499
  • File Size3.49 GB

spk cond tts pflow Telugu
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Telugu.
BHARATGEN
  • See Upvoters0
  • Downloads0
  • Views85
  • File Size3.38 GB

spk cond tts pflow Bengali
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Bengali.
BHARATGEN
  • See Upvoters0
  • Downloads1
  • Views83
  • File Size3.49 GB

spk cond tts pflow Tamil
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Tamil.
BHARATGEN
  • See Upvoters0
  • Downloads4
  • Views139
  • File Size3.47 GB

Tamil vb
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Tamil.
BHARATGEN
  • See Upvoters0
  • Downloads2
  • Views549
  • File Size3.17 GB

Marathi vb
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Marathi.
BHARATGEN
  • See Upvoters0
  • Downloads2
  • Views508
  • File Size3.18 GB

Hindi vb
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Hindi.
BHARATGEN
  • See Upvoters0
  • Downloads4
  • Views1,079
  • File Size3.18 GB

Telugu vb
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Telugu.
BHARATGEN
  • See Upvoters0
  • Downloads2
  • Views489
  • File Size3.17 GB

Bengali vb
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Bengali.
BHARATGEN
  • See Upvoters0
  • Downloads1
  • Views318
  • File Size3.17 GB


Related Models by Organization Related Models by Organization

microsoft/Phi-3-vision-128k-instruct
Phi-3-vision-128k-instruct is a state-of-the-art multimodal model by Microsoft, designed to process both text and visual inputs with a context length of up to 128,000 tokens.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads12
  • Views287
  • File Size0

Phi-3-Medium-4K-Instruct - Scalable AI Model for Reasoning and Code Generation
A 14B parameter AI model optimized for 4K and 128K token context lengths, designed for advanced reasoning, math, and coding applications, supporting both research and commercial use.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads0
  • Views271
  • File Size0

BiomedBERT - Domain-Specific Biomedical Language Model
A biomedical NLP model pre-trained from scratch on abstracts and full-text articles from PubMed and PubMed Central, achieving state-of-the-art performance on biomedical language understanding tasks.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads102
  • Views2,001
  • File Size0

Phi-3-Medium-128K-Instruct ONNX-CUDA - Optimized AI Model for NVIDIA GPU Inference
An ONNX-optimized version of Phi-3-Medium-128K-Instruct, designed for high-speed, efficient inference on NVIDIA GPUs, supporting FP16 and INT4 quantization for enhanced performance.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads2
  • Views168
  • File Size0

Phi-3-Small-8K-Instruct ONNX-CUDA - Optimized AI Model for NVIDIA GPU Inference
An ONNX-optimized version of Phi-3-Small-8K-Instruct, designed for efficient inference on NVIDIA GPUs, supporting FP16 and INT4 quantization for fast, scalable AI applications.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads1
  • Views174
  • File Size0

Phi-3-Mini-4K-Instruct GGUF - Optimized AI Model for GGUF Inference
A GGUF-optimized version of Phi-3-Mini-4K-Instruct, designed for efficient low-memory AI inference, supporting quantized and full-precision formats for balanced performance and quality.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters1
  • Downloads30
  • Views1,589
  • File Size0

Phi-3-Medium-128K-Instruct - Advanced Long-Context AI for Code and Logical Reasoning
A 14B parameter AI model optimized for 128K token context length, designed for complex reasoning, mathematical problem-solving, and efficient AI applications in commercial and research environments.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads3
  • Views133
  • File Size0

Phi-3-Medium-128K-Instruct ONNX-CPU - Optimized AI Model for CPU Inference
An ONNX-optimized version of Phi-3-Medium-128K-Instruct, designed for efficient long-context inference on CPUs, supporting structured reasoning, text generation, and code processing.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads2
  • Views216
  • File Size0

Phi-3-Mini-128K-Instruct ONNX - Optimized AI Model for Long-Context ONNX Inference
An ONNX-optimized version of Phi-3-Mini-128K-Instruct, quantized for fast, scalable inference on CPUs, GPUs, and mobile devices, supporting long-context AI applications.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads1
  • Views121
  • File Size0

Phi-3-Medium-4K-Instruct ONNX-DirectML - Optimized AI Model for Windows GPU Inference
An ONNX-optimized version of Phi-3-Medium-4K-Instruct, designed for efficient AI inference on Windows machines using DirectML, supporting INT4 quantization for high-performance execution on AMD, Intel, and NVIDIA GPUs.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads0
  • Views212
  • File Size0


Related Models by Sector Related Models by Sector

spk cond tts pflow Hindi
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Hindi.
BHARATGEN
  • See Upvoters0
  • Downloads3
  • Views126
  • File Size3.46 GB

spk cond tts pflow Marathi
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Marathi.
BHARATGEN
  • See Upvoters0
  • Downloads0
  • Views499
  • File Size3.49 GB

spk cond tts pflow Telugu
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Telugu.
BHARATGEN
  • See Upvoters0
  • Downloads0
  • Views85
  • File Size3.38 GB

spk cond tts pflow Bengali
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Bengali.
BHARATGEN
  • See Upvoters0
  • Downloads1
  • Views83
  • File Size3.49 GB

spk cond tts pflow Tamil
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Tamil.
BHARATGEN
  • See Upvoters0
  • Downloads4
  • Views139
  • File Size3.47 GB

Tamil vb
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Tamil.
BHARATGEN
  • See Upvoters0
  • Downloads2
  • Views549
  • File Size3.17 GB

Marathi vb
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Marathi.
BHARATGEN
  • See Upvoters0
  • Downloads2
  • Views508
  • File Size3.18 GB

Hindi vb
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Hindi.
BHARATGEN
  • See Upvoters0
  • Downloads4
  • Views1,079
  • File Size3.18 GB

Telugu vb
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Telugu.
BHARATGEN
  • See Upvoters0
  • Downloads2
  • Views489
  • File Size3.17 GB

Bengali vb
This 150 M-parameter non-autoregressive speech generative model, developed by BharatGen, is designed for speaker-conditioned Text-to-speech in Bengali.
BHARATGEN
  • See Upvoters0
  • Downloads1
  • Views318
  • File Size3.17 GB


Related Models by Task Type Related Models by Task Type

microsoft/Phi-3-vision-128k-instruct
Phi-3-vision-128k-instruct is a state-of-the-art multimodal model by Microsoft, designed to process both text and visual inputs with a context length of up to 128,000 tokens.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads12
  • Views287
  • File Size0

Phi-3.5-Vision-Instruct ONNX - Optimized Multimodal AI for ONNX Runtime
An ONNX-optimized version of Phi-3.5 Vision-Instruct, quantized to int4 precision for fast and efficient inference on CPUs and GPUs, supporting vision and text-based AI applications.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads4
  • Views338
  • File Size0

Phi-3-Vision-128K-Instruct ONNX - Optimized Multimodal AI for ONNX Runtime
An ONNX-optimized version of Phi-3-Vision-128K-Instruct, designed for efficient multimodal AI inference on CPUs and GPUs, supporting vision and text-based reasoning with INT4 quantization.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads6
  • Views221
  • File Size0

Phi-4 Multimodal Instruct ONNX - Optimized AI Model for ONNX Runtime
Phi-4-multimodal-instruct-onnx is an ONNX version of Microsoft's Phi-4-multimodal-instruct model, quantized to int4 precision to accelerate inference with ONNX Runtime.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads3
  • Views591
  • File Size0

Phi-3.5-Vision-Instruct - Multimodal AI for Image and Text Processing
A state-of-the-art multimodal AI model optimized for vision and text-based reasoning, supporting tasks like image understanding, OCR, and chart analysis, designed for both research and commercial applications.
MICROSOFT CORPORATION (INDIA) PVT. LTD.
  • See Upvoters0
  • Downloads12
  • Views647
  • File Size0

Indic-mobile
Indic-mobile is a 0.5B parameter language model built completely from scratch — no fine-tuning, no adapter on top of an existing checkpoint. Every weight was pretrained from zero, purpose-built for all 22 officially recognized Indian languages and designed for efficient deployment on mobile and edge devices.
AIKOSH OPENSOURCE COMMUNITY
  • See Upvoters2
  • Downloads18
  • Views287
  • File Size953.22 MB

NE-SpeechEmbed
NE-SpeechEmbed is a multilingual speech-text embedding model by MWire Labs for Northeast Indian languages. The model supports semantic speech search, cross-modal retrieval, and audio-text embeddings across Khasi, Garo, Mizo, Nagamese, Kokborok, Assamese, Wancho, and Chakma.
MWIRE LABS
  • See Upvoters0
  • Downloads0
  • Views95
  • File Size0