BharatGen - Param-1-7B-MoE Advancing Multilingual GenAI for India
Param-1-7B-MoE is a multilingual large language model developed under the Param-1 family as part of BharatGen – A Suite of Generative AI Technologies for India. With 7 billion parameters and a Mixture of Experts (MoE) architecture, the model is designed to better understand and generate text across English, Hindi, and 14 additional Indian languages. The model is pretrained from scratch with a strong focus on linguistic diversity, cultural context, and large-scale multilingual representation.
.png){kind=small}
About Model
Key Highlights * 7B parameter Mixture of Experts (MoE) language model * Multilingual: English, Hindi + 14 Indian languages *Base Pre-Trained Checkpoint * Trained on 4 trillion tokens * Uses 64 specialized experts, dynamically activated per token * Supports long-context understanding (up to 4096 tokens) * Designed as a pretrained (PT) base model for downstream fine-tuning Supported Languages In addition to English and Hindi, the model has been trained on data from the following 14 Indian languages: * Assamese, Bengali, Gujarati, Kannada, Maithili, Malayalam, Marathi, Nepali, Oriya, Punjabi, Sanskrit, Sindhi, Tamil, Telugu This broad language coverage enables better performance in region-specific applications and improves inclusivity across India’s linguistic landscape.
BharatGen - Param-1-7B-MoE Advancing Multilingual GenAI for India
Metadata
Attribution 4.0 International (CC BY- 4.0)
bharatgenai
Transformers
PyTorch
Open
Other
06/01/26 12:29:55
0
Activity Overview
1
82 
0 
1,240
Tags
- safetensors
- region:us
- mixtral
License Control
Attribution 4.0 International (CC BY- 4.0)
More Models from BharatGen
- 0
0 - 8.37 GB
23
- 1
- 45
- 57.29 GB
- 1,196
- 0
- 9
- 1.25 GB
- 896
© 2026 - Copyright AIKosh. All rights reserved.