BharatGen - Param-1-7B-MoE Advancing Multilingual GenAI for India

About Model

Key Highlights * 7B parameter Mixture of Experts (MoE) language model * Multilingual: English, Hindi + 14 Indian languages *Base Pre-Trained Checkpoint * Trained on 4 trillion tokens * Uses 64 specialized experts, dynamically activated per token * Supports long-context understanding (up to 4096 tokens) * Designed as a pretrained (PT) base model for downstream fine-tuning Supported Languages In addition to English and Hindi, the model has been trained on data from the following 14 Indian languages: * Assamese, Bengali, Gujarati, Kannada, Maithili, Malayalam, Marathi, Nepali, Oriya, Punjabi, Sanskrit, Sindhi, Tamil, Telugu This broad language coverage enables better performance in region-specific applications and improves inclusivity across India’s linguistic landscape.