Indian Flag
Government Of India
A-
A
A+
ORGANISATION
UpvoteUpvote
1

Large-Scale Multilingual Training Datasets for Large Language Models

AI Kosh, in collaboration with Meta and Sarvam AI, is releasing 12 billion synthetic Indic training tokens across 10 Indian languages, along with a fine-tuned 17B Llama-4 MoE model and its fine-tuning pipeline. The initiative strengthens India’s AI training infrastructure and enables scalable multilingual model development across sectors such as science, healthcare, education, and agriculture.

About Use Case

As part of its mission to strengthen India’s AI ecosystem, AI Kosh, in collaboration with Meta and Sarvam AI, is making available 12 billion high-quality synthetic Indic training tokens along with a fine-tuned 17B Llama-4 MoE model and the reproducible fine-tuning pipeline used to train it.

The release includes 14 structured datasets designed for training and fine-tuning large language models in 10 major Indian languages: Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, and Telugu.

The datasets are generated through large-scale synthetic data generation pipelines to augment scarce high-quality training data in low-resource Indian languages. They are structured across three real-world linguistic styles: formal language for academic and official use cases, code-mixed conversational formats such as Hinglish and Tanglish, and transliteration across Roman and native scripts.

The datasets cover a broad range of high-impact tasks, including reasoning, instruction following, conversational AI, code generation, agentic workflows, multilingual translation, creative writing, and summarization. This breadth enables the development of multilingual AI systems that can operate reliably across domains such as scientific research support, healthcare communication, educational copilots, agricultural advisory systems, citizen services, and enterprise knowledge platforms.

In addition to the datasets, the initiative includes a fine-tuned 17B Llama-4 MoE model with 17B active parameters across 16 experts and 109B total parameters. Trained on a subset of the released dataset, the model demonstrates measurable improvements in Indic language fluency, instruction following, domain-specific reasoning, and cross-lingual performance.

The model release includes training configuration details, fine-tuning scripts, preprocessing utilities, evaluation setup, and reproducibility documentation. Together, these assets provide a reference implementation for open, large-scale multilingual model training and lower the barrier for researchers, startups, enterprises, and public institutions to build high-performance AI systems aligned with India’s linguistic diversity.

Tags Tags

  • Model Fine-Tuning
  • Synthetic Data
  • LLM Training
  • Multilingual AI
  • Indic Languages
  • MoE Model
  • Open-Source AI
  • Foundation Models

Tags Sector

Sector Agnostic

Related Datasets Related Datasets

Creative Tasks
Creative Tasks
3.8Information•Updated 9 month(s) ago
FACEBOOK INDIA ONLINE SERVICES PRIVATE LIMITED
  • See Upvoters0
  • Downloads10
  • Views305
  • File Size100.33 MB

Crosslingual
Crosslingual
3.8Information•Updated 9 month(s) ago
FACEBOOK INDIA ONLINE SERVICES PRIVATE LIMITED
  • See Upvoters0
  • Downloads7
  • Views268
  • File Size185.06 MB

Deliberative Alignment
Deliberative Alignment
3.8Information•Updated 9 month(s) ago
FACEBOOK INDIA ONLINE SERVICES PRIVATE LIMITED
  • See Upvoters0
  • Downloads12
  • Views261
  • File Size10.32 MB

gsm8k Train
gsm8k Train
3.8Information•Updated 9 month(s) ago
FACEBOOK INDIA ONLINE SERVICES PRIVATE LIMITED
  • See Upvoters0
  • Downloads11
  • Views537
  • File Size18.77 MB

Multiturn Code
Multiturn Code
3.8Information•Updated 9 month(s) ago
FACEBOOK INDIA ONLINE SERVICES PRIVATE LIMITED
  • See Upvoters0
  • Downloads17
  • Views256
  • File Size321.09 MB

Nemotron GPT oss Reasoning
Nemotron GPT oss Reasoning
3.8Information•Updated 9 month(s) ago
FACEBOOK INDIA ONLINE SERVICES PRIVATE LIMITED
  • See Upvoters0
  • Downloads5
  • Views260
  • File Size6.59 GB

STEM Code Reasoning
STEM Code Reasoning
3.8Information•Updated 9 month(s) ago
FACEBOOK INDIA ONLINE SERVICES PRIVATE LIMITED
  • See Upvoters0
  • Downloads3
  • Views264
  • File Size1.02 GB

Teacher Student
Teacher Student
3.8Information•Updated 9 month(s) ago
FACEBOOK INDIA ONLINE SERVICES PRIVATE LIMITED
  • See Upvoters0
  • Downloads17
  • Views387
  • File Size191.74 MB

Translation Related
Translation Related
3.8Information•Updated 9 month(s) ago
FACEBOOK INDIA ONLINE SERVICES PRIVATE LIMITED
  • See Upvoters0
  • Downloads4
  • Views227
  • File Size133.16 MB

Web Summarisation
Web Summarisation
3.8Information•Updated 9 month(s) ago
FACEBOOK INDIA ONLINE SERVICES PRIVATE LIMITED
  • See Upvoters0
  • Downloads5
  • Views214
  • File Size28.08 MB

Related Models Related Models

Meta Llama-4-Scout-17B-16E-Instruct
Llama 4 Scout 17B-16E-Instruct is a 17B-parameter, instruction-tuned multimodal model developed in collaboration with AI Kosh, Meta and Sarvam. Built on a 16-expert Mixture-of-Experts architecture, it is optimized for high-performance text generation and image understanding, delivering strong efficiency and scalable multimodal reasoning capabilities.
FACEBOOK INDIA ONLINE SERVICES PRIVATE LIMITED
  • See Upvoters0
  • Downloads23
  • Views1,058
  • File Size162.05 GB