Indian Flag
Government Of India
A-
A
A+
UpvoteUpvote
0

CoRil-Parallel

Indic Parallel Corpus: 11 Indian Language Pairs for Machine Translation
2.75
AI Score infoDownload score report
  • Upvotes0upvotes
  • Downloads9downloads
  • Views266views
  • File size160.72 MB
  • Shares0Shares
  • Last updatedupdated 15 Dec 2025
UpvoteUpvote
0
COVERAGE Country
LICENSE Attribution 4.0 International (CC BY- 4.0)
TYPE Structured
SECTOR Science, Technology and Research
UPDATES NA

About Dataset

Indic Parallel Corpus: 11 Indian Language Pairs for Machine Translation

This dataset contains parallel corpus for machine translation across 11 Indian language pairs. The data is curated to cover three distinct domains: Governance, Health, and General. This dataset is designed to help researchers and developers build and evaluate robust machine translation models for Indian languages.

Description:
The corpus provides parallel sentences for a variety of language pairs, with a focus on Hindi as a pivot language. All translation pairs are bidirectional. The data has been sourced and cleaned to be useful for training Neural Machine Translation (NMT) models.

The dataset includes the following 11 language pairs:

Source Language Target Language Language Codes
Hindi Gujarati hi - gu
Hindi Kashmiri hi - ks
Hindi Telugu hi - te
Hindi Kannada hi - kn
Hindi Punjabi hi - pa
Hindi Oriya hi - or
Hindi Urdu hi - ur
Hindi Sindhi hi - sd
Hindi Dogri hi - doi
English Hindi en - hi
Telugu English te - en

Dataset Structure and Statistics:
The data is organized by language pair and domain. Each language pair directory contains sub-directories for the specific domains. The following table provides a detailed breakdown of the number of parallel sentences for each language pair, domain, and data split (train/dev/test). An empty cell indicates that data for that specific domain is not available.

Language Pair General (Train) General (Dev) General (Test) Governance (Train) Governance (Dev) Governance (Test) Health (Train) Health (Dev) Health (Test)
dg_hi 12,411 500 500 6,947 500 500
en_hi 38,790 500 500 10,043 500 500
en_te 9,976 500 500 17,237 500 500
gu_hi 18,850 500 500
hi_dg 30,359 500 500 4,343 500 500
hi_en 42,964 500 500 12,187 500 500
hi_gu 26,335 500 500 4,899 500 500
hi_kn 27,531 500 500 16,351 500 500
hi_ks 21,103 500 500
hi_or 24,291 500 500 9,387 500 500
hi_pa 30,373 500 500 11,328 500 500
hi_sd 21,548 500 500 13,233 500 500
hi_te 8,061 500 500 11,911 500 500
hi_ur 8,956 500 500 9,929 500 500 5,271 500 500
kn_hi 16,040 500 500 19,148 500 500
ks_ur 2,606 500 500
or_hi 11,581 500 500 18,308 500 500
pa_hi 22,098 500 500 22,532 500 500
sd_hi 3,499 500 500
te_en 5,527 500 500 6,008 500 500
te_hi 4,405 500 500 18,246 500 500
ur_hi 27,791 500 500 8,938 500 500 6,259 500 500
ur_ks 22,820 500 500

Domains:
    1. Governance: Includes sentences from government documents, press releases, and legal texts.
    2. Health: Comprises text from medical journals, healthcare advisories, and public health communications.
    3. General: A broad category including sentences from news articles, websites, and miscellaneous sources.

Data Format:
Each dataset configuration is provided as a single tab-separated text file (.txt).
Each line in the file represents a parallel sentence pair, with the source language sentence and the target language sentence separated by a single tab character (\t).

Citation :

If you use this dataset in your research, please consider citing it.

@misc{bhattacharjee2025corilenrichingindianlanguage,
      title={CorIL: Towards Enriching Indian Language to Indian Language Parallel Corpora and Machine Translation Systems}, 
      author={Soham Bhattacharjee and Mukund K Roy and Yathish Poojary and Bhargav Dave and Mihir Raj and Vandan Mujadia and Baban Gain and Pruthwik Mishra and Arafat Ahsan and Parameswari Krishnamurthy and Ashwath Rao and Gurpreet Singh Josan and Preeti Dubey and Aadil Amin Kak and Anna Rao Kulkarni and Narendra VG and Sunita Arora and Rakesh Balbantray and Prasenjit Majumdar and Karunesh K Arora and Asif Ekbal and Dipti Mishra Sharma},
      year={2025},
      eprint={2509.19941},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2509.19941}, 
}

Paper Link:
https://arxiv.org/abs/2509.19941

Tags

  • Parallel Corpus
  • Indian language processing
  • language:hi
  • language:en
  • language:te
  • language:ta
  • language:kn
  • task_categories:translation
  • language:ks
  • language:or
  • language:sd
  • language:pa
  • language:ur
  • language:doi

Related Datasets Related Datasets

Chat-Dataset
Chat-Dataset
4.8Information•Updated 13 day(s) ago
AIKOSH OPENSOURCE COMMUNITY
  • See Upvoters2
  • Downloads9
  • Views79
  • File Size777.41 KB

DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads1
  • Views246
  • File Size12.78 MB

SOCIETY FOR ELECTRONIC TRANSACTIONS AND SECURITY (SETS), UNDER OFFICE OF PSA TO GOVT. OF INDIA
  • See Upvoters0
  • Downloads13
  • Views130
  • File Size27.06 GB

IITG-Mizo-MT-data-v2
IITG-Mizo-MT-data-v2
4.5Information•Updated 30 day(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads0
  • Views69
  • File Size5.14 MB

IITG-Mizospeech-201026-V2
IITG-Mizospeech-201026-V2
2.8Information•Updated 30 day(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads1
  • Views53
  • File Size329.57 MB

IITG-Mizospeech-220726-V1
IITG-Mizospeech-220726-V1
4.5Information•Updated 1 month(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads4
  • Views332
  • File Size293.34 MB

SOCIETY FOR ELECTRONIC TRANSACTIONS AND SECURITY (SETS), UNDER OFFICE OF PSA TO GOVT. OF INDIA
  • See Upvoters0
  • Downloads8
  • Views75
  • File Size43.39 GB

SOCIETY FOR ELECTRONIC TRANSACTIONS AND SECURITY (SETS), UNDER OFFICE OF PSA TO GOVT. OF INDIA
  • See Upvoters0
  • Downloads10
  • Views79
  • File Size26.69 GB

SPICOR TTS 3 Sourashtra Corpus
SPICOR TTS 3 Sourashtra Corpus
-Information•Updated 1 month(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads13
  • Views108
  • File Size0

SPICOR TTS_1.0 Indian English Corpus
SPICOR TTS_1.0 Indian English Corpus
-Information•Updated 1 month(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads2
  • Views51
  • File Size0


Related Datasets by Organization Related Datasets by Organization

Chat-Dataset
Chat-Dataset
4.8Information•Updated 13 day(s) ago
AIKOSH OPENSOURCE COMMUNITY
  • See Upvoters2
  • Downloads9
  • Views79
  • File Size777.41 KB

DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads1
  • Views246
  • File Size12.78 MB

SOCIETY FOR ELECTRONIC TRANSACTIONS AND SECURITY (SETS), UNDER OFFICE OF PSA TO GOVT. OF INDIA
  • See Upvoters0
  • Downloads13
  • Views130
  • File Size27.06 GB

IITG-Mizo-MT-data-v2
IITG-Mizo-MT-data-v2
4.5Information•Updated 30 day(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads0
  • Views69
  • File Size5.14 MB

IITG-Mizospeech-201026-V2
IITG-Mizospeech-201026-V2
2.8Information•Updated 30 day(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads1
  • Views53
  • File Size329.57 MB

IITG-Mizospeech-220726-V1
IITG-Mizospeech-220726-V1
4.5Information•Updated 1 month(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads4
  • Views332
  • File Size293.34 MB

SOCIETY FOR ELECTRONIC TRANSACTIONS AND SECURITY (SETS), UNDER OFFICE OF PSA TO GOVT. OF INDIA
  • See Upvoters0
  • Downloads8
  • Views75
  • File Size43.39 GB

SOCIETY FOR ELECTRONIC TRANSACTIONS AND SECURITY (SETS), UNDER OFFICE OF PSA TO GOVT. OF INDIA
  • See Upvoters0
  • Downloads10
  • Views79
  • File Size26.69 GB

SPICOR TTS 3 Sourashtra Corpus
SPICOR TTS 3 Sourashtra Corpus
-Information•Updated 1 month(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads13
  • Views108
  • File Size0

SPICOR TTS_1.0 Indian English Corpus
SPICOR TTS_1.0 Indian English Corpus
-Information•Updated 1 month(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads2
  • Views51
  • File Size0


Related Datasets by Sector Related Datasets by Sector

Chat-Dataset
Chat-Dataset
4.8Information•Updated 13 day(s) ago
AIKOSH OPENSOURCE COMMUNITY
  • See Upvoters2
  • Downloads9
  • Views79
  • File Size777.41 KB

DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads1
  • Views246
  • File Size12.78 MB

SOCIETY FOR ELECTRONIC TRANSACTIONS AND SECURITY (SETS), UNDER OFFICE OF PSA TO GOVT. OF INDIA
  • See Upvoters0
  • Downloads13
  • Views130
  • File Size27.06 GB

IITG-Mizo-MT-data-v2
IITG-Mizo-MT-data-v2
4.5Information•Updated 30 day(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads0
  • Views69
  • File Size5.14 MB

IITG-Mizospeech-201026-V2
IITG-Mizospeech-201026-V2
2.8Information•Updated 30 day(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads1
  • Views53
  • File Size329.57 MB

IITG-Mizospeech-220726-V1
IITG-Mizospeech-220726-V1
4.5Information•Updated 1 month(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads4
  • Views332
  • File Size293.34 MB

SOCIETY FOR ELECTRONIC TRANSACTIONS AND SECURITY (SETS), UNDER OFFICE OF PSA TO GOVT. OF INDIA
  • See Upvoters0
  • Downloads8
  • Views75
  • File Size43.39 GB

SOCIETY FOR ELECTRONIC TRANSACTIONS AND SECURITY (SETS), UNDER OFFICE OF PSA TO GOVT. OF INDIA
  • See Upvoters0
  • Downloads10
  • Views79
  • File Size26.69 GB

SPICOR TTS 3 Sourashtra Corpus
SPICOR TTS 3 Sourashtra Corpus
-Information•Updated 1 month(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads13
  • Views108
  • File Size0

SPICOR TTS_1.0 Indian English Corpus
SPICOR TTS_1.0 Indian English Corpus
-Information•Updated 1 month(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads2
  • Views51
  • File Size0


Related Datasets by Task Type Related Datasets by Task Type

Chat-Dataset
Chat-Dataset
4.8Information•Updated 13 day(s) ago
AIKOSH OPENSOURCE COMMUNITY
  • See Upvoters2
  • Downloads9
  • Views79
  • File Size777.41 KB

DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads1
  • Views246
  • File Size12.78 MB

SOCIETY FOR ELECTRONIC TRANSACTIONS AND SECURITY (SETS), UNDER OFFICE OF PSA TO GOVT. OF INDIA
  • See Upvoters0
  • Downloads13
  • Views130
  • File Size27.06 GB

IITG-Mizo-MT-data-v2
IITG-Mizo-MT-data-v2
4.5Information•Updated 30 day(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads0
  • Views69
  • File Size5.14 MB

IITG-Mizospeech-201026-V2
IITG-Mizospeech-201026-V2
2.8Information•Updated 30 day(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads1
  • Views53
  • File Size329.57 MB

IITG-Mizospeech-220726-V1
IITG-Mizospeech-220726-V1
4.5Information•Updated 1 month(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads4
  • Views332
  • File Size293.34 MB

SOCIETY FOR ELECTRONIC TRANSACTIONS AND SECURITY (SETS), UNDER OFFICE OF PSA TO GOVT. OF INDIA
  • See Upvoters0
  • Downloads8
  • Views75
  • File Size43.39 GB

SOCIETY FOR ELECTRONIC TRANSACTIONS AND SECURITY (SETS), UNDER OFFICE OF PSA TO GOVT. OF INDIA
  • See Upvoters0
  • Downloads10
  • Views79
  • File Size26.69 GB

SPICOR TTS 3 Sourashtra Corpus
SPICOR TTS 3 Sourashtra Corpus
-Information•Updated 1 month(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads13
  • Views108
  • File Size0

SPICOR TTS_1.0 Indian English Corpus
SPICOR TTS_1.0 Indian English Corpus
-Information•Updated 1 month(s) ago
DIGITAL INDIA BHASHINI DIVISION
  • See Upvoters0
  • Downloads2
  • Views51
  • File Size0