COMI-LINGUA

About Dataset

COMI-LINGUA (COde-MIxing and LINGuistic Insights on Natural Hinglish Usage and Annotation) is a high-quality Hindi-English code-mixed dataset, manually annotated by three expert annotators. It serves as a multitask benchmark for multilingual NLP models.

COMI-LINGUA provides annotations for the following key NLP tasks:

1. Language Identification (LID): Token-wise classification of Hindi, English, and other units.

Initial predictions generated using the Microsoft LID tool; annotators reviewed and corrected.

• Example sentence: प्रधानमंत्री नरेन्द्र मोदी डिजिटल इंडिया मिशन को आगे बढ़ाने के लिए पिछले सप्ताह Google के CEO सुंदर पिचाई से मुलाकात की थी ।
• LID tags: hi hi hi en en en hi hi hi hi hi hi hi en hi en hi hi hi hi hi ot

2. Matrix Language Identification (MLI): Sentence-level dominant language annotation.

• Example sentence: किसानों को अपनी फसल बेचने में दिक्कत न हो इसके लिये Electronic National Agriculture Market यानि ई-नाम योजना तेजी से काम हो रहा है।
• Matrix Language: hi

3. Part-of-Speech (POS) Tagging: Syntactic categorization of tokens.

Tags pre-assigned using the CodeSwitch NLP library; annotators reviewed and corrected.

• Example sentence: भारत द्वारा बनाया गया Unified Payments Interface यानि UPI भारत की एक बहुत बड़ी success story है ।
• POS tags: PROPN ADP VERB VERB PROPN PROPN PROPN CONJ PROPN PROPN ADP DET ADJ ADJ NOUN NOUN VERB X

4. Named Entity Recognition (NER): Identification of named entities in Hinglish text.

Tokens pre-tagged for language using the CodeSwitch NLP library; annotators reviewed and corrected.

• Example sentence: मालूम हो कि पेरिस स्थित Financial Action Task Force, FATF ने जून 2018 में पाकिस्तान को ग्रे लिस्ट में रखा था।
• NER tags: "पेरिस": GPE, "Financial Action Task Force, FATF": ORGANISATION, "2018": DATE, "पाकिस्तान": GPE

5. Machine Translation (MT): Parallel translation in three variants: English, Romanized Hindi, Devanagari Hindi. Initial translation predictions were generated using the Llama 3.3 LLM; which annotators then refined and corrected by annotators.

• Example Sentence: भारत में भी green growth, climate resilient infrastructure और ग्रीन transition पर विशेष रूप से बल दिया जा रहा है।
• English: In India too, special emphasis is being given to green growth, climate resilient infrastructure, and green transition.
• Romanized Hindi: Bharat mein bhi green growth, climate resilient infrastructure aur green transition par vishesh roop se bal diya ja raha hai.
• Devanagari Hindi: भारत में भी हरित विकास, जलवायु सहनशील आधारिक संरचना और हरित संक्रमण पर विशेष रूप से बल दिया जा रहा है।

6. Text Normalization (TN): Sentence-level normalization of noisy, informal, code-mixed Hinglish to standardized form. Common variants (e.g. hain/hai/hayn/hay/he → hain) mapped consistently. Initial suggestions from GPT-OSS-120B; annotators reviewed and corrected.

• Example Sentence: Janmdin ki dheron shubhkaamnaayen sir... Aap swasth rahen... Dirghayu ho... Yahi hamare poore pariwaar ki shubhkaamnaayen hain
• Normalized: Janmadin ki dheron shubhkaamnayein sir. Aap swasth rahe. Dirghayu ho, yahi hamare pure parivaar ki shubhkaamnayein hain.

Citation:

@inproceedings{sheth-etal-2025-comi,
    title = "{COMI}-{LINGUA}: Expert Annotated Large-Scale Dataset for Multitask {NLP} in {H}indi-{E}nglish Code-Mixing",
    author = "Sheth, Rajvee  and
      Beniwal, Himanshu  and
      Singh, Mayank",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.findings-emnlp.422/",
    pages = "7973--7992",
    ISBN = "979-8-89176-335-7",
}