COMI-LINGUA-LID

About Model

Hindi-English Token-level LID Tagger

A LoRA-adapted Transformer LLM fine-tuned for token-level Language Identification (LID) on Hindi–English (Hinglish) code-mixed text.

Supported Language Tags

• en - English
• hi - Hindi
• ot - Other (punctuation, symbols, hashtags, emojis, URLs, mixed/unknown script elements)

Model Overview

• Model type: LoRA-adapted Transformer LLM
• Base model: CohereForAI/aya-expanse-8b
• Total parameters: 8B
• Trainable parameters: ~32M
• License: Apache 2.0
• Languages: Hindi, English (code-mixed)

Performance

Achieves 94.90 F1 on the COMI-LINGUA LID test set (5K instances), outperforming zero-shot inference from strong closed-source LLMs (e.g., gpt-4o ≈ 92.7 F1) and traditional tools (e.g., Microsoft LID ≈ 74.4 F1), establishing strong performance among open-weight models for fine-grained Hinglish language tagging.

Example Inference

Identify the language of each token (en = English, hi = Hindi, ot = Other) in the sentence:

New Delhi/Alive News : आज के दिन में इंडिया की टीम ने ऑस्ट्रेलिया को हराया। #INDvAUS

Output:

[
  {"New": "en"},
  {"Delhi": "en"},
  {"/": "ot"},
  {"Alive": "en"},
  {"News": "en"},
  {":": "ot"},
  {"आज": "hi"},
  {"के": "hi"},
  {"दिन": "hi"},
  {"में": "hi"},
  {"इंडिया": "en"},
  {"की": "hi"},
  {"टीम": "en"},
  {"ने": "hi"},
  {"ऑस्ट्रेलिया": "en"},
  {"को": "hi"},
  {"हराया": "hi"},
  {"।": "ot"},
  {"#INDvAUS": "ot"}
]

Citation

@inproceedings{sheth-etal-2025-comi,
  title = "{COMI}-{LINGUA}: Expert Annotated Large-Scale Dataset for Multitask {NLP} in {H}indi-{E}nglish Code-Mixing",
  author = "Sheth, Rajvee and
               Beniwal, Himanshu and
               Singh, Mayank",
  editor = "Christodoulopoulos, Christos and
               Chakraborty, Tanmoy and
               Rose, Carolyn and
               Peng, Violet",
  booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
  month = nov,
  year = "2025",
  address = "Suzhou, China",
  publisher = "Association for Computational Linguistics",
  url = "https://aclanthology.org/2025.findings-emnlp.422/",
  pages = "7973--7992",
  isbn = "979-8-89176-335-7"
}