Indian Flag
Government Of India
A-
A
A+
/IIT Bombay ORGANISATION
UpvoteUpvote
0

LecaDis

Dictionary Constrained Disambiguation for Improved NMT
  • Upvotes0upvotes
  • Downloads38downloads
  • Views1,045views
  • File size4.70 GB
  • Shares0Shares
  • Last updatedupdated 14 May 2025
UpvoteUpvote
0
LICENSE CC0 1.0 Public Domain
SECTOR Education and Skill Development

About Model

Domain-specific neural machine translation (NMT) systems (, in educational applications) are socially significant with the potential to help make information accessible to a diverse set of users in multilingual societies. Such NMT systems should be lexically constrained and draw from domain-specific dictionaries. Dictionaries could present multiple candidate translations for a source word/phrase due to the polysemous nature of words. The onus is then on the NMT model to choose the contextually most appropriate candidate. Prior work has largely ignored this problem and focused on the single candidate constraint setting wherein the target word or phrase is replaced by a single constraint. In this work, we present DictDis, a lexically constrained NMT system that disambiguates between multiple candidate translations derived from dictionaries. We achieve this by augmenting training data with multiple dictionary candidates to actively encourage disambiguation during training by implicitly aligning multiple candidate constraints. We demonstrate the utility of DictDis via extensive experiments on English-Hindi, English-German, and English-French datasets across a variety of domains including regulatory, finance, engineering, health and standard benchmark test datasets. In comparison with existing approaches for lexically constrained and unconstrained NMT, we demonstrate superior performance for the copy constraint and disambiguation-related measures on all domains, while also obtaining improved fluency of up to 2-3 BLEU points on some domains. We also release our test set consisting of 4K English-Hindi sentences in multiple domains.

Metadata Metadata

CC0 1.0 Public Domain

Education and Skill Development

IIT Bombay

N.A.

IIT Bombay

ALVIYA AJOY RAJ

N.A.

N.A.

N.A.

N.A.

N.A.

N.A.

N.A.

Open

14/05/25 11:54:12

N.A.

4.70 GB

Tags

  • Multilingual Dataset
  • Indian Language
  • natural language processing (NLP)

Related Models Related Models

NCERT-TUTOR-v3
NCERT-aligned tutor LoRA for on-device K-12 tutoring. Runs on 4 GB VRAM (RTX 3050). Stage-3 chained adapter: 15.7k curated questions (Grades 6-12, 13 subjects) + 3 DPO stages (math reasoning, mined factuality, multi-subject). 58% on held-out NCERT math with worked steps.
AIKOSH OPENSOURCE COMMUNITY
  • See Upvoters0
  • Downloads5
  • Views209
  • File Size65.36 MB

IndNumNLI-Classifier-v1
IndNumNLI Classifier is a soft-voting ensemble of Random Forest and Gradient Boosting trained on 16 numerical reasoning features extracted from Indian government statistical premise-hypothesis pairs. It classifies NLI labels as entailment, neutral, or contradiction with a test accuracy of 60 percent and macro F1 of 0.64.
SYMBIOSIS SKILLS AND PROFESSIONAL UNIVERSITY
  • See Upvoters0
  • Downloads1
  • Views67
  • File Size584.97 KB

NEET-BioBERT
NEET-BioBERT is a fine-tuned version of DistilBERT (base uncased) specifically trained to classify the correct option for NEET-style multiple-choice biology questions. It selects the best answer among four choices (A, B, C, D).
AIKOSH OPENSOURCE COMMUNITY
  • See Upvoters0
  • Downloads8
  • Views165
  • File Size256.10 MB

parrotlet e
Parrotlet-e is a state of the art multilingual medical entity embedding model designed for understanding and linking medical terms across Indian languages.
EKA CARE
  • See Upvoters3
  • Downloads22
  • Views580
  • File Size0

KhasiBERT
Khasi language model trained on 3.6M sentences using RoBERTa architecture. 110M parameters. Supports NLP tasks for Khasi text processing.
MWIRE LABS
  • See Upvoters1
  • Downloads34
  • Views1,322
  • File Size0

AI4Bharat - Romansetu-200M -Multilingual LLM for Indian langauges using romanization
RomanSetu is Efficiently unlocking multilingual (Indian Languages) capabilities of Large Language Models via Romanization.
AI4BHARAT
  • See Upvoters0
  • Downloads6
  • Views369
  • File Size0

AI4Bharat - Romansetu-100M - Multilingual LLM for Indian langauges using romanization
RomanSetu is Efficiently unlocking multilingual (Indian Languages) capabilities of Large Language Models via Romanization.
AI4BHARAT
  • See Upvoters0
  • Downloads11
  • Views711
  • File Size0

AI4Bharat – Romanized Path – Base to Supervised Fine-Tuning (SFT)
Romansetu model is built on base pretrained model which is supervised fine tuned on instuction-following tasks using romanized Indian languages.
AI4BHARAT
  • See Upvoters0
  • Downloads3
  • Views268
  • File Size0

AI4Bharat – Native-Script Path – Base to Supervised Fine-Tuning (SFT)
Romansetu model is built on base pretrained model which is supervised fine-tuned on instruction datasets in native scripts of Indian languages.
AI4BHARAT
  • See Upvoters0
  • Downloads1
  • Views72
  • File Size0

AI4Bharat - Romansetu-300M -Multilingual LLM for Indian Languages using Native script
RomanSetu is Efficiently unlocking multilingual capabilities of Large Language Models via Native Script.
AI4BHARAT
  • See Upvoters0
  • Downloads3
  • Views74
  • File Size0


Related Models by Sector Related Models by Sector

NCERT-TUTOR-v3
NCERT-aligned tutor LoRA for on-device K-12 tutoring. Runs on 4 GB VRAM (RTX 3050). Stage-3 chained adapter: 15.7k curated questions (Grades 6-12, 13 subjects) + 3 DPO stages (math reasoning, mined factuality, multi-subject). 58% on held-out NCERT math with worked steps.
AIKOSH OPENSOURCE COMMUNITY
  • See Upvoters0
  • Downloads5
  • Views209
  • File Size65.36 MB

IndNumNLI-Classifier-v1
IndNumNLI Classifier is a soft-voting ensemble of Random Forest and Gradient Boosting trained on 16 numerical reasoning features extracted from Indian government statistical premise-hypothesis pairs. It classifies NLI labels as entailment, neutral, or contradiction with a test accuracy of 60 percent and macro F1 of 0.64.
SYMBIOSIS SKILLS AND PROFESSIONAL UNIVERSITY
  • See Upvoters0
  • Downloads1
  • Views67
  • File Size584.97 KB

NEET-BioBERT
NEET-BioBERT is a fine-tuned version of DistilBERT (base uncased) specifically trained to classify the correct option for NEET-style multiple-choice biology questions. It selects the best answer among four choices (A, B, C, D).
AIKOSH OPENSOURCE COMMUNITY
  • See Upvoters0
  • Downloads8
  • Views165
  • File Size256.10 MB

KhasiBERT
Khasi language model trained on 3.6M sentences using RoBERTa architecture. 110M parameters. Supports NLP tasks for Khasi text processing.
MWIRE LABS
  • See Upvoters1
  • Downloads34
  • Views1,322
  • File Size0


Related Models by Task Type Related Models by Task Type

parrotlet e
Parrotlet-e is a state of the art multilingual medical entity embedding model designed for understanding and linking medical terms across Indian languages.
EKA CARE
  • See Upvoters3
  • Downloads22
  • Views580
  • File Size0

AI4Bharat - Romansetu-200M -Multilingual LLM for Indian langauges using romanization
RomanSetu is Efficiently unlocking multilingual (Indian Languages) capabilities of Large Language Models via Romanization.
AI4BHARAT
  • See Upvoters0
  • Downloads6
  • Views369
  • File Size0

AI4Bharat - Romansetu-100M - Multilingual LLM for Indian langauges using romanization
RomanSetu is Efficiently unlocking multilingual (Indian Languages) capabilities of Large Language Models via Romanization.
AI4BHARAT
  • See Upvoters0
  • Downloads11
  • Views711
  • File Size0

AI4Bharat – Romanized Path – Base to Supervised Fine-Tuning (SFT)
Romansetu model is built on base pretrained model which is supervised fine tuned on instuction-following tasks using romanized Indian languages.
AI4BHARAT
  • See Upvoters0
  • Downloads3
  • Views268
  • File Size0

AI4Bharat – Native-Script Path – Base to Supervised Fine-Tuning (SFT)
Romansetu model is built on base pretrained model which is supervised fine-tuned on instruction datasets in native scripts of Indian languages.
AI4BHARAT
  • See Upvoters0
  • Downloads1
  • Views72
  • File Size0

AI4Bharat - Romansetu-300M -Multilingual LLM for Indian Languages using Native script
RomanSetu is Efficiently unlocking multilingual capabilities of Large Language Models via Native Script.
AI4BHARAT
  • See Upvoters0
  • Downloads3
  • Views74
  • File Size0

AI4Bharat - Romansetu-200M -Multilingual LLM for Indian Languages using Native script
RomanSetu is Efficiently unlocking multilingual capabilities of Large Language Models via Native Script.
AI4BHARAT
  • See Upvoters0
  • Downloads1
  • Views70
  • File Size0

AI4Bharat - Romansetu-100M -Multilingual LLM for Indian Languages using Native script
RomanSetu is Efficiently unlocking multilingual (Indian Langauages) capabilities of Large Language Models via Native Scripts.
AI4BHARAT
  • See Upvoters0
  • Downloads1
  • Views188
  • File Size0

AI4Bharat - Romansetu-300M -Multilingual LLM for Indian langauges using romanization
RomanSetu is Efficiently unlocking multilingual capabilities (Indian Languages) of Large Language Models via Romanization.
AI4BHARAT
  • See Upvoters0
  • Downloads3
  • Views398
  • File Size0

sarvamtranslate
Translation model for 22 Indian Languages
SARVAM AI
  • See Upvoters1
  • Downloads129
  • Views2,507
  • File Size0