An Urdu-language evaluation model aimed at assessing multilingual Large Language Models by comparing outputs to English reference responses.
Hercule-ur is an Urdu-language evaluation model from the CIA Suite, designed to assess the performance of multilingual Large Language Models (LLMs). It utilizes English reference responses to evaluate and score Urdu outputs, ensuring accurate assessments aligned with human judgments. The model is fine-tuned on the INTEL dataset and supports zero-shot evaluations on languages not seen during training. Feedback is provided on a scale of 1 to 5, and users can access wrapper functions and classes for seamless integration from the associated GitHub repository.
MIT
Sumanth Doddapaneni and Mohammed Safi Ur Rahman Khan and Dilip Venkatesh and Raj Dabre and Anoop Kunchukuttan and Mitesh M. Khapra
Evaluator Language model
N.A.
Open
Sector Agnostic
21/02/25 13:21:00
0
MIT
© 2026 - Copyright AIKosh. All rights reserved. This portal is developed by National e-Governance Division for AIKosh mission.