Hercule-ur is an Urdu-language evaluation model from the CIA Suite, designed to assess the performance of multilingual Large Language Models (LLMs). It utilizes English reference responses to evaluate and score Urdu outputs, ensuring accurate assessments aligned with human judgments. The model is fine-tuned on the INTEL dataset and supports zero-shot evaluations on languages not seen during training. Feedback is provided on a scale of 1 to 5, and users can access wrapper functions and classes for seamless integration from the associated GitHub repository.
MIT
Sector Agnostic
Sumanth Doddapaneni and Mohammed Safi Ur Rahman Khan and Dilip Venkatesh and Raj Dabre and Anoop Kunchukuttan and Mitesh M. Khapra
N.A.
N.A.
N.A.
N.A.
N.A.
N.A.
N.A.
N.A.
Open
21/02/25 13:21:00
N.A.
0
© 2026 - Copyright AIKosh. All rights reserved.