Hercule is a cross-lingual evaluation model from the CIA Suite, designed to assess multilingual LLMs, with a focus on Bengali. It evaluates multilingual outputs using English reference responses and is fine-tuned on the INTEL dataset for enhanced alignment with human judgments. Hercule outperforms zero-shot models like GPT-4 on the RECON test set, excelling in low-resource scenarios and supporting zero-shot evaluation for unseen languages. Built on Llama-3.1-8B-Instruct, it uses LoRA for efficient multilingual assessment.
MIT
Sector Agnostic
Sumanth Doddapaneni and Mohammed Safi Ur Rahman Khan and Dilip Venkatesh and Raj Dabre and Anoop Kunchukuttan and Mitesh M. Khapra
N.A.
N.A.
N.A.
N.A.
N.A.
N.A.
N.A.
N.A.
Open
21/02/25 13:21:54
N.A.
0
© 2026 - Copyright AIKosh. All rights reserved.