Hercule is a cross-lingual evaluation model from the CIA Suite, designed to assess multilingual LLMs, with a focus on Hindi. It evaluates multilingual outputs using English reference responses and aligns closely with human judgments, outperforming zero-shot models like GPT-4 on the RECON test set. Fine-tuned on the INTEL dataset, it excels in low-resource scenarios and supports zero-shot evaluation for unseen languages. Built on Llama-3.1-8B-Instruct, it employs a structured 1-5 scoring system and supports lightweight fine-tuning with LoRA.
MIT
Sector Agnostic
Sumanth Doddapaneni and Mohammed Safi Ur Rahman Khan and Dilip Venkatesh and Raj Dabre and Anoop Kunchukuttan and Mitesh M. Khapra
N.A.
N.A.
N.A.
N.A.
N.A.
N.A.
N.A.
N.A.
Open
21/02/25 13:21:55
N.A.
0
© 2026 - Copyright AIKosh. All rights reserved.