AI4Bharat- Hercules Bengali LORA: Evaluation Model

About Model

Hercule is a cross-lingual evaluation model from the CIA Suite, designed to assess multilingual LLMs, with a focus on Bengali. It evaluates multilingual outputs using English reference responses and is fine-tuned on the INTEL dataset for enhanced alignment with human judgments. Hercule outperforms zero-shot models like GPT-4 on the RECON test set, excelling in low-resource scenarios and supporting zero-shot evaluation for unseen languages. Built on Llama-3.1-8B-Instruct, it uses LoRA for efficient multilingual assessment.