AI4Bharat- Hercules Urdu LORA Model: Evaluation Model

About Model

Hercule is an advanced cross-lingual evaluation model introduced as part of the CIA Suite to assess multilingual Large Language Models (LLMs). It is fine-tuned on the INTEL dataset and demonstrates strong alignment with human evaluations, outperforming zero-shot proprietary models like GPT-4 on the RECON test set. Designed for low-resource languages, Hercule enables reference-based evaluation by scoring multilingual outputs using English reference responses. It provides structured feedback with a 1-5 scale scoring rubric and highlights the effectiveness of lightweight fine-tuning techniques like LoRA.