AI4Bharat- LLaMA Prometheus Model

About Model

Llama-Prometheus is an advanced English evaluation model introduced as part of the CIA Suite to assess the performance of multilingual Large Language Models (LLMs). It is fine-tuned on the Feedback-Collection dataset using the same setup as Prometheus 2, but with Llama-3.1-8B-Instruct as its base model. Llama-Prometheus provides structured evaluation using a reference-guided direct assessment method, requiring evaluation instructions, a response, a scoring rubric, and a reference answer. It ensures accurate benchmarking of LLMs with detailed scoring criteria.