BhashaBench-Legal

BhashaBench-Legal (BBL): Benchmarking AI on Indian Legal Knowledge

About Dataset

BhashaBench-Legal (BBL) is the comprehensive benchmark designed to rigorously evaluate AI models on Indian legal knowledge. Tailored for India's complex legal framework, constitutional structure, and diverse jurisprudential traditions, BBL draws from official judicial service exams, bar examinations, and legal education assessments to test models' ability to provide accurate, contextually relevant, and legally sound advice within the Indian legal system. Key Features * Languages: English and Hindi (with plans for more Indic languages) * Exams: 50+ unique legal government and institutional exams across India * Domains: 20+ legal and allied disciplines, spanning over 200 specialized topics * Questions: 24,365 rigorously validated, exam-based questions * Difficulty Levels: Easy (8,200), Medium (12,150), Hard (4,015) * Question Types: Multiple Choice, Assertion-Reasoning, Match the Column, Rearrange the Sequence, Fill in the Blanks * Focus: Practical, context-rich, jurisdiction-specific legal knowledge essential for Indian legal practice

Purpose of Dataset

Bhashabench-legal Fills A Vital Gap In Evaluating Ai Systems For India’s Intricate Legal Ecosystem One Shaped By Constitutional Complexity, Diverse Statutory Traditions, And Rapidly Evolving Jurisprudence. As Ai Begins To Influence Legal Research, Judicial Assistance, Compliance Workflows, And Public-facing Legal Advisory Tools, This Benchmark Offers The Essential Foundation For: * Assessing Ai’s Capability To Reason Within India’s Unique Legal Frameworks, Including Procedural Law, Constitutional Mandates, And Jurisdiction-specific Nuances * Developing Trustworthy, Legally Compliant, And Context-aware Ai Systems For Courts, Law Firms, Government Bodies, And Citizen-facing Platforms * Strengthening Access To Justice Through Accurate, Bilingual (And Soon Multilingual) Legal Assistance Tools * Preserving And Digitizing Authoritative Legal Knowledge Derived From Real Judicial Exams And Professional Assessments * Ensuring Safety, Reliability, And Accountability In Ai Models That May Influence Legal Decision-making Or Public Understanding