IndNumNLI - Numerical NLI for Indian Statistical Data

About Dataset

IndNumNLI is a Natural Language Inference benchmark dataset constructed from five authoritative Indian government sources: Census of India 2011, MOSPI National Accounts Statistics 2023, RBI Annual Report 2022-23, National Family Health Survey 5 (NFHS-5 2019-21), and MoSRTH Road Transport Yearbook 2021-22. The dataset contains 92 premise-hypothesis pairs across three labels: entailment, neutral, and contradiction. Unlike existing Indic NLI benchmarks such as IndicXNLI or XTREME-XNLI, IndNumNLI specifically stress-tests quantitative and numerical reasoning rather than semantic or lexical inference. Models must verify arithmetic, compare thresholds, compute percentages, assess ratio claims, and recognise out-of-scope causal or temporal extrapolations to assign the correct label. Each example includes the premise, hypothesis, label, domain, sub-domain, reasoning category (14 types), difficulty level (easy, medium, hard), source citation, and an explicit arithmetic reasoning chain that makes label assignment fully traceable and reproducible. The dataset is split into train (64), validation (13), and test (15) subsets using stratified sampling by label.