ICL-Prompt-Sensitivity-Risk-Dataset
Simulated few-shot accuracy variance for 2800 in-context-learning configurations across five India-relevant tasks, four example-selection methods, prompt-order optimization, calibration, and formatting consistency, grounded in the foundational and 2021-2023 ICL robustness literature. Forward-looking task predicts prompt fragility from just a few cheap evaluation probes.
About Dataset
Simulates how few-shot accuracy swings by tens of points purely from example order, selection, and formatting choices. 2800 configs across five tasks (three India-relevant: vernacular sentiment, agri-query intent, grievance routing), four selection methods, order/calibration/format flags, shot count. Ten sources (2020–2024, weighted 2021–2023), including GPT-3, Calibrate Before Use, Fantastically Ordered Prompts, Rethinking the Role of Demonstrations, and the prompt-format-sensitivity paper.
Purpose of Dataset
Lets Teams Designing A Few-shot Prompt Estimate Fragility From A Few Cheap Probes Before Spending Api Budget On A Full Variance Sweep.
Dataset Metadata
Attribution 4.0 International (CC BY- 4.0)
India
Science, Technology and Research
Jai Mali
Structured
NA
NA
N.A.
08/08/26 12:44:40
Open
Hosted
Synthetic
Manually Curated
Activity Overview
0
0 
7.63 MB 
4
Tags
- Large Language Model
License Control
Attribution 4.0 International (CC BY- 4.0)
ICL-Prompt-Sensitivity-Risk-Dataset.json ( 7.63 MB )
To preview this file, you need to be a registered user. Please complete the registration process to gain access and continue viewing the content.
Data Quality Score Beta
Version Control
Version 1(7.63 MB)
- JAI MALI·2 day(s) ago
ICL-Prompt-Sensitivity-Risk-Dataset.json
© 2026 - Copyright AIKosh. All rights reserved.