Odia Monolingual News Corpus

A monolingual Odia news corpus containing news articles collected from online Odia news websites to support natural language processing.

About Dataset

The Odia Monolingual News Corpus is a large-scale collection of Odia-language news articles gathered from publicly accessible online news portals. Developed to address the scarcity of digital resources for the Odia language, the corpus provides diverse textual data spanning multiple news domains and writing styles. The dataset is intended to support natural language processing. The website from which the articles were extracted - Sambad, Dharitri, Samaja, Prameya, Pragativadi, Samaya. By providing an openly available monolingual corpus, the dataset contributes to the development of language resources and AI technologies for the Odia language. The citation of the dataset - Sahoo, S. K. (2021). OdiaMonolingualNewsCorpus. Kaggle. https://doi.org/10.34740/KAGGLE/DSV/2885507. This dataset was identified and facilitated for onboarding as part of the Dataset Onboarding Support Team (DOST) initiative led by CivicDataLab (CDL), partnering with the Gates Foundation in collaboration with BHASHINI. CivicDataLab provided support for dataset discovery, validation, metadata preparation and onboarding facilitation. All dataset ownership and intellectual property rights remain with the original author(s).

Purpose of Dataset

The Purpose Of This Dataset Is To Support The Development Of Language Technologies For The Odia Language By Providing A Large Monolingual News Corpus. It Can Be Used For Language Modeling, Text Classification, Text Mining, Information Retrieval, Text Summarization, Named Entity Recognition, Machine Translation Support, Corpus Linguistics, And Other Natural Language Processing Applications. It Can Also Be Used For Sentence Auto-completion, Masked Token Prediction, Questioning And Answering.

Dataset Metadata

License

Attribution 4.0 International (CC BY- 4.0)

Geographical coverage

Odisha

Sector

Science, Technology and Research

Author

Soumendra Kumar Sahoo

Source Organisation

Digital India BHASHINI Division

Uploaded by

Nikil Augustine

Data Quality Score (Beta)

Dataset type

Structured

Frequency

Time Granularity

Static

Year range

N.A.

Date & Time

07/07/26 11:39:38

Visibility

Open

Hosted / Redirected

Redirected

Data Type

Hybrid

If Redirection which source

Kaggle

Data Collection Method

The Dataset Was Prepared By Automatically Collecting News Articles From Odia News Websites Using The Scrapy Web Scraping Framework. Individual Spiders Were Developed For Each News Portal By Analyzing The Website Structure And Extracting Article Content, Titles, Publication Information, And Other Relevant Text. The Scraped Data Was Cleaned, Structured, And Stored In Json Lines (.Jl) Format To Create A Machine-readable Monolingual Corpus Suitable For Natural Language Processing And Language Modeling Tasks.