SansTib - Sanskrit-Classical Tibetan Parallel Corpus

The Dataset Was Constructed Using Sanskrit Texts From The Gretil Collection And Classical Tibetan Texts From The Acip Kangyur And Tengyur Collections. Sanskrit Texts Were Preprocessed By Removing Headers And Html Markup, Segmenting Text Using Punctuation, And Applying Sandhi And Compound Splitting. Tibetan Texts Were Segmented Using Punctuation, With Punctuation And Folio Markers Removed. Word And Phrase Pairs Extracted From The Uma Dictionary, The Yogācārabhūmi Index, And The Mahāvyutpatti Dictionary Were Combined To Support Coarse Sentence Alignment Using The Yasa Aligner. A Multilingual Sbert Model Was Trained Using The Coarse Alignments And Bilingual Dictionary Data, Followed By Hnsw-based Retrieval Of Additional Text Pairs. Final Sentence Alignment Was Performed Using Vecalign And Repeated Twice. Three Manually Aligned Gold-standard Datasets Were Created To Evaluate Alignment Quality, And The Aligned Corpus Was Subsequently Integrated Into The Buddhanexus Database.