SKT-ST-X-0-3B

About Model

# 🚀 SKT-ST-X-0-3B: Sovereign Indian Intelligence **A Compact, High-Efficiency Mixture of Experts (MoE) Language Model** ### 🌟 Overview **SKT-ST-X-0-3B** is a state-of-the-art Small Language Model (SLM) engineered by **SKT AI LABS, India**. Designed for maximum performance on a minimal footprint, it utilizes advanced **Mixtral-inspired MoE stability techniques** to deliver intelligent, lightning-fast responses. | Feature | Specification | |---|---| | **Architecture** | Mixture of Experts (MoE) | | **Total Params** | ~3 Billion | | **Active Params** | ~1.1 Billion (2 expert/token) | | **Context Window** | 8K Tokens | | **Training Data** | 40B Tokens (SKT-OMNI-CORPUS-2T) | ### 🧠 Why SKT-ST-X-0-3B? * **Bilingual Mastery:** Optimized for both English and Hindi. * **Efficiency:** Designed for edge deployment with low VRAM requirements via 4-bit quantization. * **Reasoning Power:** Robust performance in logic, coding, and complex problem-solving. * **Sovereign AI:** Built in India, for the world, by **SKT AI LABS** . ### 🛠️ Quick Setup ```python from transformers import AutoModelForCausalLM, AutoTokenizer import torch # Load with 4-bit quantization for extreme efficiency model = AutoModelForCausalLM.from_pretrained( "sKT-Ai-Labs/SKT-ST-X-0-3B", device_map="auto", load_in_4bit=True ) tokenizer = AutoTokenizer.from_pretrained("sKT-Ai-Labs/SKT-ST-X-0-3B") ``` ### 📊 Performance Metrics (MTEB) * **Amazon Polarity Classification:** 56.88% Accuracy * **Amazon Counterfactual Classification:** 39.35% F1-Score ### 📜 License Released under the **Apache-2.0 License**. ### 📧 Contact For partnerships or technical inquiries, reach out to **support@sktailabs.in**.