Orca 2-13Billion Parameter Model

About Model

Orca-2-13B is an advanced language model developed by Microsoft, fine-tuned from Meta's LLaMA-2 13B base model. This model focuses on bolstering the reasoning skills of smaller language models through training on a synthetic dataset tailored for complex reasoning tasks. It excels in single-turn interactions, effectively handling tasks like reasoning over user-provided data, reading comprehension, mathematical problem-solving, and text summarization. Evaluations indicate that Orca-2-13B outperforms baseline models of similar size and achieves performance levels comparable to models significantly larger, particularly in zero-shot reasoning scenarios. Despite its strengths, the model is not optimized for chat functionalities and lacks training involving reinforcement learning from human feedback (RLHF) or Direct Preference Optimization (DPO). Researchers interested in deploying the model for conversational purposes or specific applications are advised to undertake additional fine-tuning. Orca-2-13B is accessible under the Microsoft Research License, promoting further exploration in the field of small language model development and alignment.