MiniGPT

About Model

MyGPT Assistant (38M) A custom 38-million parameter Large Language Model built entirely from scratch in PyTorch. This model was created as an educational project to understand the complete lifecycle of LLM development—from defining the transformer architecture and dataset processing, all the way to exporting weights into GGUF format for Ollama. Training Details This model was trained in two distinct stages: Pre-training: Trained on a 50MB subset of the TinyStories dataset. This allowed the model to learn basic English grammar, syntax, and structural logic. Supervised Fine-Tuning (SFT): Fine-tuned on a subset of the Alpaca-Cleaned dataset to learn the User: / Assistant: conversational format and basic instruction following. Architecture Details: Parameters: ~38 Million Context Window: 256 tokens Layers: 6 Transformer Blocks Attention Heads: 6 Embedding Dimension: 384 Disclaimer Because this model is extremely small (38M parameters) and trained on children’s stories, it is meant purely for educational purposes and demonstrations of the LLM pipeline. It has very limited real-world knowledge and will heavily hallucinate facts.