Category
nlp
9 articles across 3 sub-topics
SFT for LLMs: A Practical Guide to Supervised Fine-Tuning
TLDR: Supervised fine-tuning (SFT) is the stage where a pretrained model learns task-specific response behavior from curated input-output examples. It is usually the first alignment step after pretraining and often the foundation for later RLHF. Good...
Why Embeddings Matter: Solving Key Issues in Data Representation
TLDR: Embeddings convert words (and images, users, products) into dense numerical vectors in a geometric space where semantic similarity = geometric proximity. "King - Man + Woman ≈ Queen" is not magic — it is the arithmetic property of well-trained ...
Text Decoding Strategies: Greedy, Beam Search, and Sampling
TLDR: An LLM doesn't "write" text — it generates a probability distribution over all possible next tokens and then uses a decoding strategy to pick one. Greedy, Beam Search, and Sampling are different rules for that choice. Temperature controls the c...
Prompt Engineering Guide: From Zero-Shot to Chain-of-Thought
TLDR: Prompt Engineering is the art of writing instructions that guide an LLM toward the answer you want. Zero-Shot, Few-Shot, and Chain-of-Thought are systematic techniques — not guesswork — that can dramatically improve accuracy without changing a ...
A Guide to Pre-training Large Language Models
TLDR: Pre-training is the phase where an LLM learns "Language" and "World Knowledge" by reading petabytes of text. It uses Self-Supervised Learning to predict the next word in a sentence. This creates the "Base Model" which is later fine-tuned. 📖 ...
Tokenization Explained: How LLMs Understand Text
TLDR: LLMs don't read words — they read tokens. A token is roughly 4 characters. Byte Pair Encoding (BPE) builds an efficient subword vocabulary by iteratively merging frequent character pairs. Tokenization choices directly affect cost, context limit...
Natural Language Processing (NLP): Teaching Computers to Read
TLDR: 🌟 NLP turns raw text into numbers so machines can read, understand, and generate language. The field evolved from counting words (Bag-of-Words) to contextual Transformers — each leap brings richer meaning, new capabilities, and different engin...
How Transformer Architecture Works: A Deep Dive
TLDR: The Transformer is the architecture behind every major LLM (GPT, BERT, Claude, Gemini). Its core innovation is Self-Attention — a mechanism that lets the model weigh relationships between all tokens in a sequence simultaneously, regardless of d...