AI & Machine Learning Landscape — Practitioner Reference (2026)¶

The Three-Layer Structure of AI in 2026¶

Frontier model race — fast-moving, narrow quality gaps between top models (~2.7% lead at the top as of March 2026)
Production engineering stack — maturing: vLLM/SGLang serving, MCP-standardized agents, contextual-retrieval RAG
Classical ML core — stable: gradient boosting still wins most tabular problems

Inference cost collapse: >280-fold reduction from $20.00 to $0.07 per million tokens between Nov 2022 and Oct 2024 (GPT-3.5-equivalent quality, per Stanford HAI). This makes capability cheap and engineering discipline the binding constraint.

Frontier Model Landscape (Mid-2026)¶

No single model dominates. The top model leads by only ~2.7% (Stanford HAI AI Index, March 2026). The leading closed-weight model's edge over the top open-weight model on Chatbot Arena narrowed from 8.04% (Jan 2024) to 1.70% (Feb 2025).

Key practitioner principle: Build behind an abstraction layer (LiteLLM/OpenRouter) to avoid lock-in. Model names and rankings shift weekly.

Hosted APIs vs Open Weights — Decision Rule¶

Factor	Hosted API	Open Weights
Peak capability, zero ops	✓
Privacy, air-gap, data residency		✓
Fine-tuning control		✓
Predictable cost at sustained high QPS		✓
Fastest to start	✓

Open-weight options: DeepSeek-V4 (MIT-licensed, 1M context, 1.6T/49B active MoE), Qwen3, Meta Llama 4, Mistral, IBM Granite 4.0.

Threshold to self-host: sustained high QPS where API spend exceeds fully-loaded GPU fleet cost, or a hard data-residency requirement.

Model Selection Strategy (Staged)¶

Start with hosted frontier behind an abstraction layer. Default to strong mid-tier (Claude Sonnet-class, GPT-4.1-mini-class, Gemini Flash-class); escalate to top-tier only for hard tasks.
Implement model routing once volume justifies it. A cheap classifier sends easy queries to small/cheap models with confidence-gated escalation. RouteLLM (UC Berkeley/Anyscale/Canva, ICLR 2025) achieved 95% of GPT-4 quality using only ~26% of GPT-4 calls, with up to 85% cost reduction on MT-Bench.
Move to open weights self-hosted on vLLM/SGLang when privacy, air-gap, predictable cost at scale, or fine-tuning control becomes the binding constraint.

Build Pattern for LLM Applications¶

Prompting → RAG → Fine-tuning (in that order)

Few-shot prompting for behavioral change
RAG for dynamic/factual/auditable knowledge — use hybrid retrieval (BM25 + dense) + reranking + contextual retrieval
Fine-tune (LoRA/QLoRA) only for consistent format/style/jargon or cost reduction
GRPO/DPO only when you have verifiable rewards or preference data and a reasoning/behavior target

Reasoning Models & Test-Time Compute¶

The o1/o3/R1 paradigm: spending more inference compute on chain-of-thought "thinking tokens" is a complementary scaling axis to parameters/data.

Process Reward Models (PRMs): score intermediate reasoning steps; outperform outcome-only rewards on math (GenPRM: 1.5B model beats GPT-4o on ProcessBench via test-time scaling).

GRPO (Group Relative Policy Optimization): critic-free, group-relative RL with verifiable rewards (RLVR). Dominant post-training method for reasoning. Spawned variants DAPO, Dr. GRPO, GSPO, GMPO.

When reasoning tokens are worth it: genuine multi-step reasoning (math, code generation, legal analysis). Wasteful when wired into pipelines expecting short outputs.

Classical ML — Tabular Data¶

Gradient-boosted trees remain state-of-the-art on most tabular problems. Deep learning requires more tuning and rarely wins on tabular without genuine unstructured signal.

2025 challenger: TabPFN (transformer tabular foundation model, published in Nature) beats GBDTs on small/medium single tables. XGBoost still wins on large tables.

Practitioner default: GBDT-first (LightGBM → CatBoost → XGBoost depending on data characteristics). Only reach for deep learning with genuine unstructured signal or very large data.

Mathematical Foundations¶

Linear algebra: tensors are batched multidimensional arrays; matmul/dot products implement every linear layer and attention score; SVD underpins PCA and low-rank adaptation (LoRA literally learns low-rank ΔW). Norms drive regularization (L1 sparsity/Lasso, L2 weight decay/Ridge) and gradient clipping.

Probability: KL/JS/Wasserstein divergences anchor VAEs, GANs, and distribution matching. Cross-entropy is the default classification loss (= MLE under a categorical model). Bias–variance is operationalized by regularization and ensembling.

Optimization: backprop = reverse-mode autodiff applying chain rule over the computational graph. Deep loss landscapes are non-convex but navigable — saddle points (not bad local minima) dominate, and SGD noise helps escape them. AdamW is the safe default; cosine annealing with linear warmup is the standard LR schedule. First-order methods dominate because Hessian-based methods don't scale to billions of parameters.

Classical ML Reference¶

Regression & Classification¶

OLS with Ridge/Lasso/ElasticNet for regularized selection
GLMs (logistic, Poisson, negative binomial) for non-Gaussian targets
Gaussian Process Regression for small-data + calibrated uncertainty
Logistic regression: strong, interpretable baseline — always try it first
SVMs (kernel trick, RBF): still win on small high-dimensional data

Calibration¶

Calibration matters in production. Temperature scaling (post-hoc, single-parameter) for neural nets; isotonic/Platt for others. Brier score and reliability diagrams for diagnosis.

Unsupervised & Anomaly Detection¶

Clustering: k-means(++)/DBSCAN/HDBSCAN/GMM/spectral
Dimensionality reduction: PCA/UMAP/t-SNE (UMAP preferred for preserving global structure and speed)
Anomaly detection: isolation forest/LOF/autoencoders

Model Selection¶

Stratified/group/time-series CV to prevent leakage
Metric choice: AUC-PR for imbalanced, MCC as balanced single number, log-loss for probabilistic
Optuna (TPE/Bayesian) + Hyperband/ASHA for hyperparameter tuning
Leakage is the silent killer: target leakage, train/test contamination via preprocessing, and temporal leakage cause the most "great offline, broken online" failures

Deep Learning Foundations¶

Activations & Normalization¶

ReLU: default for CNNs
GELU/SiLU(Swish): dominate transformers (smooth, work well with normalization)
BatchNorm: CNNs
LayerNorm/RMSNorm: transformers (RMSNorm is cheaper, now standard in LLMs)
Pre-norm (LayerNorm before sublayer): dominates modern transformers for stability at depth

Initialization & Precision¶

He/Kaiming for ReLU nets; Xavier for tanh; orthogonal for RNNs
BF16: LLM-training default (wider dynamic range than FP16, no loss scaling)
FP8: production on Hopper/Blackwell (DeepSeek-V3 trained in FP8)
Mixed precision memory tools: gradient checkpointing, gradient accumulation, activation recomputation

Residual Connections¶

Enable very deep networks by giving gradients an identity path.

The Transformer & Attention¶

Architecture types: - Encoder-only (BERT): classification/embedding - Encoder-decoder (T5): seq2seq tasks - Decoder-only (GPT): dominates generative LLMs

Attention components: - Scaled dot-product attention over Q,K,V - Multi-head attention - RoPE (rotary positional encoding): now standard; YaRN/position interpolation extends context - FlashAttention (v1→v3): IO-aware, exact (not approximate), tiling attention in SRAM — mitigates O(n²) attention cost - KV cache: makes autoregressive decoding tractable - FFN uses SwiGLU

Mixture of Experts (MoE): dominant frontier-scaling lever. Sparse gating + routing + load balancing. DeepSeek-V3 activates 37B of 671B params per token. Mixtral, DeepSeek-MoE, Qwen3-MoE all use this architecture.

LLM Training & Post-Training¶

Tokenization¶

BPE/WordPiece/SentencePiece/tiktoken; larger vocabulary = shorter sequences but larger embedding tables
Chinchilla scaling (~20:1 tokens:params) guided compute-optimal training; inference-cost economics now push toward "overtrain a smaller model"

Pre-training Infrastructure¶

Curated/deduplicated/quality-filtered corpora + heavy synthetic data (Phi "textbooks" thesis)
Tensor/pipeline/data parallelism; ZeRO/DeepSpeed; FSDP; Megatron-LM

Alignment Pipeline¶

SFT (Supervised Fine-Tuning): high-quality instruction examples; "LIMA / Less is More" — a few thousand high-quality examples often beat massive noisy sets
DPO (Direct Preference Optimization): reference-model based, no explicit reward model; simple default
GRPO: critic-free, group-relative, RLVR; dominates reasoning post-training

PEFT (Parameter-Efficient Fine-Tuning)¶

LoRA: low-rank adapters; workhorses of production fine-tuning
QLoRA: 4-bit NF4 base + LoRA; enables fine-tuning large models on consumer hardware
DoRA/VeRA: refinements of LoRA

LLM Inference Optimization¶

Quantization Formats¶

Format	Use case	Notes
AWQ	Best throughput in vLLM; INT4 weight-only	<2% degradation on most tasks; noticeable on math/code/reasoning
GPTQ	Best throughput in vLLM	Similar trade-offs to AWQ
GGUF	llama.cpp/Ollama; CPU+GPU offload	Q4_K_M–Q6_K near-BF16 quality
FP8 W8A8	Near-lossless on Hopper/Blackwell	Production preferred when hardware supports it

Rule: avoid INT4 for math/code/reasoning-critical paths.

Serving Engines¶

Engine	Strength
vLLM	PagedAttention; broadest model/hardware support; V1 architecture
SGLang	RadixAttention; ~29% higher H100 throughput; up to 6.4× on prefix-heavy RAG/chat
TensorRT-LLM	Maximum NVIDIA throughput
Ollama/llama.cpp	Local deployment; CPU offload

Continuous batching + disaggregated prefill/decode (separate compute-bound prefill and bandwidth-bound decode pools) are the 2026 production patterns.

Speculative decoding (draft+verify, Medusa/EAGLE): 2–4× latency wins losslessly.

Evaluation & Benchmark Skepticism¶

The contamination problem is severe. A single model can swing 17–35 points between SWE-bench Verified and the standardized-scaffold Pro leaderboard — scaffold and harness move scores more than model swaps.

Key benchmarks: - MMLU(-Pro), GPQA Diamond: knowledge/reasoning - HumanEval/MBPP, SWE-bench Verified/Pro: coding - GSM8K/MATH: math reasoning - MMMU: multimodal - LiveBench/LiveCodeBench: contamination-resistant - Chatbot Arena Elo: human preference - RULER/HELMET: long-context - TruthfulQA/FActScore: factuality - RAGAS: RAG evaluation

Decision rule for benchmarks: - Never select on a single leaderboard number - Build a 50–200 case internal eval of your real workflows - Weight SWE-bench Pro (standardized scaffold) over Verified for coding - When a benchmark and your production results diverge sharply, trust contamination/scaffold effects over the leaderboard - LLM-as-judge is widespread but carries position/verbosity/self-preference biases

RAG Fundamentals (Quick Reference)¶

Production default (2026): hybrid search (BM25 + dense via RRF) + cross-encoder reranking + parent-child chunking.

Anthropic Contextual Retrieval (Sept 2024): prepend LLM-generated chunk summary before embedding/indexing. - Contextual Embeddings alone: 35% reduction in retrieval failure (5.7%→3.7%) - + Contextual BM25: 49% reduction (→2.9%) - + Reranking: 67% reduction (→1.9%)

RAG vs fine-tuning vs long-context: - RAG: dynamic/proprietary knowledge with citations - Fine-tuning: changing behavior/format/tone/domain style - Long-context: single-document deep reasoning where doc fits - They combine

Agent Stack & MCP¶

Model Context Protocol (MCP): Anthropic, Nov 2024; donated to Linux Foundation Dec 2025; adopted by OpenAI, Google, Microsoft, Amazon. "USB-C for AI tools." Over 97M monthly SDK downloads and 10,000 active servers at donation time.

Production failure modes: runaway loops, unbounded retries, silent context truncation. Iteration limits, cost ceilings, and termination conditions are mandatory, not polish.

Single-agent suffices for ~80% of cases. Multi-agent adds cost and non-determinism.

Framework selection: - LangGraph: production default for regulated/auditable workflows; graph state machines with checkpointing - CrewAI: fastest role-based multi-agent prototyping - Pydantic AI: typed, minimal boilerplate

Generative AI Across Modalities¶

Image¶

GANs and VAEs gave way to diffusion (DDPM→DDIM→Latent Diffusion = Stable Diffusion)
Now flow matching / rectified flow (backbone of SD3, FLUX)
Classifier-free guidance, ControlNet (pose/depth/edge conditioning), LoRA/DreamBooth personalization are standard
FLUX (Black Forest Labs): open-weight image generation leader in 2026

Video¶

Standardized on diffusion transformers (DiT) over spacetime patches (Sora, Veo, Kling, Seedance, WAN, Hunyuan)
By early 2026: 8–25s clips at native resolution with synchronized audio and plausible physics
Weaknesses: long-range temporal coherence and quadratic attention cost

Audio¶

TTS: Tacotron→FastSpeech→VITS→XTTS/F5-TTS/Kokoro; ElevenLabs leads commercial quality
ASR: Whisper dominates
Discrete audio tokens: EnCodec/DAC

Multimodal/VLMs¶

Vision encoders trend SigLIP over CLIP, with tile-based high-res
Open options: LLaVA, Qwen-VL, InternVL, Molmo, Pixtral

MLOps & Infrastructure¶

Hardware¶

Chip	Key spec	Notes
NVIDIA H100/H200 (Hopper)	80GB/141GB HBM3	Current production standard
B200/GB200 (Blackwell)	192GB HBM3e at 8TB/s	~2.5–3× H100 training; GB200 NVL72 trained Llama 3.1 405B in ~10 min on 5,120 GPUs (MLPerf)
AMD MI300X/MI325X/MI355X	192–288GB HBM3	More memory; CUDA/ROCm moat keeps NVIDIA at ~80%+ share
Google TPU v5/v6e	—	JAX/XLA affinity
Groq LPU	—	Deterministic high-throughput inference

Frameworks¶

PyTorch: dominant (eager + torch.compile/Inductor)
JAX: TPU/research
Hugging Face stack: Transformers, Diffusers, PEFT, TRL, Datasets, Accelerate — the default
Experiment tracking: W&B (research favorite), MLflow (open, ubiquitous)

Observability & Monitoring¶

LLM serving/observability: LangSmith, Langfuse, Arize Phoenix, Helicone, Braintrust, W&B Weave
Drift/monitoring: Evidently, NannyML, Arize, WhyLabs, Fiddler; PSI/KS tests; distinguish covariate/label/concept shift
Explainability: SHAP/LIME/Integrated Gradients (caution: attention weights ≠ explanations)

Advanced Architectures¶

State-Space Models (SSMs)¶

Mamba (selective state spaces, Mamba-2): linear complexity, ~5× throughput
Critical weakness: pure SSMs fail at retrieval/in-context copying — removing attention layers drops retrieval accuracy to near-zero
Hybrids win in production: Jamba (AI21), Zamba2, NVIDIA Nemotron-H, IBM Granite 4.0 — small fraction of attention layers interleaved with Mamba blocks

Long Context¶

1M+ token windows are table stakes across flagships
"Lost in the middle" and needle-in-haystack failures persist
RAG vs long-context is a cost/freshness/auditability decision, not a pure capability one

Reasoning Research¶

Test-time compute scaling via PRMs
MCTS for reasoning
Self-consistency voting
Formal-verification systems (AlphaProof/AlphaGeometry 2)

Safety, Alignment & Governance¶

Alignment Approaches¶

Constitutional AI / RLAIF (Anthropic): AI self-critique for alignment
Mechanistic interpretability: sparse autoencoders decompose superposed/polysemantic activations; circuit tracing with cross-layer transcoders lets researchers trace and intervene on causal pathways (Anthropic, open-sourced May 2025; MIT Tech Review 2026 Breakthrough Technology)
Outer alignment (specification) vs inner alignment (goal generalization)
Reward hacking and Goodhart's Law: recurring failure modes

EU AI Act Timeline (Critical Deadlines)¶

Date	What takes effect
Aug 1, 2024	In force
Feb 2, 2025	Prohibited practices + AI literacy obligations
Aug 2, 2025	GPAI (General Purpose AI) obligations
Aug 2, 2026	GPAI enforcement powers and fines; high-risk system obligations
Aug 2, 2027	Legacy-GPAI compliance deadline

Fines: up to €35M or 7% of global turnover (prohibited practices) — higher ceiling than GDPR. The voluntary GPAI Code of Practice offers a "presumption of conformity" safe harbor.

Action: If you touch EU users, classify your system now against the AI Act risk tiers. If you fine-tune a GPAI model substantially, you may become a "provider" with heavier obligations.

Fairness¶

Demographic parity vs equalized odds vs individual fairness are mutually incompatible (impossibility theorems). Tools: Fairlearn, AI Fairness 360.

Privacy¶

Differential privacy, federated learning, and defenses against membership-inference/model-inversion/poisoning attacks.

US Framework¶

NIST AI RMF (Govern/Map/Measure/Manage) is the US reference standard.

Durable Principles (Outlast Model Names)¶

Baseline-first: always establish a simple baseline before reaching for complex architectures
Data quality over architecture: garbage in, garbage out — no model overcomes bad data
Route, don't over-provision: tiered model fleets beat one expensive frontier model
RAG before fine-tune: try prompting and retrieval before committing to training
Guardrails and observability before launch: build I/O filtering, tracing, and cost monitoring into the initial design
Benchmark skepticism: build internal evals on your real workflows; never select on a single leaderboard number
Open vs closed is a cost/privacy/capability trade-off, not an ideology: evaluate quarterly as quality gaps narrow

Plugin: ai-and-data · View SKILL.md on GitHub