Large Language Models

Definition

Large Language Models (LLMs) are neural networks trained on massive text datasets to predict and generate human-like text. They are the technology underlying tools like ChatGPT, Claude, and Gemini — and represent the current frontier of AI capability in language, reasoning, and code generation.

Why It Matters

LLMs represent a qualitative shift in AI capability — they are not specialized tools (like chess engines or image classifiers) but general-purpose systems that can write, reason, translate, summarize, code, and converse across an enormous range of tasks. Understanding how they work is essential for using them well and governing them responsibly.

How They Work

• Trained to predict the next token (word fragment) in a sequence, given all preceding tokens
• Scale is the key driver: more parameters + more data + more compute = more capable models
• Emergent capabilities appear at scale that were not present in smaller models — reasoning, code generation, few-shot learning
• They do not “know” facts; they pattern-match against training data, which produces both remarkable coherence and confident hallucination

Key Properties

• Hallucination: LLMs generate plausible-sounding text even when producing false information — they have no ground truth to verify against
• Context window: The amount of text they can “see” at once; beyond it, they lose access to earlier content
• Prompt sensitivity: Small changes in phrasing can dramatically change outputs — prompting is a skill, not just input
• Alignment gap: What an LLM says it will do and what it actually does can diverge

Connection to Broader AI Themes

• The jagged frontier (Co-Intelligence): LLMs are powerful in unexpected places and weak in equally unexpected ones
• The alignment problem (The Alignment Problem): Getting LLMs to reliably do what users want is technically unsolved
• Governance (The Coming Wave): LLMs are proliferating faster than regulatory frameworks can adapt

Related Concepts

• Human-AI Collaboration — LLMs are the primary tool for AI collaboration in knowledge work
• AI Ethics — LLMs encode biases from training data and can generate harmful content at scale
• AI Safety — aligning LLMs with human values is the core challenge
• Future of Work — LLMs are the technology most immediately disrupting knowledge work
• Prompt Engineering — the practice of eliciting good outputs from LLMs

Key Books

• Co-Intelligence — the most practical guide to working with LLMs
• The Alignment Problem — why making LLMs reliably safe and helpful is hard
• The Coming Wave — LLMs as a component of the broader wave requiring governance