Neural networks in one paragraph
A neural network is a stack of simple functions — linear transforms followed by non-linearities — trained to map inputs to outputs by adjusting millions of parameters. Deep learning is the practice of using very deep versions of these networks, plus a lot of data and compute.
Why depth changes everything
Shallow models can only combine features linearly. Deep models learn hierarchical representations — pixels become edges, edges become shapes, shapes become objects. This is why they work well for language, vision and audio: those domains are naturally hierarchical.
The architectures you should know
- Convolutional networks (CNNs). The workhorse of computer vision.
- Recurrent networks (RNNs, LSTMs). Once the default for sequences, now largely displaced.
- Transformers. The architecture behind ChatGPT, Claude, Gemini and almost every current frontier system.
If you learn only one, learn the Transformer.
The three levers
Training a deep model comes down to three things: data, compute, and objective. Almost every headline breakthrough in the last five years has come from scaling one of them.
Practical advice
- Use pre-trained models. Training from scratch is rarely the right call.
- Fine-tuning and LoRA are the modern default.
- Retrieval-augmented generation beats fine-tuning for most enterprise problems.
Deep learning is deep in name only — the concepts fit on one page. The difficulty is in the engineering.