GPT-3 — Language Models Scale Up

In June 2020, OpenAI released GPT-3 (Generative Pre-trained Transformer 3), a language model with 175 billion parameters — 100 times larger than its predecessor GPT-2. Trained on a vast corpus of internet text, GPT-3 demonstrated remarkable "few-shot" abilities: given just a few examples in a prompt, it could write essays, generate code, translate languages, answer questions, and even compose poetry. The model revealed that scaling neural networks produced emergent capabilities — behaviors not explicitly programmed but arising from the interaction of billions of learned parameters. GPT-3 was initially available only through an API, and its outputs amazed and unsettled the tech community in equal measure. It proved that the path to more capable AI might simply be "more compute, more data, more parameters."