large language model (LLM)
acronymprogrammingops culture
A model trained on very large amounts of text to predict what comes next, which turns out to be enough to do a great many other things.
The capability people find surprising is emergent rather than designed: nobody wrote rules for summarising or translating or writing code. Worth holding onto when reading claims about what these systems 'understand' - next-token prediction at scale is what is actually happening, and how much that explains is genuinely contested.
Also known as: llm, language model
Sources
- Vaswani et al., 'Attention Is All You Need' (2017)