Art of the Problem | The Making of ChatGPT: 35 Year History
I follow the journey that led to the explosion of Large Language Models. From Jordan's pioneering work in 1986 to today's GPT-4, this documentary traces how AI learned to talk.
2023-11-27 19:00:00 - Art of the Problem
Featuring insights from AI pioneers including Chomsky, Hofstadter, Hinton, and LeCun, exploring the revolutionary concepts that made ChatGPT possible: transformer architecture, attention mechanism, next-token prediction, and emergent capabilities. Next video following open ai's o1 model.
00:00 - Introduction
00:32 - Hofstader's thoughts on chatGPT
01:00 - Recap of supervised learning
01:55 - First paper on sequential learning
02:55 - First use of state units (RNN)
04:33 - First observation of word boundary detection
05:30 - First observation of word clustering
07:16 - First "large" language model Hinton/Sutskever
10:10 - Sentiment neuron (Ilya | OpenAI)
12:30 - Transformer explanation
15:50 - GPT-1
17:00 - GPT-2
17:55 - GPT-3
18:20 - In-context learning
19:40 - ChatGPT / GPT-4
21:10 - Tool use
23:25 - Philosophical question: what is thought?