In this special edition, Dr. Tim Scarfe, Yannic Kilcher and Dr. Keith Duggar speak with Professor Gary Marcus, Dr. Walid Saba and Connor Leahy about GPT-3.
We have all had a significant amount of time to experiment with GPT-3 and show you demos of it in use and the considerations.
Do you think GPT-3 is a step towards AGI? Answer in the comments!
00:00:00 | Connor's take on LinkedIn
00:00:47 | Show teaser
00:20:02 | Tim Introduction
00:26:55 | First look at GPT-3, python sorting
00:31:05 | Search strategy in LMs
00:38:28 | Character analogies and Melanie Mitchell
00:44:27 | Substitution cipher
00:47:21 | Database prompt
00:53:00 | Broader Impact Generation
01:02:47 | Gary Marcus Interview (Robust.AI)
01:29:11 | Connor Leahy Interview (Eleuther.AI)
01:32:29 | Connor -- Tabular data
01:33:41 | Connor -- other surprising examples?
01:34:54 | Connor -- Is interpolated stuff new?
01:37:43 | Connor -- structure of the brain / How GPT works
01:41:21 | Connor -- Why cant GPT-3 reason?
01:46:30 | Connor -- Missing information problem and ideas on our our brains work
01:54:28 | Connor -- Topology of brain/models
01:58:49 | Connor -- Hardware lottery / LSTM / Transformer
02:01:41 | Connor -- NNs are just matrix program search
02:10:32 | Connor -- Google -- information retrieval, the new paradigm, how to extract info from GPT-3, RL controller on top?
02:19:38 | Connor -- Database example / "pattern matching is Turing complete"
02:23:55 | Connor -- Did gpt3 understand?
02:26:30 | Connor -- Are the GOFAI people right?
02:27:40 | Walid Saba on GPT-3
02:30:41 | Walid -- What is understanding and pattern recognition
02:35:56 | Walid -- Chomsky would be happy
02:42:13 | Walid -- Redefining success
02:46:05 | Walid on Hinton
02:47:34 | Walid on software 3.0
02:53:11 | Keith -- We use machine learning because we cant write code to do the same thing
02:59:36 | Keith -- What is pattern recognition and understanding
03:14:06 | GPT-3 trials -- Turing Dialog
03:15:35 | GPT-3 trials -- Mary Enjoyed a Sandwich
03:16:19 | GPT-3 trials -- BBC has five offices in Germany.
03:16:55 | GPT-3 trials -- Database prompt
03:20:23 | GPT-3 trials -- Python
03:20:31 | GPT-3 trials -- Patterns
03:21:01 | GPT-3 trials -- Database again
03:25:11 | GPT-3 trials -- GPT-3 experiment -- the trophy doesn’t fit in the suitcase
03:27:32 | GPT-3 trials -- Scrambling words
03:30:41 | GPT-3 trials -- PDF cleanup example (Gwern)
03:35:03 | GPT-3 trials -- Word breaking and simple text patterns
03:37:16 | GPT-3 trials -- Typing of entities
03:38:30 | GPT-3 trials -- Basic Python append
03:39:07 | GPT-3 trials -- Automatic programming?
03:42:31 | GPT-3 trials -- Passive aggressive dialog input
03:44:39 | GPT-3 trials -- symptoms of depression
03:45:43 | GPT-3 trials -- Red shirts reasoning challenge
03:49:59 | GPT-3 trials -- Binary encoding
03:50:36 | Concluding statements from Walid, Tim and Yannic