6 Deep Learning handbook
7 Blitz into GANs
(Adapated from this)
7.1 What is a GAN?
GANs can create images that look like photographs of human faces, even though the faces don’t belong to any real person.
7.2 How does it work?
If we build a space with 116412 (N) axes, each face will be a point in that space. A celebrity-face probability distribution function \(P(X)\) would map each face to a non-negative real number so that the sum of all these numbers for all faces equals 1.
7.3 The face space
7.4 Definition of the problem
8 Large Language Models (ChatGPT and friends)
8.1 Disclaimer
- I am not a NLP expert
- I am not a RL expert
- I am not a psychologist
8.2 Large Language Models
8.3 Transformer
Transformers are “CNN” for sequences (text, audio, etc.):
- Encoding the word “it” in encoder
#5(the top encoder in the stack) - Attention mechanism focus on “The Animal”
\(\Rightarrow\) baked a part of its representation into the encoding of “it”.
As by essence, Deep Learning is fit for complex multi-scale problems, layers in DL are “learning” to capture different levels of dependencies in the data.
The power of transformers in DL is to leverage this “multi-scale learning” to sequences data, e.g. not only the words in sentences, but also the sentences in paragraphs, and the paragraphs in documents and so on.
8.4 (Very) Big Transformers…

8.5 … are based on (very) Large Text Corpora
Those models are trained on very large text corpora, e.g. the entire English Wikipedia, or the entire English Web, and so on.
It is then used as a “foundation” for other tasks, e.g. translation, summarization, question answering, etc.
8.6 RLHF
The idea is to use a large language model as a “foundation” for a reinforcement learning phase with (a lot of) humans. An agent (a tranfer learning on top of the base model) is trained to maximize the reward given by a human.
\(\Rightarrow\) The human is asked to give a reward to the agent, based on the quality of its answer. The agent is then trained to maximize the reward given by the human. And so on. So the
8.7 Censorship
“Toxic” content is filtered by low-wage workers, on a large scale.
8.8 Main considerations
- Likelihood != Truth
- Appeals to our need to “humanize” things (Eliza effect)
- No structure, “rules” or “knowledge” of the world
- Easy to detect, easy to circumvent (but by humans)
8.9 AutoGPT
Multi-agents system plugged on top on ChatGPT.
The next best thing? Not quite.
One Reddit user claims that, given a budget of $100 to spend within a server instance, Auto-GPT made a wiki page on cats, exploited a flaw in the instance to gain admin-level access and took over the Python environment in which it was running — and then “killed” itself.
There’s also ChaosGPT, a modified version of Auto-GPT tasked with goals like “destroy humanity” and “establish global dominance.” Unsurprisingly, ChaosGPT hasn’t come close to bringing about the robot apocalypse — but it has tweeted rather unflatteringly about humankind.



