MIT introduction to Deep Learning

I watched the first two lessons of the Introduction to Deep Learning from MIT, but I have not found those useful and in-depth as I expected. They cover an immense amount of concepts in a short time, skipping concepts that are really important to understand the topic.

In the first lesson, MIT introduction to Deep Learning, the teacher Alexander Amini explains lots of complex concepts in under an hour (the first 10 minutes are an introduction).

The topics range from the simple perceptron to overfitting and regularization.

Alexander is an excellent teacher, but the large amount of concepts explained in less than one hour does not allow him to go deep into those concepts or to show a sufficient number of practical examples.

The MIT course is great if you want context, or maybe if you already know many ML concepts and want to fill the gaps in your knowledge. But I prefer building foundations and intuition before abstraction, so I stopped watching those videos for now. Maybe I’ll return to them later.

Watching LLM videos from Andrej Karpathy now

I use ChatGPT and other LLMs (Large Language Models) almost every day, but the only things I know are that

• LLMs are trained on a big chunk of the Web
• LLMs use a NN that predicts the next token using probabilities

I also realise than I am not leveraging these surprising tools in an optimal way, and my prompting skills are pretty basic. I am not even sure what is a good prompt.

These are some questions that I ask myself:

• How do LLMs work under the hood?
• How are they trained?
• How can they answer like a human and even show personality?

I recently discovered the video series from Andrej Karpathy. He is an AI scientist who worked at Tesla and is a cofounder of OpenAI, but most notably, he’s a great teacher who created some of the best video tutorials about LLMs.

I’ll focus on Deep Dive into LLMs like ChatGPT, and will soon share my discoveries on LLMs.