L11: The Principles of Diffusion Models - New Course Launch
Transcript
-
Hello everyone. I'm very excited to be Hello everyone. I'm very excited to be recording this video today because uh recording this video today because uh recording this video today because uh I'm going to make an announcement for I'm going to make an announcement for I'm going to make an announcement for the launch of a new course. Before we the launch of a new course. Before we the launch of a new course. Before we discuss about the course, first let us discuss about the course, first let us discuss about the course, first let us rewind and go back to the year 2022. rewind and go back to the year 2022. rewind and go back to the year 2022. I remember this was the year where I I remember this was the year where I I remember this was the year where I truly truly truly understood the power and scope of understood the power and scope of understood the power and scope of generative AI and the main reason was generative AI and the main reason was generative AI and the main reason was that applications like midjourney and that applications like midjourney and that applications like midjourney and stable diffusion came out which could stable diffusion came out which could stable diffusion came out which could generate images which were very very generate images which were very very generate images which were very very realistic realistic realistic and I remember being completely blown and I remember being completely blown and I remember being completely blown away by the realism of the images which away by the realism of the images which away by the realism of the images which these AI applications could generate and these AI applications could generate and these AI applications could generate and that was the time where I really started that was the time where I really started that was the time where I really started taking AI generation and generative AI taking AI generation and generative AI taking AI generation and generative AI overall as a field very seriously. overall as a field very seriously. overall as a field very seriously. I I remember initially I I I did not I I remember initially I I I did not I I remember initially I I I did not focus on the technology behind focus on the technology behind
-
focus on the technology behind uh midjourney or stable diffusion but I uh midjourney or stable diffusion but I uh midjourney or stable diffusion but I was so excited to try this out that I was so excited to try this out that I was so excited to try this out that I took this to a lot of schools in India took this to a lot of schools in India took this to a lot of schools in India and uh I started showing teachers what and uh I started showing teachers what and uh I started showing teachers what you can do with applications like you can do with applications like you can do with applications like midjourney. We had a fun exercise where midjourney. We had a fun exercise where midjourney. We had a fun exercise where I used to tell the teachers that tell me I used to tell the teachers that tell me I used to tell the teachers that tell me a name of an animal, a place and an a name of an animal, a place and an a name of an animal, a place and an activity and uh let's generate an image activity and uh let's generate an image activity and uh let's generate an image using midjourney. So we used to come up using midjourney. So we used to come up using midjourney. So we used to come up with crazy prompts like monkeys playing with crazy prompts like monkeys playing with crazy prompts like monkeys playing cricket on Mars etc. And as you remember cricket on Mars etc. And as you remember cricket on Mars etc. And as you remember the smile it had on teachers faces it the smile it had on teachers faces it the smile it had on teachers faces it was really great to see. was really great to see. was really great to see. uh when I started using these models the uh when I started using these models the uh when I started using these models the word diffusion really caught my word diffusion really caught my word diffusion really caught my attention I used to wonder is this attention I used to wonder is this attention I used to wonder is this related to the diffusion which I know related to the diffusion which I know
-
related to the diffusion which I know about I know about diffusion where let's about I know about diffusion where let's about I know about diffusion where let's say I spray a perfume in one corner of say I spray a perfume in one corner of say I spray a perfume in one corner of the room the smell slowly percolates in the room the smell slowly percolates in the room the smell slowly percolates in the room and reaches to the other corner the room and reaches to the other corner the room and reaches to the other corner of the room so this process of of the room so this process of of the room so this process of percolation percolation percolation and uh slow movement of particles from and uh slow movement of particles from and uh slow movement of particles from one area to another That's diffusion. one area to another That's diffusion. one area to another That's diffusion. And I always wondered at the back of my And I always wondered at the back of my And I always wondered at the back of my mind where does the word diffusion in mind where does the word diffusion in mind where does the word diffusion in stable diffusion come from. So recently stable diffusion come from. So recently stable diffusion come from. So recently in this year I started studying more in this year I started studying more in this year I started studying more about it and I realized that there is a about it and I realized that there is a about it and I realized that there is a beautiful story behind the development beautiful story behind the development beautiful story behind the development of midjourney and stable diffusion. of midjourney and stable diffusion. of midjourney and stable diffusion. And the word diffusion actually is used And the word diffusion actually is used And the word diffusion actually is used in the same sense like we use in in in the same sense like we use in in in the same sense like we use in in physics. physics. physics. This development and journey and the This development and journey and the This development and journey and the research which has happened to develop research which has happened to develop research which has happened to develop these diffusion models have
-
these diffusion models have these diffusion models have lasted for 10 to 15 years and every lasted for 10 to 15 years and every lasted for 10 to 15 years and every development has used breakthroughs made development has used breakthroughs made development has used breakthroughs made by a previous researcher. So it has been by a previous researcher. So it has been by a previous researcher. So it has been truly an incremental journey. truly an incremental journey. truly an incremental journey. But why I was hooked onto this field But why I was hooked onto this field But why I was hooked onto this field more was that more was that more was that firstly it was a field where I could see firstly it was a field where I could see firstly it was a field where I could see mathematics being influenced by physics. mathematics being influenced by physics. mathematics being influenced by physics. So we are actually looking at the So we are actually looking at the So we are actually looking at the physical diffusion process and then physical diffusion process and then physical diffusion process and then uh using the intuition from that process uh using the intuition from that process uh using the intuition from that process to develop an AI technique which I found to develop an AI technique which I found to develop an AI technique which I found to be amazing and the literature reveals to be amazing and the literature reveals to be amazing and the literature reveals that there are three different tracks that there are three different tracks that there are three different tracks which kind of merge together in the year which kind of merge together in the year which kind of merge together in the year 2022 and that's when we get these 2022 and that's when we get these 2022 and that's when we get these brilliant models. So I wanted to brilliant models. So I wanted to brilliant models. So I wanted to understand what these three different understand what these three different understand what these three different tracks are and can we understand them tracks are and can we understand them tracks are and can we understand them from the fundamentals. from the fundamentals. from the fundamentals. So this series has been made from that
-
So this series has been made from that So this series has been made from that point of view. I want to explain point of view. I want to explain point of view. I want to explain diffusion models to you completely from diffusion models to you completely from diffusion models to you completely from scratch. And I'm giving the title to scratch. And I'm giving the title to scratch. And I'm giving the title to this series as the principles of this series as the principles of this series as the principles of diffusion models from origins to diffusion models from origins to diffusion models from origins to advances. advances. advances. And we are going to go from complete And we are going to go from complete And we are going to go from complete basics and we are going to understand basics and we are going to understand basics and we are going to understand how it has evolved over 10 15 years and how it has evolved over 10 15 years and how it has evolved over 10 15 years and why we are able to create such realistic why we are able to create such realistic why we are able to create such realistic images. Now how are mathematics and images. Now how are mathematics and images. Now how are mathematics and physics so beautifully combined and how physics so beautifully combined and how physics so beautifully combined and how they work well with each other. they work well with each other. they work well with each other. Diffusion models are not just limited to Diffusion models are not just limited to Diffusion models are not just limited to images. You can create beautiful videos, images. You can create beautiful videos, images. You can create beautiful videos, audios, protein synthesis. There are so audios, protein synthesis. There are so audios, protein synthesis. There are so many application of diffusion models many application of diffusion models many application of diffusion models that this whole field is very that this whole field is very that this whole field is very fascinating. fascinating. fascinating. Now let me quickly show you how exactly Now let me quickly show you how exactly Now let me quickly show you how exactly I have structured this course and how I I have structured this course and how I
-
I have structured this course and how I plan to teach it. It'll be very similar plan to teach it. It'll be very similar plan to teach it. It'll be very similar to Vijuara's other teaching material and to Vijuara's other teaching material and to Vijuara's other teaching material and I'll be teaching these series of I'll be teaching these series of I'll be teaching these series of lectures and it'll be uploaded on lectures and it'll be uploaded on lectures and it'll be uploaded on YouTube completely for free. Now let's YouTube completely for free. Now let's YouTube completely for free. Now let's dive in and understand the curriculum in dive in and understand the curriculum in dive in and understand the curriculum in detail. Thank you. Let us try to understand how this course Let us try to understand how this course is going to be structured. The title of is going to be structured. The title of is going to be structured. The title of this course is the principles of this course is the principles of this course is the principles of diffusion models from origins to advances. As I said before in the introduction, As I said before in the introduction, As I said before in the introduction, diffusion models has really diffusion models has really diffusion models has really revolutionized the text to image revolutionized the text to image revolutionized the text to image synthesis as they produce images which synthesis as they produce images which synthesis as they produce images which are very realistic and some examples we are very realistic and some examples we are very realistic and some examples we see are Delhi 3 and stable diffusion. see are Delhi 3 and stable diffusion. see are Delhi 3 and stable diffusion. And apart from these applications uh And apart from these applications uh And apart from these applications uh which also include midjourney, there are which also include midjourney, there are which also include midjourney, there are also other domains where diffusion also other domains where diffusion also other domains where diffusion models find models find models find a lot of applications. a lot of applications. a lot of applications. The next domain that we are going to The next domain that we are going to The next domain that we are going to discuss is that of protein
-
discuss is that of protein discuss is that of protein structure prediction. structure prediction. structure prediction. This has been a problem which has This has been a problem which has This has been a problem which has fascinated biologists for a very long fascinated biologists for a very long fascinated biologists for a very long time. And it turns out that now you can time. And it turns out that now you can time. And it turns out that now you can use diffusion models to generate 3D use diffusion models to generate 3D use diffusion models to generate 3D molecular structures and protein molecular structures and protein molecular structures and protein confirmations. confirmations. confirmations. And there are some excellent models And there are some excellent models And there are some excellent models which have been released in the last 2 which have been released in the last 2 which have been released in the last 2 three years which have diffusion models three years which have diffusion models three years which have diffusion models at the heart of um the methodology for at the heart of um the methodology for at the heart of um the methodology for predicting these protein structures and predicting these protein structures and predicting these protein structures and I find this application very I find this application very I find this application very fascinating. fascinating. Another application which is kind of a Another application which is kind of a Another application which is kind of a direct synthesis or extension of image direct synthesis or extension of image direct synthesis or extension of image generation capabilities is the ability generation capabilities is the ability generation capabilities is the ability to generate videos using AI. to generate videos using AI. to generate videos using AI. Video generation models are still Video generation models are still Video generation models are still improving but we are now seeing improving but we are now seeing improving but we are now seeing increasingly models becoming more increasingly models becoming more increasingly models becoming more effective at generating realistic effective at generating realistic effective at generating realistic videos. It started from generating 3 videos. It started from generating 3 videos. It started from generating 3 seconds, 5 seconds, 10 seconds and now seconds, 5 seconds, 10 seconds and now
-
seconds, 5 seconds, 10 seconds and now you can generate up to 5 minute videos you can generate up to 5 minute videos you can generate up to 5 minute videos using u generative AI and a lot of these using u generative AI and a lot of these using u generative AI and a lot of these models like Lumiere by Google research models like Lumiere by Google research models like Lumiere by Google research have diffusion process at the heart of have diffusion process at the heart of have diffusion process at the heart of how these models are prepared in the how these models are prepared in the how these models are prepared in the first place. first place. first place. Another very key application which I Another very key application which I Another very key application which I really want to highlight is that of really want to highlight is that of really want to highlight is that of modern robot learning. modern robot learning. modern robot learning. All of us have, you know, always been All of us have, you know, always been All of us have, you know, always been mesmerized by the possibilities of mesmerized by the possibilities of mesmerized by the possibilities of robotics. You see all these Boston robotics. You see all these Boston robotics. You see all these Boston Dynamics robots flying around doing Dynamics robots flying around doing Dynamics robots flying around doing these acrobatic stunts. But the question these acrobatic stunts. But the question these acrobatic stunts. But the question is that why haven't we seen a lot of is that why haven't we seen a lot of is that why haven't we seen a lot of these robots in our houses doing these robots in our houses doing these robots in our houses doing household chores? household chores? household chores? And there has been a lot of focus in the And there has been a lot of focus in the And there has been a lot of focus in the AI community to advance the field of AI community to advance the field of AI community to advance the field of robotics robotics robotics and diffusion has given a huge push to and diffusion has given a huge push to and diffusion has given a huge push to this advancement. Recently we are seeing
-
this advancement. Recently we are seeing this advancement. Recently we are seeing a lot of foundational robotics models a lot of foundational robotics models a lot of foundational robotics models being released which are trained on being released which are trained on being released which are trained on massive amounts of robotics trajectories massive amounts of robotics trajectories massive amounts of robotics trajectories similar to how a LLM is trained on similar to how a LLM is trained on similar to how a LLM is trained on massive amounts of web data. Similarly, massive amounts of web data. Similarly, massive amounts of web data. Similarly, a robotics foundation model is trained a robotics foundation model is trained a robotics foundation model is trained on massive amounts of robotics on massive amounts of robotics on massive amounts of robotics trajectories. And then trajectories. And then trajectories. And then you have these foundational models which you have these foundational models which you have these foundational models which can do a lot of tasks like folding can do a lot of tasks like folding can do a lot of tasks like folding clothes, pushing a block, pick and place clothes, pushing a block, pick and place clothes, pushing a block, pick and place operations etc. operations etc. operations etc. A fascinating example of this is the A fascinating example of this is the A fascinating example of this is the model pi0 which is developed by physical model pi0 which is developed by physical model pi0 which is developed by physical intelligence and they use diffusion intelligence and they use diffusion intelligence and they use diffusion models at the heart of their policy. models at the heart of their policy. models at the heart of their policy. And this is fascinating right because And this is fascinating right because And this is fascinating right because the process of diffusion which appears the process of diffusion which appears the process of diffusion which appears to be limited to physics somehow seems to be limited to physics somehow seems to be limited to physics somehow seems to have applications in a lot of these to have applications in a lot of these to have applications in a lot of these different domains. That leads us to different domains. That leads us to different domains. That leads us to wonder is there something intrinsic wonder is there something intrinsic
-
wonder is there something intrinsic which is happening in diffusion that is which is happening in diffusion that is which is happening in diffusion that is common in all these applications. But common in all these applications. But common in all these applications. But robotics is a field which has fascinated robotics is a field which has fascinated robotics is a field which has fascinated me for quite some time and diffusion me for quite some time and diffusion me for quite some time and diffusion models have you know advanced the field models have you know advanced the field models have you know advanced the field of robot policy learning significantly. of robot policy learning significantly. of robot policy learning significantly. Uh I I I I will be following the Uh I I I I will be following the Uh I I I I will be following the progress of physical intelligence very progress of physical intelligence very progress of physical intelligence very closely. I'm very impressed by the work closely. I'm very impressed by the work closely. I'm very impressed by the work uh done by their team and uh this model uh done by their team and uh this model uh done by their team and uh this model which appears to be a frontier model in which appears to be a frontier model in which appears to be a frontier model in uh in in the class of robotics uh in in the class of robotics uh in in the class of robotics foundation model is very interesting and foundation model is very interesting and foundation model is very interesting and the architecture the development process the architecture the development process the architecture the development process is also very fascinating. is also very fascinating. is also very fascinating. Another application of diffusion models Another application of diffusion models Another application of diffusion models is to generate realistic music and is to generate realistic music and is to generate realistic music and speech maintaining the temporal speech maintaining the temporal speech maintaining the temporal coherence and style. So you can imagine coherence and style. So you can imagine coherence and style. So you can imagine diffusion models taking care of not just diffusion models taking care of not just diffusion models taking care of not just images and videos but also audio
-
images and videos but also audio images and videos but also audio synthesis. Now that we have established that Now that we have established that diffusion models have a huge number of diffusion models have a huge number of diffusion models have a huge number of applications and they have a lot of applications and they have a lot of applications and they have a lot of potential. They have already potential. They have already potential. They have already revolutionized revolutionized revolutionized few industries and they might have a lot few industries and they might have a lot few industries and they might have a lot of other applications waiting to be of other applications waiting to be of other applications waiting to be uncovered. This is a field which has a uncovered. This is a field which has a uncovered. This is a field which has a lot of potential not just from the lot of potential not just from the lot of potential not just from the research side but also application side. research side but also application side. research side but also application side. So anyone who is looking to understand So anyone who is looking to understand So anyone who is looking to understand AI from a more deeper lens and develop a AI from a more deeper lens and develop a AI from a more deeper lens and develop a skill set to build AI applications skill set to build AI applications skill set to build AI applications should definitely have a very clearcut should definitely have a very clearcut should definitely have a very clearcut fundamental understanding of diffusion fundamental understanding of diffusion fundamental understanding of diffusion models in their arsenal. models in their arsenal. models in their arsenal. However, very little is known about how However, very little is known about how However, very little is known about how they work. Let's do a small they work. Let's do a small they work. Let's do a small study. What we'll do is we will go to study. What we'll do is we will go to study. What we'll do is we will go to YouTube and we'll try to understand what kind of videos are available on what kind of videos are available on diffusion models. So let's see diffusion diffusion models. So let's see diffusion diffusion models. So let's see diffusion models explained.
-
So you see there are these small videos So you see there are these small videos 15 minutes, 30 minutes, 12 minutes and 15 minutes, 30 minutes, 12 minutes and 15 minutes, 30 minutes, 12 minutes and some of them have got really good views some of them have got really good views some of them have got really good views but all of these videos are very surface but all of these videos are very surface but all of these videos are very surface level. They don't really help you level. They don't really help you level. They don't really help you understand what is happening underneath. understand what is happening underneath. understand what is happening underneath. You get a sense that okay fine I have You get a sense that okay fine I have You get a sense that okay fine I have understood diffusion but really you understood diffusion but really you understood diffusion but really you don't and my level of understanding was don't and my level of understanding was don't and my level of understanding was this two years back. I broadly unders this two years back. I broadly unders this two years back. I broadly unders understood what was happening but then I understood what was happening but then I understood what was happening but then I realized that there is a whole realized that there is a whole realized that there is a whole mystery waiting to be discovered. It's mystery waiting to be discovered. It's mystery waiting to be discovered. It's like this is just covering the tip of like this is just covering the tip of like this is just covering the tip of the iceberg whereas the iceberg is very the iceberg whereas the iceberg is very the iceberg whereas the iceberg is very huge huge huge and the rich mystery and the rich and the rich mystery and the rich and the rich mystery and the rich fundamentals are are are not presented fundamentals are are are not presented fundamentals are are are not presented anywhere. anywhere. anywhere. All of these videos are 10 minutes 15 All of these videos are 10 minutes 15 All of these videos are 10 minutes 15 minutes maximum 30 minutes long. So very minutes maximum 30 minutes long. So very
-
minutes maximum 30 minutes long. So very little is known about how they work and little is known about how they work and little is known about how they work and I think that diffusion models should be I think that diffusion models should be I think that diffusion models should be very fundamental in every AI course. very fundamental in every AI course. very fundamental in every AI course. There should be a separate course just There should be a separate course just There should be a separate course just reserved for diffusion models. So this reserved for diffusion models. So this reserved for diffusion models. So this course is designed for people who seek a course is designed for people who seek a course is designed for people who seek a very clear conceptual and mathematically very clear conceptual and mathematically very clear conceptual and mathematically grounded understanding of diffusion grounded understanding of diffusion grounded understanding of diffusion models. models. models. It clarifies the theoretical foundations It clarifies the theoretical foundations It clarifies the theoretical foundations explaining the reasoning behind their explaining the reasoning behind their explaining the reasoning behind their diverse formulations and provides a diverse formulations and provides a diverse formulations and provides a stable footing for further study and stable footing for further study and stable footing for further study and research in this rapidly evolving field. research in this rapidly evolving field. research in this rapidly evolving field. And this is a very great or accessible And this is a very great or accessible And this is a very great or accessible entry point for learners. We'll also entry point for learners. We'll also entry point for learners. We'll also make this course very practical. We will make this course very practical. We will make this course very practical. We will include a lot of practical applications include a lot of practical applications include a lot of practical applications and practical examples uh which will and practical examples uh which will and practical examples uh which will make every single concept very clear. So make every single concept very clear. So make every single concept very clear. So we'll have a very nice balance of theory we'll have a very nice balance of theory we'll have a very nice balance of theory and practicals. and practicals. and practicals. Now the reason I'm coming out with this
-
Now the reason I'm coming out with this Now the reason I'm coming out with this course right now is because I received a course right now is because I received a course right now is because I received a major boost when this book was released. major boost when this book was released. major boost when this book was released. The principles of diffusion models from The principles of diffusion models from The principles of diffusion models from origins to advances and the writers of origins to advances and the writers of origins to advances and the writers of these books are some of the authors these books are some of the authors these books are some of the authors which have themselves contributed to which have themselves contributed to which have themselves contributed to some key advances in the development of some key advances in the development of some key advances in the development of diffusion models and it's a fabulously diffusion models and it's a fabulously diffusion models and it's a fabulously written book. It's it's it's kind of written book. It's it's it's kind of written book. It's it's it's kind of long. It's around 500 pages. My aim is long. It's around 500 pages. My aim is long. It's around 500 pages. My aim is to take this book as the primary source to take this book as the primary source to take this book as the primary source of reference and uh add my own thinking of reference and uh add my own thinking of reference and uh add my own thinking practical applications and practical applications and practical applications and uh you know relate it to some real life uh you know relate it to some real life uh you know relate it to some real life examples wherever necessary. for example examples wherever necessary. for example examples wherever necessary. for example uh related to the field of robotics then uh related to the field of robotics then uh related to the field of robotics then image generation and take real life image generation and take real life image generation and take real life examples so that the theory doesn't examples so that the theory doesn't examples so that the theory doesn't become very complicated become very complicated become very complicated and one thing uh I realized after
-
and one thing uh I realized after and one thing uh I realized after reading this book and these thoughts reading this book and these thoughts reading this book and these thoughts were also circling in my mind is that were also circling in my mind is that were also circling in my mind is that uh anyone who is learning artificial uh anyone who is learning artificial uh anyone who is learning artificial intelligence and wants to go a little intelligence and wants to go a little intelligence and wants to go a little bit deep you need to start looking at bit deep you need to start looking at bit deep you need to start looking at the subject from multiple different the subject from multiple different the subject from multiple different lenses. lenses. lenses. You know looking at diffusion models You know looking at diffusion models You know looking at diffusion models from a surface level understanding is from a surface level understanding is from a surface level understanding is okay. But if you go one level deeper you okay. But if you go one level deeper you okay. But if you go one level deeper you start to understand that there is start to understand that there is start to understand that there is something very deep which is happening something very deep which is happening something very deep which is happening which is really connecting everything which is really connecting everything which is really connecting everything together. And that is what we are going together. And that is what we are going together. And that is what we are going to understand. So at the heart of this to understand. So at the heart of this to understand. So at the heart of this is something very simple but to uncover is something very simple but to uncover is something very simple but to uncover that simple fact we are going to go that simple fact we are going to go that simple fact we are going to go through a lot of material. through a lot of material. through a lot of material. So let's try to briefly understand how I So let's try to briefly understand how I So let's try to briefly understand how I have divided the curriculum according to have divided the curriculum according to
-
have divided the curriculum according to this book. Uh and I I will not go very this book. Uh and I I will not go very this book. Uh and I I will not go very theoretical in this section. So it might theoretical in this section. So it might theoretical in this section. So it might appear that I'm skimming some of the appear that I'm skimming some of the appear that I'm skimming some of the details but it's it's very deliberate. details but it's it's very deliberate. details but it's it's very deliberate. It's to not burden you with a lot of It's to not burden you with a lot of It's to not burden you with a lot of keywords right now. But at the same time keywords right now. But at the same time keywords right now. But at the same time for people looking to know whether this for people looking to know whether this for people looking to know whether this course is relevant for them. I want to course is relevant for them. I want to course is relevant for them. I want to give a very broad level understanding of give a very broad level understanding of give a very broad level understanding of what all concepts I am planning to cover what all concepts I am planning to cover what all concepts I am planning to cover as a part of this course and how is the as a part of this course and how is the as a part of this course and how is the course really structured. course really structured. course really structured. So part A of uh the course is going to So part A of uh the course is going to So part A of uh the course is going to be introduction to deep generative be introduction to deep generative be introduction to deep generative modeling. What are the fundamentals? modeling. What are the fundamentals? modeling. What are the fundamentals? What is the meaning of deep generative What is the meaning of deep generative What is the meaning of deep generative models? models? models? What is generative models itself in in What is generative models itself in in What is generative models itself in in the first place? And what is the broader
-
the first place? And what is the broader the first place? And what is the broader objective we are trying to achieve here? objective we are trying to achieve here? objective we are trying to achieve here? This is going to be very fundamental. This is going to be very fundamental. This is going to be very fundamental. And uh it it turns out that what we are And uh it it turns out that what we are And uh it it turns out that what we are trying to do is we are trying to trying to do is we are trying to trying to do is we are trying to generate an image. But generate an image. But generate an image. But the way we do that is we try to the way we do that is we try to the way we do that is we try to look at what would be the distribution look at what would be the distribution look at what would be the distribution which is which is which is used to generate all these images. used to generate all these images. used to generate all these images. we predict the distribution and then we we predict the distribution and then we we predict the distribution and then we sample from the distribution to predict sample from the distribution to predict sample from the distribution to predict a new image. So I will explain all of a new image. So I will explain all of a new image. So I will explain all of this once we start the course but that's this once we start the course but that's this once we start the course but that's in a in a broad sense this is what we in a in a broad sense this is what we in a in a broad sense this is what we are trying to do and the field of deep are trying to do and the field of deep are trying to do and the field of deep generative modeling has evolved quite a generative modeling has evolved quite a generative modeling has evolved quite a lot in the last 10 years. As you can see lot in the last 10 years. As you can see
-
lot in the last 10 years. As you can see one of the first methods actually came one of the first methods actually came one of the first methods actually came 40 years back which was energy based 40 years back which was energy based 40 years back which was energy based methods. Since then we have made a lot methods. Since then we have made a lot methods. Since then we have made a lot of progress. Variation autoenccoders, of progress. Variation autoenccoders, of progress. Variation autoenccoders, normalizing flows, diffusion normalizing flows, diffusion normalizing flows, diffusion probabilistic models, neural oDs, probabilistic models, neural oDs, probabilistic models, neural oDs, noise condition score network, dnoising noise condition score network, dnoising noise condition score network, dnoising diffusion probabilistic models. This is diffusion probabilistic models. This is diffusion probabilistic models. This is very interesting. And then flow very interesting. And then flow very interesting. And then flow matching. matching. matching. All of these are different techniques All of these are different techniques All of these are different techniques which have given rise or expanded the which have given rise or expanded the which have given rise or expanded the breadth the the breadth of this field of breadth the the breadth of this field of breadth the the breadth of this field of deep generative modeling. deep generative modeling. deep generative modeling. Now how are we going to understand all Now how are we going to understand all Now how are we going to understand all these concepts? For example, let's take these concepts? For example, let's take these concepts? For example, let's take pick variation autoenccoders. pick variation autoenccoders. pick variation autoenccoders. What will be our strategy to understand What will be our strategy to understand What will be our strategy to understand these concepts? Let's try to understand these concepts? Let's try to understand these concepts? Let's try to understand that. that. that. So in part B, we will actually touch So in part B, we will actually touch So in part B, we will actually touch upon the core perspectives on diffusion upon the core perspectives on diffusion upon the core perspectives on diffusion models and this is why in in this part models and this is why in in this part models and this is why in in this part we will cover all of these different
-
we will cover all of these different we will cover all of these different techniques individually. techniques individually. techniques individually. So broadly speaking there are three So broadly speaking there are three So broadly speaking there are three views which I mentioned at the start of views which I mentioned at the start of views which I mentioned at the start of the video which converge very nicely. the video which converge very nicely. the video which converge very nicely. The variational view, scorebased view The variational view, scorebased view The variational view, scorebased view and flow-based view. I I don't want you and flow-based view. I I don't want you and flow-based view. I I don't want you to focus on these terminologies a lot to focus on these terminologies a lot to focus on these terminologies a lot but but but for example let's take an example of for example let's take an example of for example let's take an example of variational autoenccoder. It falls variational autoenccoder. It falls variational autoenccoder. It falls within the variational view and we will within the variational view and we will within the variational view and we will understand the variational autoenccoder understand the variational autoenccoder understand the variational autoenccoder from the complete basics. How exactly is from the complete basics. How exactly is from the complete basics. How exactly is it formulated? The mathematics behind it it formulated? The mathematics behind it it formulated? The mathematics behind it and then finally we will see a practical and then finally we will see a practical and then finally we will see a practical example of where it is implemented in example of where it is implemented in example of where it is implemented in practice. Then we'll move on to the next practice. Then we'll move on to the next practice. Then we'll move on to the next view etc. So every single subsection of view etc. So every single subsection of view etc. So every single subsection of this course is going to be devoted to this course is going to be devoted to this course is going to be devoted to one view. First we'll start with one view. First we'll start with one view. First we'll start with variational view then we will cover variational view then we will cover
-
variational view then we will cover score based view. Then we will cover score based view. Then we will cover score based view. Then we will cover flow-based view and then after that we flow-based view and then after that we flow-based view and then after that we will see how these views are coming will see how these views are coming will see how these views are coming together. For example, if you see this together. For example, if you see this together. For example, if you see this robotics application which I mentioned robotics application which I mentioned robotics application which I mentioned pi0 pi 0 is built on a policy which uses pi0 pi 0 is built on a policy which uses pi0 pi 0 is built on a policy which uses this flow-based view this flow-based view this flow-based view and if you look at midjourney and if you look at midjourney and if you look at midjourney midjourney is built on midjourney is built on midjourney is built on a policy which uses variational view. So a policy which uses variational view. So a policy which uses variational view. So variational view and flowbased view are variational view and flowbased view are variational view and flowbased view are very common but everything started from very common but everything started from very common but everything started from score based view. So we need to look at score based view. So we need to look at score based view. So we need to look at that as well so that we get a very that as well so that we get a very that as well so that we get a very complete picture of complete picture of complete picture of how the different different perspectives how the different different perspectives how the different different perspectives on diffusion models are and how they on diffusion models are and how they on diffusion models are and how they work together with each other. work together with each other. work together with each other. So this is how we are going to So this is how we are going to So this is how we are going to understand each lecture. And then understand each lecture. And then understand each lecture. And then finally in part three we are going to finally in part three we are going to
-
finally in part three we are going to understand how to sample from these understand how to sample from these understand how to sample from these diffusion models. How to steer the diffusion models. How to steer the diffusion models. How to steer the generation using classifier guidance, generation using classifier guidance, generation using classifier guidance, classifier free guidance etc. We are classifier free guidance etc. We are classifier free guidance etc. We are going to look at a lot of practical going to look at a lot of practical going to look at a lot of practical examples as as you can see in this examples as as you can see in this examples as as you can see in this figure. figure. figure. This is an example where you know This is an example where you know This is an example where you know Chinese letters are generated using the Chinese letters are generated using the Chinese letters are generated using the process of diffusion and it's it's very process of diffusion and it's it's very process of diffusion and it's it's very exciting right you can see that exciting right you can see that exciting right you can see that when the green bar is almost zero it's when the green bar is almost zero it's when the green bar is almost zero it's complete noise but we are generating complete noise but we are generating complete noise but we are generating something from noise. So initially you something from noise. So initially you something from noise. So initially you start with noise and then slowly you start with noise and then slowly you start with noise and then slowly you start the process of denoising and then start the process of denoising and then start the process of denoising and then finally you get what you want. [snorts] finally you get what you want. [snorts] finally you get what you want. [snorts] So there is there is a feeling of some So there is there is a feeling of some So there is there is a feeling of some magic happening here and that is what magic happening here and that is what magic happening here and that is what actually attracted me to this field. I actually attracted me to this field. I
-
actually attracted me to this field. I want I I I wanted to study it in in in want I I I wanted to study it in in in want I I I wanted to study it in in in more depth. more depth. more depth. People who have taken Vijuara's courses People who have taken Vijuara's courses People who have taken Vijuara's courses before uh know our style of teaching is before uh know our style of teaching is before uh know our style of teaching is completely from the basics. We do not completely from the basics. We do not completely from the basics. We do not assume any prior knowledge and we try to assume any prior knowledge and we try to assume any prior knowledge and we try to impart the intuition to you which is impart the intuition to you which is impart the intuition to you which is going to be crucial for you to develop a going to be crucial for you to develop a going to be crucial for you to develop a very deep understanding and uh enjoyment very deep understanding and uh enjoyment very deep understanding and uh enjoyment of the subject because we believe that of the subject because we believe that of the subject because we believe that if you start enjoying the subject of AI if you start enjoying the subject of AI if you start enjoying the subject of AI uh the the sky is the limit for you then uh the the sky is the limit for you then uh the the sky is the limit for you then you can take it up ahead in a lot of you can take it up ahead in a lot of you can take it up ahead in a lot of different ways. different ways. different ways. a bit about your course instructor. U a bit about your course instructor. U a bit about your course instructor. U I have taught multiple courses before I have taught multiple courses before I have taught multiple courses before including reasoning LLM from scratch and including reasoning LLM from scratch and including reasoning LLM from scratch and hands-on reinforcement learning. The hands-on reinforcement learning. The hands-on reinforcement learning. The reinforcement learning course just reinforcement learning course just
-
reinforcement learning course just finished a while back and uh after that finished a while back and uh after that finished a while back and uh after that I have come up with this diffusion I have come up with this diffusion I have come up with this diffusion series. Both these courses have received series. Both these courses have received series. Both these courses have received very good traction and every single day very good traction and every single day very good traction and every single day I see people either commenting or I see people either commenting or I see people either commenting or reaching out to us personally regarding reaching out to us personally regarding reaching out to us personally regarding how this course has helped them. how this course has helped them. how this course has helped them. So this is how we are going to uh teach So this is how we are going to uh teach So this is how we are going to uh teach you this course also in the same manner you this course also in the same manner you this course also in the same manner and my teaching style is very simple. I and my teaching style is very simple. I and my teaching style is very simple. I focus on three things. First is the focus on three things. First is the focus on three things. First is the fundamentals and then assume no prior fundamentals and then assume no prior fundamentals and then assume no prior knowledge and mix theory and practical. knowledge and mix theory and practical. knowledge and mix theory and practical. Why the fundamentals are important is Why the fundamentals are important is Why the fundamentals are important is because you can look at these YouTube because you can look at these YouTube because you can look at these YouTube videos and you can understand broad videos and you can understand broad videos and you can understand broad level what's happening but level what's happening but level what's happening but you know when you have understood you know when you have understood you know when you have understood something from the very basics and that something from the very basics and that something from the very basics and that feeling you will not get by looking at feeling you will not get by looking at
-
feeling you will not get by looking at these concepts from a very surface level these concepts from a very surface level these concepts from a very surface level to get that you need to understand the to get that you need to understand the to get that you need to understand the mathematics and the physics behind these mathematics and the physics behind these mathematics and the physics behind these concepts as I explained in the concepts as I explained in the concepts as I explained in the introduction the many of the concepts introduction the many of the concepts introduction the many of the concepts are actually inspired from physics which are actually inspired from physics which are actually inspired from physics which is also very interesting. is also very interesting. is also very interesting. So this is how we are planning to So this is how we are planning to So this is how we are planning to conduct this course and uh I'm very conduct this course and uh I'm very conduct this course and uh I'm very excited to teach this course to all of excited to teach this course to all of excited to teach this course to all of you. I'm planning to release every you. I'm planning to release every you. I'm planning to release every lecture either once in a week or once in lecture either once in a week or once in lecture either once in a week or once in two weeks. two weeks. two weeks. Thank you very much everyone and I look Thank you very much everyone and I look Thank you very much everyone and I look forward to seeing you in this course.