cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 00:
02 | All right, class. Class, shall we begin? All right, class. Class, shall we begin?
Shall we begin? Shall we begin? Shall we begin?
Welcome to 10799. Welcome to 10799. Welcome to 10799.
My name is Kelly… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 01:
24 | 2025 will be the Italian brain rods. Uh
so for those of you guys who don't know so for those of you guys who don't know so for those of you guys who don't know
this is this is this is
this is bombado … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 02:
23 | the same underlying technology and that the same underlying technology and that
is is is
hello and that is we got stuck. Oh and that is and that is we got stuck. Oh and that is
this uh diffuser um and… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 03:
15 | implement these things um like in Python implement these things um like in Python
which is the language that we use every which is the language that we use every which is the language that we use ever… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 04:
06 | probably need to first answer what is a probably need to first answer what is a
good generation uh image generation good generation uh image generation good generation uh image generation
model right … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 04:
52 | they like have a lot of like weird
artifacts? Um so you probably don't artifacts? Um so you probably don't artifacts? Um so you probably don't
really understand what what weird really understand what … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 05:
34 | or can I use another image to you know or can I use another image to you know
to specify what I want to generate? For to specify what I want to generate? For to specify what I want to generate? For
ex… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 06:
09 | model. Can we do that? Um so that's the
second aspect. And then the third aspect second aspect. And then the third aspect second aspect. And then the third aspect
is basically how fast is the generati… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 06:
45 | I reduce war clock time with the same
amount uh same number of network amount uh same number of network amount uh same number of network
evaluation which means that like you evaluation which means tha… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 07:
31 | beginnings village and where you you you beginnings village and where you you you
get to like fail pretty fast and safely get to like fail pretty fast and safely get to like fail pretty fast and safel… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 08:
07 | bosses throughout the way. And uh, these bosses throughout the way. And uh, these
ones are not going to kill you and ones are not going to kill you and ones are not going to kill you and
they're just … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 08:
52 | environment try to get yourself familiar environment try to get yourself familiar
with GPU training and also just build with GPU training and also just build with GPU training and also just build
two … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 09:
42 | Okay. And uh at the end of the class uh Okay. And uh at the end of the class uh
which is the last week uh of class, it's which is the last week uh of class, it's which is the last week uh of class, it… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 10:
19 | to actually present your poster in class
you know tell your classmate what you you know tell your classmate what you you know tell your classmate what you
did and then the third thing is you did and t… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 10:
58 | right. Uh so a couple of things to
remind you guys is that uh this class is remind you guys is that uh this class is remind you guys is that uh this class is
completely AI friendly and open completely… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 11:
53 | people. However, uh because we're so people. However, uh because we're so
flexible on our you know just like open flexible on our you know just like open flexible on our you know just like open
source… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 12:
33 | like it's totally fine and yeah just in like it's totally fine and yeah just in
general do not plagiarize. It's you know general do not plagiarize. It's you know general do not plagiarize. It's you kn… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 13:
20 | basically um
so uh myself and the TAs are going to so uh myself and the TAs are going to so uh myself and the TAs are going to
start hosting office hours um next week. start hosting office hours um ne… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 14:
00 | they do know um you know they they have
done research in terms of well Douglas done research in terms of well Douglas done research in terms of well Douglas
has done research in terms of flow has done… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 14:
44 | probably your classmate are going to
figure like a lot of things out on your figure like a lot of things out on your figure like a lot of things out on your
own through ex own through ex own through e… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 15:
27 | by anything I mean you're not allowed to by anything I mean you're not allowed to
use your laptop at all. So just going to use your laptop at all. So just going to use your laptop at all. So just goin… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 16:
07 | website. Basically, they're just like website. Basically, they're just like
easy conceptual questions. Uh, and they easy conceptual questions. Uh, and they easy conceptual questions. Uh, and they
shou… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 16:
54 | with me uh to collect feedback on your
learning experience in this class. And learning experience in this class. And learning experience in this class. And
uh yeah basically it's just going to be uh y… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 17:
31 | talk about like how people have been
trying to improve upon those uh basic trying to improve upon those uh basic trying to improve upon those uh basic
algorithms throughout the years. Uh and algorithm… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 18:
17 | then week six is going to be we're going then week six is going to be we're going
to be talking about probably a lot of to be talking about probably a lot of to be talking about probably a lot of
you … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 19:
06 | post it on there. Um and uh post it on there. Um and uh
the gray scope, which I have not add the gray scope, which I have not add the gray scope, which I have not add
anyone to, by the way, so don't p… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 19:
56 | have announcement thank you um forum uh have announcement thank you um forum uh
uh uh discussions you and feel free to uh uh discussions you and feel free to uh uh discussions you and feel free to
sha… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 20:
39 | constant basis? Raise your hand if you
do. All right. Amazing. All right. Amazing.
Actually uh a lot of people do. For Actually uh a lot of people do. For Actually uh a lot of people do. For
those of … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 21:
30 | free. Uh so in total you should be able
to get around like 280 GP hours on L4DS. to get around like 280 GP hours on L4DS. to get around like 280 GP hours on L4DS.
Um and that should be plenty and shou… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 22:
19 | AWS is, but it's just like another uh AWS is, but it's just like another uh
cloud comput service that is very very cloud comput service that is very very cloud comput service that is very very
popular… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 23:
05 | you are a MLDD student or LTI student or
probably any SCES student at this point probably any SCES student at this point probably any SCES student at this point
uh you can probably get access to the u… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 23:
45 | any lab clusters, uh then uh I have
listed a bunch of uh freeish and listed a bunch of uh freeish and listed a bunch of uh freeish and
cheapish uh cloud comput service that cheapish uh cloud comput se… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 24:
40 | you're not using it. Do not go bankrupt you're not using it. Do not go bankrupt
uh when you're training models. This is uh when you're training models. This is uh when you're training models. This is
… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 25:
23 | finalize the list of registered student
and uh yeah but like whether or not you and uh yeah but like whether or not you and uh yeah but like whether or not you
are going to be able to register please … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 26:
02 | And uh oh yeah, one other thing that I
need to mention there are a couple of need to mention there are a couple of need to mention there are a couple of
people already done that but which you people a… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 26:
43 | streamed on Zoom like we have seen here.
Thank you Jacob for doing that. Um so Thank you Jacob for doing that. Um so Thank you Jacob for doing that. Um so
yeah so basically what you can do here yeah s… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 27:
27 | Ponupto I don't know uh like the the the Ponupto I don't know uh like the the the
thing that I also shared on Discord and thing that I also shared on Discord and thing that I also shared on Discord an… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 28:
09 | uh
feel free to read it especially this one feel free to read it especially this one feel free to read it especially this one
book uh so this book is written by a book uh so this book is written by a … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 28:
50 | going to be the same um yeah so feel
free to read this book this is amazing free to read this book this is amazing free to read this book this is amazing
uh and besides this class if you want to uh an… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 29:
37 | and diffusion and flow matching models. and diffusion and flow matching models.
Uh so nothing is important, nothing else Uh so nothing is important, nothing else Uh so nothing is important, nothing el… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 30:
14 | in the class if you don't want to. Uh I in the class if you don't want to. Uh I
know I don't like taking classes uh when know I don't like taking classes uh when know I don't like taking classes uh wh… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 30:
51 | some of my survey you know if you need some of my survey you know if you need
some extra credits. Um but anyway um some extra credits. Um but anyway um some extra credits. Um but anyway um
yeah but ju… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 32:
01 | given Y. Is that what you said? Okay.
So, but what is P? >> I mean, I guess I could model X. >> I mean, I guess I could model X.
>> Okay. Yeah. Yeah. Yeah. Pretty much. >> Okay. Yeah. Yeah. Yeah. Pret… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 33:
04 | of Y right what are they? So X and Y's of Y right what are they? So X and Y's
are random variables. So basically just are random variables. So basically just are random variables. So basically just
th… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 33:
47 | know which one is more unfortunate
actually. Um but uh basically we usually actually. Um but uh basically we usually actually. Um but uh basically we usually
denote it as P of X. denote it as P of X. … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 34:
38 | consider X or only consider Y. And the consider X or only consider Y. And the
conditional distribution is when you are conditional distribution is when you are conditional distribution is when you are… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 35:
21 | just do a little bit of algebra you're just do a little bit of algebra you're
going to be able to get this very nice going to be able to get this very nice going to be able to get this very nice
formu… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 36:
11 | y tells you nothing about x or knowing x
tells you nothing about y. So you uh tells you nothing about y. So you uh tells you nothing about y. So you uh
just do not change your posterior. Oh just do no… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 37:
01 | person distribution we have like the person distribution we have like the
average rate as our uh parameter. average rate as our uh parameter. average rate as our uh parameter.
Uh basically it's just l… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 37:
57 | we call the probability of the data we call the probability of the data
given our learn the model parameter as given our learn the model parameter as given our learn the model parameter as
the likelih… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 39:
28 | Richard Feman. Uh apparently people
found it on his blackboard after he died found it on his blackboard after he died found it on his blackboard after he died
or something. I don't know if that after … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 40:
06 | bunch of images of a bedroom uh and then
the label of the bedroom is whether or the label of the bedroom is whether or the label of the bedroom is whether or
not it's a luxurious bedroom. I've been no… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 40:
51 | we're trying to learn is what a bedroom
should look like or what a luxurious or should look like or what a luxurious or should look like or what a luxurious or
what a cheap I guess uh bedroom should w… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 41:
36 | should be? So basically do we believe
this is a gausian? Do we believe this is this is a gausian? Do we believe this is this is a gausian? Do we believe this is
a mixture of gausian? Uh what kind of a… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 42:
26 | necessarily able to uh be able to tell necessarily able to uh be able to tell
like how likely are your uh like is some like how likely are your uh like is some like how likely are your uh like is some… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 43:
06 | labeling or like uh annotations and
stuff like that. Okay, stuff like that. Okay, stuff like that. Okay,
cool. So how to train your generated cool. So how to train your generated cool. So how to train… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 44:
05 | if my um parameters are correct then the if my um parameters are correct then the
real data point should have high real data point should have high real data point should have high
likelihood. Uh whic… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 44:
52 | image is composed by a bunch of um
pixels pixels pixels
uh then um by the probability chain rule uh then um by the probability chain rule uh then um by the probability chain rule
which just means that… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 45:
39 | able to get this thing that is very nice
where the log likely the conditional log where the log likely the conditional log where the log likely the conditional log
likelihood of your case element is o… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 46:
23 | LLMs. Yes, we got LM already. Yeah, LLMs. Yes, we got LM already. Yeah,
super easy. This is why people love it. super easy. This is why people love it. super easy. This is why people love it.
Um, yeah… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 47:
07 | we cannot see directly. Uh so for
example how a person look is largely example how a person look is largely example how a person look is largely
determined by their genes which is not determined by th… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 47:
52 | such a beautiful paper. I was like oh my
god this this thing is like who who god this this thing is like who who god this this thing is like who who
thought of this thing? This is like thought of this… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 48:
37 | maximizing the likelihood of the data we maximizing the likelihood of the data we
also need to do another thing which is also need to do another thing which is also need to do another thing which is
t… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 49:
19 | two distributions. So we're trying to two distributions. So we're trying to
minimize the difference between the minimize the difference between the minimize the difference between the
distribution of … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 50:
04 | measure it by like so all the divergence
that you output should be greater than that you output should be greater than that you output should be greater than
zero and it should only be equal greater z… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 50:
53 | kind of the intuition. Uh but basically
what's going to happen here is that what's going to happen here is that what's going to happen here is that
we're going to be using the KL we're going to be usi… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 51:
41 | when you're trying to decode it given when you're trying to decode it given
unz and then the other thing is what we unz and then the other thing is what we unz and then the other thing is what we
call… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 52:
29 | a expectation of this thing and then
like By bas rule again you are going to like By bas rule again you are going to like By bas rule again you are going to
be able to get from the log px is be able t… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 53:
16 | part is the the elbow
the elbow. The second part is the kale the elbow. The second part is the kale the elbow. The second part is the kale
divergence that we're trying to uh divergence that we're tryi… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 54:
07 | distribution uh like then it's just distribution uh like then it's just
going to be like the the the the going to be like the the the the going to be like the the the the
integral of the the joint uh … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 55:
03 | >> yeah because the because log is concave.
Yeah, that's right. Uh so yeah, Jensen Yeah, that's right. Uh so yeah, Jensen Yeah, that's right. Uh so yeah, Jensen
inequality here and we get an inequalit… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 55:
51 | about today. It's called uh Catch Me If about today. It's called uh Catch Me If
You Can. Uh basically, I don't know if You Can. Uh basically, I don't know if You Can. Uh basically, I don't know if
you… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 56:
27 | he was wearing a suit or something like he was wearing a suit or something like
that and people were laughing me hah and that and people were laughing me hah and that and people were laughing me hah a… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 57:
08 | but because he's getting like really
good at like faking things so he's able good at like faking things so he's able good at like faking things so he's able
to like fool the detective in into to like … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 57:
45 | uh the model that we're going to be
talking about. So you have a generator talking about. So you have a generator talking about. So you have a generator
uh and then you have a discriminator. uh and th… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 58:
28 | network or GAN. Uh this was also
developed in I guess 2014. So you see developed in I guess 2014. So you see developed in I guess 2014. So you see
how like a decade ago I don't know how like a decade … |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 59:
02 | first just like sample something um from
a easy to sample distribution such as a easy to sample distribution such as a easy to sample distribution such as
like gausian and then you have a like gausian… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 59:
48 | that uh basically this is just a regular
you know classification loss right like you know classification loss right like you know classification loss right like
binary classification law for the for b… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 60:
59 | something from it I guess why does the
well actually this is not exactly the well actually this is not exactly the well actually this is not exactly the
case we're going to be talking about how case w… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 61:
37 | we don't even have P of Z given X in our
final loss. Actually our final loss is final loss. Actually our final loss is final loss. Actually our final loss is
this thing. So our final loss is this this… |
cmu-10799-diffusion-flow |
L4: CMU 10799 S26: Diffusion & Flow Matching - Lecture 1 - Basics of Probabilistic & Generative Modeling | 62:
30 | understand how diffusion works. Uh so in understand how diffusion works. Uh so in
uh general next class we're going to be uh general next class we're going to be uh general next class we're going to b… |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 00:
03 | To really understand about how this To really understand about how this
video links to image diffusion, I video links to image diffusion, I video links to image diffusion, I
believe that we first need… |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 00:
49 | two. So this is my vector.
That's how you represent this vector. That's how you represent this vector. That's how you represent this vector.
But how would you represent an entire But how would you rep… |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 01:
31 | this is 128x 128 pixels, I will arrange
these 128x 128 pixels in a in a these 128x 128 pixels in a in a these 128x 128 pixels in a in a
u column. So imagine that you have this u column. So imagine tha… |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 02:
23 | somewhere here in a pixel space. This is
another image. It may live somewhere another image. It may live somewhere another image. It may live somewhere
here in pixel space. So pixel space is here in p… |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 03:
17 | you really need to understand.
How is the concept of probability How is the concept of probability How is the concept of probability
distribution linked to that of an image distribution linked to that… |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 04:
03 | and this is what I would have explained
to you if chat GPT or Gemini did not to you if chat GPT or Gemini did not to you if chat GPT or Gemini did not
exist but now let me actually take you exist but … |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 06:
01 | Let's see. So the reason I'm showing Let's see. So the reason I'm showing
this to you is because I want this this to you is because I want this this to you is because I want this
learning material to … |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 06:
46 | right? It wouldn't even look like a cat
or a tree. It would look like random or a tree. It would look like random or a tree. It would look like random
noise. So 99% of points in the pixel noise. So 99… |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 07:
43 | where all Chinese symbols are living. where all Chinese symbols are living.
That's the probability distribution of That's the probability distribution of That's the probability distribution of
this im… |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 08:
29 | higher probability of finding the
Chinese symbol. Right? So now if you Chinese symbol. Right? So now if you Chinese symbol. Right? So now if you
take a look at this 3D plot, right? If I take a look at… |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 09:
19 | pixels. Pixels are converted into flat
vectors. Those are represented in higher vectors. Those are represented in higher vectors. Those are represented in higher
dimensional space. dimensional space. … |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 10:
13 | saying that we are finding this this
probability distribution. If you ask if probability distribution. If you ask if probability distribution. If you ask if
you go to chat GPT right now or let's you g… |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 11:
26 | finding the true probability
distribution or getting as close to it distribution or getting as close to it distribution or getting as close to it
as possible. Right? So as possible. Right? So as possi… |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 12:
25 | Right? So if I the red line is my true Right? So if I the red line is my true
probability distribution probability distribution probability distribution
and the purple line the purple line is and the … |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 13:
22 | definitely do look like Chinese symbols definitely do look like Chinese symbols
because this purple probability because this purple probability because this purple probability
distribution is close to… |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 14:
16 | sampled that purple probability
distribution. distribution.
Sampled from that purple probability Sampled from that purple probability Sampled from that purple probability
distribution and created this… |
diffusion-lm-vizuara |
L4: Lecture 3: Generative AI through a probabilistic lens | 15:
26 | underlying probability distribution. Let
me write it here. Purpose Purpose of a diffusion model is to find Purpose of a diffusion model is to find
the underlying probability distribution.
That is the … |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 00:
03 | Hello everyone and welcome to the first Hello everyone and welcome to the first
lecture of the course principles of lecture of the course principles of lecture of the course principles of
diffusion mo… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 01:
40 | the heights of maybe a few friends of
yours and uh you have noted them down in yours and uh you have noted them down in yours and uh you have noted them down in
your diary. your diary. your diary.
So … |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 02:
47 | your school and uh you want to make a your school and uh you want to make a
guess a reasonable guess about what guess a reasonable guess about what guess a reasonable guess about what
would be the hei… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 03:
50 | will be that I think the new student
will have uh you know a height of maybe will have uh you know a height of maybe will have uh you know a height of maybe
5' 5 in. 5' 5 in. 5' 5 in.
So your predicti… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 05:
15 | are very less to be seen.
Now if we have this distribution with Now if we have this distribution with Now if we have this distribution with
us, we can easily say that us, we can easily say that us, we… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 06:
29 | curve is the true distribution.
So this is an example of a decent match. So this is an example of a decent match. So this is an example of a decent match.
I can see that the model distribution I can s… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 07:
24 | height of the new student what I see is
that I'll make a guess that okay fine that I'll make a guess that okay fine that I'll make a guess that okay fine
the new student will have a height of the new … |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 08:
24 | Uh true distribution. So true
distribution is something which is uh in distribution is something which is uh in distribution is something which is uh in
in supervised learning you're trying to in supe… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 09:
34 | models are trained to do. They are
trained so that the predicted trained so that the predicted trained so that the predicted
distribution matches as close as distribution matches as close as distribut… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 10:
35 | see something visually it stays in our
mind much longer than reading a text or mind much longer than reading a text or mind much longer than reading a text or
hearing from someone. So let's look at he… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 12:
11 | So visually if you just keep this So visually if you just keep this
uh video in mind and uh video in mind and uh video in mind and
you saw how the initial curve slowly you saw how the initial curve sl… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 13:
37 | probably not thought of it as a case probably not thought of it as a case
study which comes under the domain of study which comes under the domain of study which comes under the domain of
deep generat… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 15:
00 | and the deep generative model outputs a
trained neural network that distribution.
So there are three main points here. The So there are three main points here. The So there are three main points here.… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 16:
03 | So there are two main goals of deep
generative modeling. generative modeling. generative modeling.
The first goal is that we always aim for The first goal is that we always aim for The first goal is t… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 17:
04 | generative modeling. Realistic
generation and controllable generation. generation and controllable generation. generation and controllable generation.
Now let's move to the goal of deep Now let's move… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 18:
09 | named as generative AI. But where does
the name generative really come from? It the name generative really come from? It the name generative really come from? It
it comes from the development of it co… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 19:
10 | to generate new images of cats in this to generate new images of cats in this
case. Okay. So now let's formally look at a Okay. So now let's formally look at a
more complex example more complex exampl… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 20:
21 | gives gives that to us beforehand.
We get a small sneak peek into the true We get a small sneak peek into the true We get a small sneak peek into the true
distribution through the data samples distrib… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 21:
24 | represent a probability distribution.
So orange contour is the true So orange contour is the true So orange contour is the true
probability distribution and the black probability distribution and the … |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 22:
33 | they developed they developed
models which are trained to approximate models which are trained to approximate models which are trained to approximate
the true data distribution of cats and the true da… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 23:
30 | whenever we look at a cat in our mind we
know that this is a cat because we have know that this is a cat because we have know that this is a cat because we have
formed some assumption of the formed so… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 24:
36 | if we take two numbers I can clearly see
that the the discrepancy is the that the the discrepancy is the that the the discrepancy is the
difference between the two numbers. difference between the two … |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 25:
38 | screen. I'm initially going to just run
this video and then um I'm going to this video and then um I'm going to this video and then um I'm going to
explain different parts of it. Okay. So let's try to… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 28:
10 | difference is a great start but it
doesn't capture everything. doesn't capture everything. doesn't capture everything.
For example, if we take the absolute For example, if we take the absolute For exa… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 29:
13 | between the true distribution and the
model distribution is actually less than model distribution is actually less than model distribution is actually less than
one. which means that our model has one… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 30:
13 | So we take the log of the ratio for all So we take the log of the ratio for all
these different points and then just add these different points and then just add these different points and then just a… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 31:
16 | into account by multiplying by a factor
which is dependent on how likely the which is dependent on how likely the which is dependent on how likely the
case is under the true distribution. case is unde… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 32:
24 | distribution with a model distribution
Q. It is zero when P and Q are identical Q. It is zero when P and Q are identical Q. It is zero when P and Q are identical
and grows as Q becomes worse. Especial… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 33:
33 | how do we start to train these deep
generative models. So let's understand generative models. So let's understand generative models. So let's understand
that. objectives of objectives of
deep generati… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 35:
07 | should always add up to one. should always add up to one.
Take an example of a coin toss. Take an example of a coin toss. Take an example of a coin toss.
Probability of heads is 0.5. Probability Proba… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 36:
05 | of these operations will make sure that of these operations will make sure that
the output is positive. A very common the output is positive. A very common the output is positive. A very common
choice… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 37:
04 | you divide all of these numbers by 90.
So you get Alice, Bob and Charlie as 44, So you get Alice, Bob and Charlie as 44, So you get Alice, Bob and Charlie as 44,
33 and 22. And these numbers add up 33… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 38:
10 | finding the scores for every single
sample in this contour is going to be sample in this contour is going to be sample in this contour is going to be
challenging because we do not even know challengin… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 39:
11 | uh we are now going to briefly discuss uh we are now going to briefly discuss
the different families of deep the different families of deep the different families of deep
generative models. generative… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 40:
17 | more probable and higher energy to data
points which are less probable. points which are less probable. points which are less probable.
So imagine that we collect the exam So imagine that we collect t… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 41:
17 | Now what happens is that
we want a model that can understand this we want a model that can understand this we want a model that can understand this
pattern. So energy based models do not pattern. So e… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 42:
15 | numbers around 80 have a lower energy.
numbers around 60 have a lower energy. numbers around 60 have a lower energy. numbers around 60 have a lower energy.
Numbers around 30 which are not very Numbers… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 43:
12 | should receive a higher score. Once we
understand this simple rule, the understand this simple rule, the understand this simple rule, the
objective of energy based models is to objective of energy bas… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 44:
48 | So energy based models uh
was the first starting point for deep was the first starting point for deep was the first starting point for deep
generative modeling. generative modeling.
Then we moved on t… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 46:
02 | After this context is seen by the LLM, After this context is seen by the LLM,
it predicts the next token it predicts the next token it predicts the next token
based on the token which gets the based o… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 47:
01 | know take a look at this video first know take a look at this video first
which explains variational autoenccoders which explains variational autoenccoders which explains variational autoenccoders
at … |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 47:
59 | image flowing through a autoenccoder you
can see that the latin space the image can see that the latin space the image can see that the latin space the image
is blurred because we are reducing the is … |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 48:
54 | which is devoted exactly on this. The
next prominent milestone is called as next prominent milestone is called as next prominent milestone is called as
normalizing flows. normalizing flows. normalizin… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 50:
01 | some simple distribution and you twist
it, bend it, turn it to reach the true it, bend it, turn it to reach the true it, bend it, turn it to reach the true
distribution which is exactly what the distr… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 51:
24 | investigate investigate
the difference between generated art and the difference between generated art and the difference between generated art and
deep fix. We basically have to identify We basically … |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 52:
45 | there is a generator and a
discriminator. The generator discriminator. The generator discriminator. The generator
generates images and the role of the generates images and the role of the generates im… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 54:
19 | not be able to recognize this as a fake not be able to recognize this as a fake
image. image. image.
So see here real image detected the the So see here real image detected the the So see here real im… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 55:
15 | student heights. But we only had a few student heights. But we only had a few
data samples. So we had to use those data samples. So we had to use those data samples. So we had to use those
data sample… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 56:
10 | distribution mainly we looked at finding
the normalizing factor is very difficult the normalizing factor is very difficult the normalizing factor is very difficult
and uh that's why we need more and u… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 57:
04 | families uh illustrate complimentary
strategies for modeling complex strategies for modeling complex strategies for modeling complex
distribution and they also provide distribution and they also provi… |
diffusion-principles-vizuara |
L10: Lecture 1 - Deep Generative Modeling | Principles of Diffusion Models | 57:
56 | look at very practical examples as as we
move along. move along. move along.
Thank you very much everyone and uh I Thank you very much everyone and uh I Thank you very much everyone and uh I
hope this… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 00:
05 | Hello everyone and welcome to the second Hello everyone and welcome to the second
lecture of the course principles of lecture of the course principles of lecture of the course principles of
diffusion … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 01:
17 | from scratch. from scratch.
And this lecture is And this lecture is And this lecture is
going to be very important for the going to be very important for the going to be very important for the
subsequ… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 02:
19 | with the understanding of VAS. with the understanding of VAS.
So first we'll take a simple example. So first we'll take a simple example. So first we'll take a simple example.
Imagine that you have co… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 03:
19 | in cursive. Some of us
u don't write in cursive. So there are a u don't write in cursive. So there are a u don't write in cursive. So there are a
lot of different ways in which people lot of different… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 04:
26 | Then another time you press a button.
Another time you get another hello. This Another time you get another hello. This Another time you get another hello. This
time it might be some different student… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 05:
35 | such that or generate a machine such
that the samples drawn from that machine that the samples drawn from that machine that the samples drawn from that machine
are from the probability distribution ar… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 06:
45 | factors that determine the style of the factors that determine the style of the
handwriting. handwriting. handwriting.
So each student handwriting depends on So each student handwriting depends on So … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 08:
03 | So my first intuition is that okay fine So my first intuition is that okay fine
my first job is to figure out all these my first job is to figure out all these my first job is to figure out all these
… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 09:
02 | final shape of the handwriting. final shape of the handwriting.
So once you capture the secret recipe So once you capture the secret recipe So once you capture the secret recipe
for for for
a student … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 09:
57 | machine looks as follows. machine looks as follows.
Secret recipe is given as an input to Secret recipe is given as an input to Secret recipe is given as an input to
the machine. The machine takes thi… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 10:
52 | interesting, right? It it it looks like
okay fine this this it it makes a lot of okay fine this this it it makes a lot of okay fine this this it it makes a lot of
sense. [snorts] sense. [snorts] sense… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 11:
53 | now um in the rest of this lecture we We now um in the rest of this lecture we We
are going to assume that are going to assume that are going to assume that
for the handwriting example that we took fo… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 12:
54 | not know these factors beforehand. That
is why they are called as latent is why they are called as latent is why they are called as latent
variables. variables. variables.
So So
intuitively we think t… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 13:
51 | y-axis is represented by the symbol zed
2. Zed1 stands for slantness of the 2. Zed1 stands for slantness of the 2. Zed1 stands for slantness of the
handwriting and zed2 stands for neatness handwriting… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 14:
48 | below it's it's not neat.
So I have made some spelling mistake So I have made some spelling mistake So I have made some spelling mistake
here. But what I want to show is I want here. But what I want t… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 15:
33 | mapping which I was talking about. For mapping which I was talking about. For
example, if you pick a point here, you example, if you pick a point here, you example, if you pick a point here, you
know … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 16:
31 | in your class. in your class.
So this is also called as the So this is also called as the So this is also called as the
distribution in the latent space of the distribution in the latent space of the … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 17:
37 | typical handwriting typical handwriting
let's say it is divided into 28x 28 let's say it is divided into 28x 28 let's say it is divided into 28x 28
pixels. pixels. pixels.
So there are 784 numbers rig… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 18:
39 | latent space is has a mean of zero and a latent space is has a mean of zero and a
standard deviation of one or any other standard deviation of one or any other standard deviation of one or any other
v… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 19:
27 | So for now what we have seen is that we
have this general architecture for the have this general architecture for the have this general architecture for the
machine which takes the secret recipe as ma… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 20:
34 | another word for this machine which is
called as decoder. called as decoder. called as decoder.
Let us quickly summarize what all we Let us quickly summarize what all we Let us quickly summarize what … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 21:
44 | vary. So here you can see that as the
yellow dot moves around the Latin space, yellow dot moves around the Latin space, yellow dot moves around the Latin space,
the handwriting on the right hand side … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 22:
44 | next. So far we have used the decoder to
generate samples from the latent generate samples from the latent generate samples from the latent
variables. But what is this decoder variables. But what is t… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 23:
55 | we create a latent space with the latent
variables. So just as before we assume variables. So just as before we assume variables. So just as before we assume
that there are two latent variables zed th… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 24:
54 | know. [snorts]
Now let's say we have chosen a point in Now let's say we have chosen a point in Now let's say we have chosen a point in
the Latin space which corresponds to the the Latin space which co… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 25:
56 | we can represent digit 5 as a bunch of we can represent digit 5 as a bunch of
pixels pixels pixels
which are divided as 28 into 28 pixels where some pixels have a value of where some pixels have a val… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 27:
08 | five, if you closely look at it, it it
does look like a five. It's it's it's does look like a five. It's it's it's does look like a five. It's it's it's
almost something like this. Okay. So it looks l… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 28:
08 | is that with this approach we will get a
fixed shape of five every time but we fixed shape of five every time but we fixed shape of five every time but we
will not get variations of it and what will n… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 29:
10 | restricted to zero and one. You are open
to the possibility of to the possibility of to the possibility of
a color which appears in between white a color which appears in between white a color which a… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 30:
08 | the decoder does not give a single value
but for every pixel the decoder gives but for every pixel the decoder gives but for every pixel the decoder gives
the mean the mean the mean
and the standard d… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 31:
13 | deterministic approach only.
So now let us So now let us So now let us
have a simple example to understand how have a simple example to understand how have a simple example to understand how
does the … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 32:
16 | So the decoder is nothing but a neural
network which is trained to take inputs network which is trained to take inputs network which is trained to take inputs
from the latin space and generate from th… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 33:
11 | somewhere towards the right, which means
that it has to be a bit bright and the that it has to be a bit bright and the that it has to be a bit bright and the
pixel value is high. Bright pixels have pi… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 34:
10 | now we have covered now we have covered
one part of the story which explains the one part of the story which explains the one part of the story which explains the
variational autoenccoder. variational… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 35:
10 | part, you will understand that we have
made a major assumption. made a major assumption. made a major assumption.
Remember when we talked about the Remember when we talked about the Remember when we t… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 36:
14 | recipe and then you need to map all
these handwriting styles to the secret these handwriting styles to the secret these handwriting styles to the secret
recipe and make all those points and recipe and… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 37:
13 | the latin space to access to generate the latin space to access to generate
digit five. One option is to access all digit five. One option is to access all digit five. One option is to access all
poss… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 38:
04 | Wouldn't it be better if we knew which
part of the Latin space to access for part of the Latin space to access for part of the Latin space to access for
the type of image we want to generate? the type… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 39:
00 | decoder.
So the overall architecture looks like So the overall architecture looks like So the overall architecture looks like
this. this. this.
First I get let's say you go to any First I get let's sa… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 40:
00 | this architecture is very similar to a
plain autoenccoder where you have an plain autoenccoder where you have an plain autoenccoder where you have an
encoder which compresses the image into encoder wh… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 41:
03 | a dis or or a region which is the most a dis or or a region which is the most
probable region of finding the digit probable region of finding the digit probable region of finding the digit
five instea… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 42:
01 | we are going to look at a nice visual we are going to look at a nice visual
description to understand this much description to understand this much description to understand this much
better. better. … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 43:
12 | individual points. So 99% of these
points, they they produce garbage. So points, they they produce garbage. So points, they they produce garbage. So
your your space your latent space is not your your … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 44:
14 | regions which tell you that this is the
region corresponding to the dog. This is region corresponding to the dog. This is region corresponding to the dog. This is
the region corresponding to the cat e… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 45:
08 | space which is continuous and the space which is continuous and the
garbage gaps are now filled with garbage gaps are now filled with garbage gaps are now filled with
distributions. So now you can saf… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 46:
07 | So the two stories put together form the
variational autoenccoder. variational autoenccoder.
We first looked at the decoder where we We first looked at the decoder where we We first looked at the deco… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 47:
03 | the areas in the latin space? How do you
know which levers to pull in the know which levers to pull in the know which levers to pull in the
decoder? This is given to you by the decoder? This is given … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 48:
08 | uh styles of the students when they uh styles of the students when they
wrote the word hello or the handwritten wrote the word hello or the handwritten wrote the word hello or the handwritten
digits a… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 49:
09 | going to give you the latent space
distribution given the image. So that's distribution given the image. So that's distribution given the image. So that's
why you see the notation kind of flipped why … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 50:
09 | the encoder like this and the decoder the encoder like this and the decoder
going from a smaller line here to a going from a smaller line here to a going from a smaller line here to a
bigger line. In … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 51:
16 | these variational autoenccoders trained.
How do you train these to generate How do you train these to generate How do you train these to generate
images? Let's say you give a sample of images? Let's s… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 52:
05 | and then sample from it. So we are going
to take a real example in the next class to take a real example in the next class to take a real example in the next class
where you will see exactly how deep … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 52:
56 | is not deterministic but rather it is is not deterministic but rather it is
probabilistic. probabilistic. probabilistic.
And finally in your mind I want you to And finally in your mind I want you to A… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 53:
48 | image which is generated at the end.
This is all that is happening in the This is all that is happening in the This is all that is happening in the
variation autoenccoders. variation autoenccoders. va… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 55:
04 | tells you the hidden factors of
variation behind that image. variation behind that image. variation behind that image.
[snorts] The machine tells you the hidden factors The machine tells you the hidde… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 56:
10 | can lie. So it appears to be a very can lie. So it appears to be a very
smart and a clever machine. smart and a clever machine. smart and a clever machine.
Okay. Now you have these hidden factors Okay… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 57:
11 | the final image should be as close as
possible to the original image. So you possible to the original image. So you possible to the original image. So you
can see that there has to be some kind can se… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 58:
10 | encode and the second neural network is
the machine which learns to decode. the machine which learns to decode. the machine which learns to decode.
So somehow we have to train the So somehow we have t… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 59:
20 | a dot like this and you pass it to the a dot like this and you pass it to the
decoder and you get a value of zero decoder and you get a value of zero decoder and you get a value of zero
which which me… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 60:
07 | our predicted distribution which should
match as close as possible to the true match as close as possible to the true match as close as possible to the true
distribution which we do not really distrib… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 61:
19 | points in the latin space which is also
given in this formula. given in this formula. given in this formula.
I I don't want to stress too much on how I I don't want to stress too much on how I I don't… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 62:
21 | the visualization before where we saw
that it's like finding or that it's like finding or that it's like finding or
sampling for from a very giant ocean and sampling for from a very giant ocean and sa… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 63:
17 | that point being real?
And this actually does not even make use And this actually does not even make use And this actually does not even make use
of the encoder. This is all decoder of the encoder. Th… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 64:
26 | Now the question is that what is the Now the question is that what is the
elbow objective? [snorts] elbow objective? [snorts] elbow objective? [snorts]
Well to understand this let's understand Well to… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 65:
25 | I will not get very mathematical here
and I will not focus too much on the and I will not focus too much on the and I will not focus too much on the
derivation but I will provide a link in derivation … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 66:
24 | So this essentially says that
whatever image you're looking at the end whatever image you're looking at the end whatever image you're looking at the end
is that image is that image is that image
uh as… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 67:
33 | in any fashion but
in more realistic settings the in more realistic settings the in more realistic settings the
distribution is more likely to be distribution is more likely to be distribution is more… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 68:
30 | will be average in terms of the will be average in terms of the
slantness, in terms of the neatness etc. slantness, in terms of the neatness etc. slantness, in terms of the neatness etc.
So we are So … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 69:
32 | how likely is the reconstructed
um output. um output. um output.
What is the probability of that? And we What is the probability of that? And we What is the probability of that? And we
want to maximiz… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 70:
35 | our encoder distribution to stay as
close as possible to a gshian which close as possible to a gshian which close as possible to a gshian which
comes from our human intuition that comes from our human… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 71:
38 | want to compress the earth cat into
something which captures the essence of something which captures the essence of something which captures the essence of
the cat. We want to send a secret recipe the… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 72:
30 | objective and we try to maximize the
elbow. elbow. elbow.
So since the true objective is always So since the true objective is always So since the true objective is always
greater than the elbow, maxi… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 73:
27 | want to move our code to the yellow
distribution. We want to move it as distribution. We want to move it as distribution. We want to move it as
close as possible to the normal close as possible to the… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 74:
23 | what remains and it makes sense from an what remains and it makes sense from an
intuitive standpoint because later intuitive standpoint because later intuitive standpoint because later
people forget t… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 75:
20 | going to come together very nicely in
this last practical example. Our task is this last practical example. Our task is this last practical example. Our task is
to train a variation autoenccoder to to… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 76:
21 | hidden factors which we have absolutely hidden factors which we have absolutely
no clue what they are but the encoder is no clue what they are but the encoder is no clue what they are but the encoder … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 77:
13 | So remember our decoder setup looks like
this. It's it it needs to take the this. It's it it needs to take the this. It's it it needs to take the
sample from the latent space and then it sample from t… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 78:
10 | So it should take the latent vector and So it should take the latent vector and
it should generate the reconstructed it should generate the reconstructed it should generate the reconstructed
image. Ri… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 79:
07 | first machine do? The first machine
actually does the reverse. It takes the actually does the reverse. It takes the actually does the reverse. It takes the
image as the input, the real image, and imag… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 80:
13 | region which has the mu of.5 comma.5 and region which has the mu of.5 comma.5 and
the sigma of 2a 2. Right? So you need the sigma of 2a 2. Right? So you need the sigma of 2a 2. Right? So you need
two … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 81:
01 | the GitHub repo in the description the GitHub repo in the description
section. section. section.
[snorts] [snorts]
Okay. So overall the encoder decoder Okay. So overall the encoder decoder Okay. So ov… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 81:
56 | So this is our entire encoder decoder So this is our entire encoder decoder
architecture whatever we discussed as architecture whatever we discussed as architecture whatever we discussed as
this machi… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 82:
51 | This compares every pixel of the input
with the output. If the original pixel with the output. If the original pixel with the output. If the original pixel
is black and the VA predicts white, the is b… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 83:
46 | zero and a standard deviation of one. To
ensure that the mean is zero, we add a ensure that the mean is zero, we add a ensure that the mean is zero, we add a
penalty if the mean deviates from zero. pe… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 84:
40 | because log of a small value is a big uh
number and there is a negative sign number and there is a negative sign number and there is a negative sign
associated with it. So this will always associated … |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 85:
34 | the uh description. But first I just
want to want to want to
describe everything that we have describe everything that we have describe everything that we have
conceptually described in a mathematical… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 86:
26 | for longer duration maybe we'll get an for longer duration maybe we'll get an
even better distribution which is even better distribution which is even better distribution which is
centered at zero and… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 87:
21 | this training proceeds and this is the this training proceeds and this is the
quality of the reconstructions because quality of the reconstructions because quality of the reconstructions because
we ha… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 88:
15 | latin space uh you see that the it's
it's kind of blur in some cases which it's kind of blur in some cases which it's kind of blur in some cases which
we'll come to in in in some time but you we'll co… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 89:
05 | is that it it produces blurry outputs
and there is a very nice reason for it. and there is a very nice reason for it. and there is a very nice reason for it.
I will share the link in the description I… |
diffusion-principles-vizuara |
L9: Lecture 2 - Variational Autoencoders Explained From Scratch | Principles of Diffusion Models | 89:
57 | applications I have seen and uh there
are encoders and distribute and and are encoders and distribute and and are encoders and distribute and and
decoders right so you also have decoders right so you … |