cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 00:
03 | Cool. Hello everyone. Oh, it's not Cool. Hello everyone. Oh, it's not
significantly fewer. I guess it's significantly fewer. I guess it's significantly fewer. I guess it's
slightly significant. Okay, … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 00:
49 | actually takes time. So I I I I mean it
it takes at least a couple of hours uh it takes at least a couple of hours uh it takes at least a couple of hours uh
on GPU. on GPU. on GPU.
Do not even try to … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 01:
21 | sponsor is going to give us a guest
lecture tomorrow. So, if you are going lecture tomorrow. So, if you are going lecture tomorrow. So, if you are going
to be using the sponsor credit, I think to be u… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 01:
58 | with cloud and it's it's going to work
I'm sure. Um and I already tested so so I'm sure. Um and I already tested so so I'm sure. Um and I already tested so so
it should work. And uh the Oh yes, and it… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 02:
41 | for the mini class than the semester for the mini class than the semester
class. So I think you guys are going to class. So I think you guys are going to class. So I think you guys are going to
be mor… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 03:
23 | anything. Okay, you don't you do not
need to submit any forms. Like it's just need to submit any forms. Like it's just need to submit any forms. Like it's just
like basically the the the the thing is … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 04:
04 | obviously I cannot grade you. Uh but if
you want to say like sit in a class you want to say like sit in a class you want to say like sit in a class
which are you already did um and or uh which are you… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 04:
38 | first quiz next next class is just going
to be the things that we have learned to be the things that we have learned to be the things that we have learned
throughout this week. There will be no throug… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 05:
17 | distribution of the data so that we can
actually draw new samples from it. Yeah. actually draw new samples from it. Yeah. actually draw new samples from it. Yeah.
And we have looked at a couple of And… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 06:
06 | you try you try to learn an encoder so
that it can encode it into some latent that it can encode it into some latent that it can encode it into some latent
space. Uh and then like it is an space. Uh a… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 06:
47 | trying to make sure that this encoding trying to make sure that this encoding
this notion of of encoding is the same this notion of of encoding is the same this notion of of encoding is the same
for b… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 07:
33 | data. Okay. So this is what we can data. Okay. So this is what we can
design but what we do not have is uh design but what we do not have is uh design but what we do not have is uh
basically the the a… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 08:
21 | try to make sure that um the encoder and try to make sure that um the encoder and
decoder are induced by the same um like decoder are induced by the same um like decoder are induced by the same um lik… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 09:
09 | You multiply by something and divide it
by something and then you get a equal by something and then you get a equal by something and then you get a equal
sign, right? Uh and basically this thing sign,… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 10:
00 | if your function is con concave then the
the sign flips. So here the log function the sign flips. So here the log function the sign flips. So here the log function
is a concave function. So that's why… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 10:
56 | remember like this thing like we do not
have a sampler of x, right? So we just have a sampler of x, right? So we just have a sampler of x, right? So we just
we need a sampler over z that is like we ne… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 11:
36 | anyone take a guess of like what what
kind of parameterization that we're kind of parameterization that we're kind of parameterization that we're
going to use here? going to use here? going to use her… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 12:
28 | parameterize Q Z uh Z over uh Z given parameterize Q Z uh Z over uh Z given
accent also a gausian also gausian right uh so also a gausian also gausian right uh so
basically it just just you just choos… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 13:
15 | of it to be a diagonal gausian then we
can literally just write out the par the can literally just write out the par the can literally just write out the par the
parameters of the distribution like th… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 14:
05 | okay. Okay. So because we have this and
both of them are Gaussian. Uh so that both of them are Gaussian. Uh so that both of them are Gaussian. Uh so that
this KL actually has a closed form. So this KL… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 14:
48 | decoder?
Anyone want to take a guess? Maybe a gian mixture more because like Maybe a gian mixture more because like
gosh is more. >> Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. >> Yeah. Yeah. Yeah. Yeah. Yeah… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 15:
58 | and then uh
it's uh magically it's uh magically it's uh magically
something that that is basically L2. So something that that is basically L2. So something that that is basically L2. So
that's why it'… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 17:
04 | genes is like expressive. So basically
they have some like lazy genes. This is they have some like lazy genes. This is they have some like lazy genes. This is
kind of like that. So you you are kind of… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 17:
53 | him? Yeah. him? Yeah.
>> Oh, okay. Okay. That's okay. Go ahead. >> Oh, okay. Okay. That's okay. Go ahead. >> Oh, okay. Okay. That's okay. Go ahead.
>> Is there variance of the uh the Q >> Is there var… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 18:
44 | >> Two he explained to me. One is that u if
it is not a dis a distribution progress it is not a dis a distribution progress it is not a dis a distribution progress
take modeling it is a one to one map… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 19:
38 | Also, by the way, they're like this is
like VAE is not the only way to do like VAE is not the only way to do like VAE is not the only way to do
latent variable like and and actually latent variable li… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 20:
28 | calculate uh the elbow loss and then you calculate uh the elbow loss and then you
uh optimize it. This is how you train a uh optimize it. This is how you train a uh optimize it. This is how you train … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 21:
15 | talk about but like they they're also talk about but like they they're also
likelihood based and we can also do a likelihood based and we can also do a likelihood based and we can also do a
likelihood… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 22:
07 | into a more complicated one. So this is into a more complicated one. So this is
like sort of like the general underlying like sort of like the general underlying like sort of like the general underlyi… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 23:
23 | from the from from the audience.
H this column. Hello, guys. H Which one H this column. Hello, guys. H Which one H this column. Hello, guys. H Which one
do you prefer? I'll let you guys pick. do you p… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 24:
14 | >> Okay. So, like, uh, yeah. So, for Gans,
it's very difficult to train, right? it's very difficult to train, right? it's very difficult to train, right?
Basically, that's what you said. Basically, th… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 25:
10 | saying right like they cannot
>> you like basically is kind of a >> you like basically is kind of a >> you like basically is kind of a
continuous continuous continuous
>> um so for g for for GANs espe… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 25:
57 | >> Uh VAES are going to kind of take the >> Uh VAES are going to kind of take the
average of a distribution because average of a distribution because average of a distribution because
they're enforcin… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 27:
01 | be able to calculate stuff one by one be able to calculate stuff one by one
and for text that's okay because text is and for text that's okay because text is and for text that's okay because text is
s… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 27:
35 | break things up that everything is going break things up that everything is going
to be really really slow, right? So say to be really really slow, right? So say to be really really slow, right? So sa… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 28:
17 | from Reddit seven years ago are like why
are VA so blurry? Yeah, that's because are VA so blurry? Yeah, that's because are VA so blurry? Yeah, that's because
uh you know uh vaes are exactly right uh y… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 29:
05 | average uh data all the time and GANs
are extremely extremely difficult to are extremely extremely difficult to are extremely extremely difficult to
train. Uh, so even though it works train. Uh, so ev… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 29:
48 | something called mode collapse which something called mode collapse which
just means that like basically it's sort just means that like basically it's sort just means that like basically it's sort
of … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 30:
33 | you sample a noise from a noise
distribution like Gausian and then you distribution like Gausian and then you distribution like Gausian and then you
apply some sort of like magical training apply some… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 31:
14 | Do you think that's easier? Yes, it is Do you think that's easier? Yes, it is
easier. This is a healthier type of easier. This is a healthier type of easier. This is a healthier type of
magic. It's no… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 32:
01 | that we take from a complete noise and that we take from a complete noise and
then we gradually try to dn noiseise it then we gradually try to dn noiseise it then we gradually try to dn noiseise it
so… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 32:
39 | is this is all makes sense. All gonna
make sense. We we just just just channel make sense. We we just just just channel make sense. We we just just just channel
your inner gausian and everything's you… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 33:
23 | We're going to talk about it or or like
does it actually need to be scaled? does it actually need to be scaled? does it actually need to be scaled?
We're going to talk about it next class. We're going… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 34:
10 | distribution by their mean and their
variance right this is basic or or yeah variance right this is basic or or yeah variance right this is basic or or yeah
so this is what's happening so this is what… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 35:
04 | >> great question in general? How big is t? >> great question in general? How big is t?
The this later on the the model that The this later on the the model that The this later on the the model that
w… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 35:
50 | I shall hope so because this is
literally the first step that how we literally the first step that how we literally the first step that how we
develop the elbow from VAE as well, develop the elbow fro… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 36:
44 | joint from time zero to time t. So we're
just going to break things up into just going to break things up into just going to break things up into
things that we know. So first of all, we things that w… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 37:
30 | you can turn the join distribution into
a chain of the the into a chain of like a chain of the the into a chain of like a chain of the the into a chain of like
the your current given the previous the … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 38:
24 | basically we can get
yeah anyway we can turn we can turn this yeah anyway we can turn we can turn this yeah anyway we can turn we can turn this
Q here into a combination of the three Q here into a com… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 39:
22 | something is equal to the sum of log of
this thing minus this thing plus this this thing minus this thing plus this this thing minus this thing plus this
thing minus the next thing and then it's thing… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 40:
19 | Yeah. A plus a plus a reconstruction Yeah. A plus a plus a reconstruction
laws. laws. laws.
Do we do we find this familiar? No way we don't find this familiar. Do No way we don't find this familiar. D… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 41:
22 | doesn't work that well. Uh so is is
there any way that we can make this work there any way that we can make this work there any way that we can make this work
because that whole thing is just too beca… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 42:
14 | thing is Markoff, right? Which means
that at each step, it just adds a very that at each step, it just adds a very that at each step, it just adds a very
small amount of noise to the previous small am… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 43:
05 | you absolutely could now you have ddpm you absolutely could now you have ddpm
so if you if if there's another slide so if you if if there's another slide so if you if if there's another slide
that you… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 43:
50 | it so we just fix it right we just have it so we just fix it right we just have
a fixed noising schedule that each time a fixed noising schedule that each time a fixed noising schedule that each time
… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 44:
38 | right? So one gausian plus another
gausian plus t gausian is still a gausian plus t gausian is still a gausian plus t gausian is still a
gausian. So instead of like at in the gausian. So instead of li… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 45:
24 | both Q and P to be the same gausian. So both Q and P to be the same gausian. So
it's it's going to be perfect every it's it's going to be perfect every it's it's going to be perfect every
time. Doesn'… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 46:
16 | know in closed form the mean and the
variance of this particular uh uh variance of this particular uh uh variance of this particular uh uh
distribution that we're going to use. distribution that we're… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 47:
00 | this this variance based on the variance
that you choose for the forward process. that you choose for the forward process. that you choose for the forward process.
But basically what we can do is we c… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 47:
53 | this mean versus the mean that you this mean versus the mean that you
predict Any predict Any predict Any
question? Nope. Alrighty. Nope. Alrighty.
All right. Now we have everything. All right. Now we… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 48:
48 | weight you should give to each based on
your your your time step. So this is the your your your time step. So this is the your your your time step. So this is the
reparameter same reparameterization r… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 49:
48 | Why is this a good thing you may ask? Why is this a good thing you may ask?
This is so complicated. Why do we want This is so complicated. Why do we want This is so complicated. Why do we want
this? I… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 50:
54 | uh of your that that can represent your
prediction. And then here is why you can prediction. And then here is why you can prediction. And then here is why you can
directly predict the noise. All right… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 51:
50 | literally they simplify into a L2 loss
between the the noise that you're adding between the the noise that you're adding between the the noise that you're adding
to your sample and the predicted noise… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 52:
44 | because the the the noise you predicted because the the the noise you predicted
is literally how much noise should I d is literally how much noise should I d is literally how much noise should I d
noi… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 53:
27 | the the the the expression for the
predicted mean of that is given by XT. predicted mean of that is given by XT. predicted mean of that is given by XT.
So your previous time your your current So your … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 54:
20 | step. You're not trying to jump from to
x0 yet. Uh in order to jump to x0 x0 yet. Uh in order to jump to x0 x0 yet. Uh in order to jump to x0
there's actually a paper about it. And there's actually a … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 55:
00 | plus a gausian noise. So this is still
this this distribution. Okay, this this distribution. Okay, this this distribution. Okay,
any other questions? Cool. Erh anyhow anyhow so that's it for Cool. Erh… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 55:
53 | then you try to like basically see if
the noise that you predict is accurate the noise that you predict is accurate the noise that you predict is accurate
uh in comparison to the the ground truth uh i… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 56:
31 | And then like 10,000 steps later, you And then like 10,000 steps later, you
get a sample. Make sense? Make sense?
Cool. Amazing. And and the model is Cool. Amazing. And and the model is Cool. Amazing.… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 57:
25 | latent let's just say and then later
we're trying to map this latent this we're trying to map this latent this we're trying to map this latent this
gausian uh into the a reconstruction of gausian uh i… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 58:
18 | X
>> no >> no >> no
>> well well basically it doesn't depend on X in well basically it doesn't depend on X in
the sense that it's not like the sense that it's not like the sense that it's not like
it … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 59:
15 | >> This will be closer to this one like you >> This will be closer to this one like you
can do interpolation which has an can do interpolation which has an can do interpolation which has an
effect. ef… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 60:
02 | should be an autoenccoder which means should be an autoenccoder which means
that uh it it should have some that uh it it should have some that uh it it should have some
compression actually. So basica… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 60:
44 | this is not very clear anymore and it's
because um it's like the the noise and because um it's like the the noise and because um it's like the the noise and
the data are of the same dimension. So the … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 61:
29 | called a unit. Okay. Uh so exactly what
it is is basically you have some input it is is basically you have some input it is is basically you have some input
like input image right and then you are lik… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 62:
16 | basically you are downsampling and
upsampling your uh your image so that upsampling your uh your image so that upsampling your uh your image so that
like basically you sort of covers both like basical… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 63:
10 | where input and outputs are about the where input and outputs are about the
same dimensionality same dimensionality same dimensionality
um spatially. Uh but basically later on um spatially. Uh but bas… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 63:
57 | because
uh we because we have this like rep like uh we because we have this like rep like uh we because we have this like rep like
this way to represent uh the the noisy this way to represent uh the t… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 64:
43 | literally just plug in this thing into
your mean and then you you should be your mean and then you you should be your mean and then you you should be
able to get a nicely formulated closed able to get… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 65:
25 | we should now know what diffusion model
is. And in this class, we basically just is. And in this class, we basically just is. And in this class, we basically just
derive the diffusion model from from … |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 00:
02 | Now that we have seen the probabilistic Now that we have seen the probabilistic
length of generative AI, let us actually length of generative AI, let us actually length of generative AI, let us actual… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 00:
50 | distribution space. Let's say this is distribution space. Let's say this is
noise and noise can live anywhere, noise and noise can live anywhere, noise and noise can live anywhere,
right? So we start … |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 01:
39 | and what good will this do to me if I and what good will this do to me if I
start from noise. Well, look at this start from noise. Well, look at this start from noise. Well, look at this
video, right?… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 02:
29 | this and we sequentially go on adding this and we sequentially go on adding
noise. We sequentially go on adding noise. We sequentially go on adding noise. We sequentially go on adding
noise until it b… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 03:
36 | I will train a machine learning model
to predict how much noise was added to predict how much noise was added to predict how much noise was added
to predict how much noise was added. Why to predict ho… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 04:
23 | step we have a model which predicts how
much noise has been added to each step. much noise has been added to each step. much noise has been added to each step.
So we have to train an ML model which So… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 05:
13 | So let's say if you have if you want to So let's say if you have if you want to
predict images which look like this predict images which look like this predict images which look like this
panda bear p… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 06:
01 | slowly. If you do this experiment at slowly. If you do this experiment at
home and if you stir this fluid very home and if you stir this fluid very home and if you stir this fluid very
fast and if you… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 07:
17 | diffusion dnoising and noising is also diffusion dnoising and noising is also
called as forward diffusion. So this called as forward diffusion. So this called as forward diffusion. So this
same intuit… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 07:
59 | So again to summarize the way diffusion So again to summarize the way diffusion
for images works is that we take a for images works is that we take a for images works is that we take a
random image fr… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 08:
53 | how to reduce the noise. how to reduce the noise.
By doing this repeatedly the model can By doing this repeatedly the model can By doing this repeatedly the model can
transform any noise to a sample t… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 09:
45 | That's the beauty of diffusion process.
So there are certain key characteristics So there are certain key characteristics So there are certain key characteristics
of diffusion. So there are key charac… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 10:
41 | here we use a machine learning model to here we use a machine learning model to
predict this noise. Um and to make it predict this noise. Um and to make it predict this noise. Um and to make it
more s… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 11:
28 | generate.
So these three are the key So these three are the key So these three are the key
characteristics of a diffusion model. characteristics of a diffusion model. characteristics of a diffusion mo… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 12:
37 | moving in a very uh straightforward
manner for image generation and people manner for image generation and people manner for image generation and people
were happy. There were companies which were hap… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 14:
14 | And we are just going to start looking And we are just going to start looking
at this field right now. I hope the at this field right now. I hope the at this field right now. I hope the
motivation for… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 00:
05 | Hello everyone and welcome to the third Hello everyone and welcome to the third
lecture of this course principles of lecture of this course principles of lecture of this course principles of
diffusion… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 01:
10 | hidden factors or the factors which hidden factors or the factors which
affect the variation in the data. For affect the variation in the data. For affect the variation in the data. For
example, if we… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 02:
12 | transform them into a Latin space with
only two dimensions X and Y data. only two dimensions X and Y data. only two dimensions X and Y data.
The Latin space the intuition behind The Latin space the in… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 03:
15 | it's also called as a style variable it's also called as a style variable
because it captures the style which is because it captures the style which is because it captures the style which is
inherent … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 04:
14 | you get the image out of it. That is
this machine is called as the decoder. this machine is called as the decoder. this machine is called as the decoder.
So the decoder takes in the latent So the deco… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 05:
25 | when we map the word hello into the
areas of the Latin space, we don't just areas of the Latin space, we don't just areas of the Latin space, we don't just
map it to one single value, but we map map i… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 06:
28 | we saw the Latin space after the
training is completed. It looks training is completed. It looks training is completed. It looks
something like this. something like this. something like this.
Now Now … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 07:
29 | space. You have a decoder which maps it
back. How do you train the variation back. How do you train the variation back. How do you train the variation
autoenccoder? What's the process that autoenccode… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 08:
32 | distribution.
So it tries to move the distribution in So it tries to move the distribution in So it tries to move the distribution in
the latin space to a gshian the latin space to a gshian the latin … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 09:
31 | of the students of your class and let's of the students of your class and let's
look at this factor which is neatness. look at this factor which is neatness. look at this factor which is neatness.
Oka… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 10:
19 | distributions that you see in in reality
and that's what people try to capture and that's what people try to capture and that's what people try to capture
over here. over here. over here.
So mathemati… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 11:
19 | understand uh the two types of losses understand uh the two types of losses
which we are encountering here. Okay. So, uh the lower bound is called Okay. So, uh the lower bound is called
as elbow which… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 12:
27 | P of X is not possible to calculate. P of X is not possible to calculate.
It's intractable It's intractable It's intractable
because you need to calculate an because you need to calculate an because y… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 13:
24 | say that is the recipe written in
standard language. Does it follow a standard language. Does it follow a standard language. Does it follow a
distribution which is shown in yellow? distribution which … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 14:
34 | And this is something we can clearly see
from this image as well. Every single from this image as well. Every single from this image as well. Every single
output image appears blur compared to output … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 15:
33 | linked to the decoder which means that linked to the decoder which means that
we have to train two neural networks in we have to train two neural networks in we have to train two neural networks in
th… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 16:
35 | which you can see over here denoising which you can see over here denoising
diffusion probabilistic models diffusion probabilistic models diffusion probabilistic models
and this paper has over and thi… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 17:
42 | literature.
So diffusion models existed before as So diffusion models existed before as So diffusion models existed before as
well but people had not proven that they well but people had not proven th… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 18:
36 | authors have written. So for that
purpose I would really recommend all of purpose I would really recommend all of purpose I would really recommend all of
you to take a print out and keep it you to tak… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 19:
42 | other corner of the room, right? So it's other corner of the room, right? So it's
it's it's like the it's it's like the it's it's like the
particles which are uh particles which are uh particles which… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 20:
45 | of artificial intelligence.
It sounded It sounded It sounded
like ridiculous because how how can you like ridiculous because how how can you like ridiculous because how how can you
apply this to AI, r… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 21:
44 | AI and uh we are going to see how the AI and uh we are going to see how the
diffusion technique is used for diffusion technique is used for diffusion technique is used for
reproducing the original dat… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 22:
43 | That's what we are going to think about
because our primary intention is to because our primary intention is to because our primary intention is to
reproduce the original or the true data reproduce th… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 23:
56 | uniform and noisy removes all the uniform and noisy removes all the
structure in the data. structure in the data. structure in the data.
What if we think of our encoder as being What if we think of ou… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 24:
58 | disappear. When people look at the final
image, they have no idea that this came image, they have no idea that this came image, they have no idea that this came
from Batman. This could have this this … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 25:
58 | step. First we will add some noise. Then step. First we will add some noise. Then
we'll add another noise and then slowly we'll add another noise and then slowly we'll add another noise and then slowl… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 26:
59 | am holding this bottle in front of me. I
take an image of this and I want take an image of this and I want take an image of this and I want
something which can transform this image something which can… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 27:
52 | see all of these individual grids. They
are one one pixel each and every single are one one pixel each and every single are one one pixel each and every single
pixel holds some value. For example, pix… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 29:
18 | obviously the value is not going to be
0.5. The value is going to be different 0.5. The value is going to be different 0.5. The value is going to be different
than 0.5. It's either going to be than 0.… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 30:
03 | the same deviation of of beta 1. and the same deviation of of beta 1. and
then we'll again sample from it. then we'll again sample from it. then we'll again sample from it.
So we will continue to do t… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 31:
06 | this. You can see that it's not this. You can see that it's not
completely noise but it's becoming completely noise but it's becoming completely noise but it's becoming
noisier, right? It's it's slowl… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 32:
07 | gshian transition kernel and you do it
several times. several times. several times.
So this is the process which is called So this is the process which is called So this is the process which is called… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 33:
16 | Let's try to understand. Let's try to understand.
Remember that we are preserving the mean Remember that we are preserving the mean Remember that we are preserving the mean
value of the pixels. The me… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 34:
31 | from a gshian which with a mean which is from a gshian which with a mean which is
slightly scaled down from this mean. slightly scaled down from this mean. slightly scaled down from this mean.
So this… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 36:
03 | down the mean, we have made sure that down the mean, we have made sure that
the mean of every single pixel goes down the mean of every single pixel goes down the mean of every single pixel goes down
f… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 36:
59 | bottles into pure noise. bottles into pure noise.
And that happens for any data that you And that happens for any data that you And that happens for any data that you
feed into the encoder. feed into … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 38:
02 | from where any image in the world can be
generated. generated. generated.
So essentially we are looking at So essentially we are looking at So essentially we are looking at
something which is at the h… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 38:
51 | >> This is given by this. The first image
I'm going to denote as X0. The second I'm going to denote as X0. The second I'm going to denote as X0. The second
image I'll denote as X1 and so on. Now image… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 39:
48 | also called as noise schedule.
So beta is what adds noise to the image. So beta is what adds noise to the image. So beta is what adds noise to the image.
Uh and we have different betas because Uh and … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 40:
38 | uh find out values for your alphas?
And this is something which uh even I And this is something which uh even I And this is something which uh even I
was thinking for some time how are the was thinkin… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 41:
45 | such that alpha square + beta square
always becomes one. That is how you choose these alphas and That is how you choose these alphas and
bet alphas. After you first choose the bet alphas. After you fi… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 43:
21 | Okay. So in the first step we import our
libraries. We are going to use numpy and libraries. We are going to use numpy and libraries. We are going to use numpy and
uh mattplot lab for this example. So… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 44:
21 | after 100 different steps.
Maybe this number you might not even Maybe this number you might not even Maybe this number you might not even
require to be this high, but people require to be this high, b… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 45:
21 | our discussion as well. So let me run
this. this. this.
Okay. Now we do the u Okay. Now we do the u Okay. Now we do the u
step-by-step forward diffusion process. step-by-step forward diffusion process… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 46:
15 | So it's it's it's it's a simple trick So it's it's it's it's a simple trick
which is to say that you which is to say that you which is to say that you
uh keep the original value or rather you can say … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 47:
26 | before here the mu gets changed by before here the mu gets changed by
alpha* alpha* alpha*
alpha* the original pixel value and the alpha* the original pixel value and the alpha* the original pixel val… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 48:
24 | total of 64x 64 pixels and for every total of 64x 64 pixels and for every
single pixel we choose a random variable single pixel we choose a random variable single pixel we choose a random variable
uh … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 49:
11 | there it really helps if we go step by
step right you can't generate an image step right you can't generate an image step right you can't generate an image
directly from noise that is too much directl… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 49:
56 | structure disappears and it becomes
uniform. So it satisfies our original uniform. So it satisfies our original uniform. So it satisfies our original
criteria that if we want this to be criteria that … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 50:
55 | want to go directly from here to here. want to go directly from here to here.
So there is a way to uh write the mean So there is a way to uh write the mean So there is a way to uh write the mean
and t… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 51:
52 | transition. If I give this as the
output, then write down the effective output, then write down the effective output, then write down the effective
mean and the effective variance for this mean and th… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 52:
37 | disappear and the second tenant is the
structure should become uniform with structure should become uniform with structure should become uniform with
time. This is exactly what we did with time. This … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 53:
31 | that we are going to handle right now.
So the the first intuition probably So the the first intuition probably So the the first intuition probably
tells us that to go from noise to the tells us that t… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 54:
26 | this image to noise
is is is
maybe scaling the mean by alpha and maybe scaling the mean by alpha and maybe scaling the mean by alpha and
adding beta. Why don't we just upscale adding beta. Why don't w… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 55:
29 | the original data.
So this will become clear as as we go So this will become clear as as we go So this will become clear as as we go
along. Let's let's move along. along. Let's let's move along. along… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 56:
26 | right. So
you can see that this is not like a you can see that this is not like a you can see that this is not like a
one-time denoising step like we did for one-time denoising step like we did for on… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 57:
31 | And that probability should be very And that probability should be very
high, right? because we want our decoder high, right? because we want our decoder high, right? because we want our decoder
to as… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 58:
30 | So if we maximize that lower bound it So if we maximize that lower bound it
also means that we are maximized the also means that we are maximized the also means that we are maximized the
true objectiv… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 59:
34 | before, we are going to get pure noise
only. So there is no doubt that this is only. So there is no doubt that this is only. So there is no doubt that this is
going to be pure noise. So this is going … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 60:
26 | So this is saying that I want to So this is saying that I want to
minimize the kale divergence between minimize the kale divergence between
these two probabilities. these two probabilities. these two … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 61:
32 | what we are trying to predict how do we
actually go from x2 to x1 this is what actually go from x2 to x1 this is what actually go from x2 to x1 this is what
our neural network will predict and our our… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 62:
24 | true data. So if we achieve this, then
we are anyways going to achieve a we are anyways going to achieve a we are anyways going to achieve a
reconstruction which is that is good. reconstruction which … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 63:
11 | So let's let's take an example and if So let's let's take an example and if
you are bothered by the word posterior you are bothered by the word posterior you are bothered by the word posterior
don't w… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 64:
08 | rain fell on this paper rain fell on this paper
how did it look like? how did it look like? how did it look like?
And that's very hard to tell because And that's very hard to tell because And that's v… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 65:
00 | maybe if I just remove some noise it
will look something similar to meat. will look something similar to meat. will look something similar to meat.
This will look something similar to me This will loo… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 66:
00 | you go from X3 to X2?
That is the question. Okay, That is the question. Okay, That is the question. Okay,
now now now
you have access to x3 and you also have you have access to x3 and you also have yo… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 67:
05 | fine but if we are able to predict this
why can't we just why can't we just why can't we just
match this directly to this? If if we match this directly to this? If if we match this directly to this? I… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 68:
18 | by the way this is one path that you by the way this is one path that you
take from the noise to the data data. take from the noise to the data data. take from the noise to the data data.
This is anot… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 69:
14 | understand first how how this posterior
is is predicted. is is predicted. is is predicted.
It turns out that this this reverse It turns out that this this reverse It turns out that this this reverse
p… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 70:
29 | How much importance should I place on X3 How much importance should I place on X3
and how much importance should I place and how much importance should I place and how much importance should I place
o… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 71:
26 | process. And I think we do have a graph
which talks about it. Yeah. Yeah. Here. which talks about it. Yeah. Yeah. Here. which talks about it. Yeah. Yeah. Here.
So here you see that uh the weightage on… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 72:
28 | So, so you should read the time as this
this to this. So, here we have the this to this. So, here we have the this to this. So, here we have the
complete noisy image complete noisy image complete nois… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 73:
23 | as a3 which is a function of all these as a3 which is a function of all these
variances. variances. variances.
So basically you go from uh the original So basically you go from uh the original So basi… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 74:
47 | the original image to the final image in
100 steps. We have already seen how to 100 steps. We have already seen how to 100 steps. We have already seen how to
uh write down this forward diffusion uh wr… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 75:
48 | alphas and betas which is mentioned in
the paper. So I have deliberately stayed the paper. So I have deliberately stayed the paper. So I have deliberately stayed
away from it. I don't want to get too … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 76:
53 | to 49 48 47 and finally you move to to 49 48 47 and finally you move to
the original image. So, so it actually the original image. So, so it actually the original image. So, so it actually
works right… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 77:
45 | their codes on GitHub.
So this alpha actually represents alpha So this alpha actually represents alpha So this alpha actually represents alpha
square in our notation and this beta square in our notati… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 78:
37 | this beta is it it represents beta this beta is it it represents beta
square which is the variance in our square which is the variance in our square which is the variance in our
notation and alpha rep… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 79:
28 | the image from noise. So the original the image from noise. So the original
image will not be given to us. image will not be given to us. image will not be given to us.
So basically we want to uh pred… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 80:
32 | Okay. So now we have two gshian
distributions. First goshian distributions. First goshian distributions. First goshian
distribution we know the mean and distribution we know the mean and distribution … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 81:
28 | which we have predicted over here.
>> [snorts] >> [snorts]
>> So now it boils down to a simple uh >> So now it boils down to a simple uh >> So now it boils down to a simple uh
algebraic manipulation. … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 82:
23 | really know. And now if you apply this
formula mu of 5 - mu² formula mu of 5 - mu² formula mu of 5 - mu²
what you see is you want to basically what you see is you want to basically what you see is you… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 83:
38 | uh the reverse transition process will
be same as that of the true posterior be same as that of the true posterior be same as that of the true posterior
because then you have the same because then you… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 84:
32 | it can be written as a function of the it can be written as a function of the
original image original image original image
and noise. and noise. and noise.
Remember this was the homework which I Remem… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 85:
34 | uh how much noise we are basically
adding at each step. adding at each step. adding at each step.
Okay. So uh but for now let's let's keep Okay. So uh but for now let's let's keep Okay. So uh but for … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 86:
55 | So So this is what this means. So x0 can
in fact be written in as in in this way. in fact be written in as in in this way. in fact be written in as in in this way.
And in the paper what they do is sin… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 87:
53 | >> Now you see the uh simplicity of this. >> Now you see the uh simplicity of this.
What this essentially means is that What this essentially means is that What this essentially means is that
what I n… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 88:
51 | we have imparted at each transition.
Once we have this noise level, we can Once we have this noise level, we can Once we have this noise level, we can
actually calculate the the the images at every si… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 90:
09 | equation which which they get
uh you know and here you see we finally uh you know and here you see we finally uh you know and here you see we finally
have epsilon minus epsilon theta whole have epsilo… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 91:
17 | diffusion models and and why they work. diffusion models and and why they work.
So uh please take a print out and go So uh please take a print out and go So uh please take a print out and go
through a… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 92:
13 | encoder as a goshian transition kernel
with a specific mean and variance. And with a specific mean and variance. And with a specific mean and variance. And
we have different iterations which go we hav… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 93:
05 | first two terms are exactly same like we
saw in VAS. The regularization term is saw in VAS. The regularization term is saw in VAS. The regularization term is
not important here because we already not … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 93:
54 | posterior in a very tractable way. And
we saw that it can be written as a we saw that it can be written as a we saw that it can be written as a
gshian with a mean and a variance. The gshian with a mea… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 94:
42 | more, we we found out that at the heart
of it, we want to make sure that the of it, we want to make sure that the of it, we want to make sure that the
original image true data which original image tru… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 95:
40 | in the forward diffusion process. in the forward diffusion process.
And this this makes a lot of sense. And this this makes a lot of sense. And this this makes a lot of sense.
That's why the name dino… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 96:
26 | please let me know
and just just sit down and go through and just just sit down and go through and just just sit down and go through
this homework problem where you find the this homework problem wher… |