cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 00:
03 | Cool. Hello everyone. Oh, it's not Cool. Hello everyone. Oh, it's not
significantly fewer. I guess it's significantly fewer. I guess it's significantly fewer. I guess it's
slightly significant. Okay, … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 00:
49 | actually takes time. So I I I I mean it
it takes at least a couple of hours uh it takes at least a couple of hours uh it takes at least a couple of hours uh
on GPU. on GPU. on GPU.
Do not even try to … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 01:
21 | sponsor is going to give us a guest
lecture tomorrow. So, if you are going lecture tomorrow. So, if you are going lecture tomorrow. So, if you are going
to be using the sponsor credit, I think to be u… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 01:
58 | with cloud and it's it's going to work
I'm sure. Um and I already tested so so I'm sure. Um and I already tested so so I'm sure. Um and I already tested so so
it should work. And uh the Oh yes, and it… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 02:
41 | for the mini class than the semester for the mini class than the semester
class. So I think you guys are going to class. So I think you guys are going to class. So I think you guys are going to
be mor… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 03:
23 | anything. Okay, you don't you do not
need to submit any forms. Like it's just need to submit any forms. Like it's just need to submit any forms. Like it's just
like basically the the the the thing is … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 04:
04 | obviously I cannot grade you. Uh but if
you want to say like sit in a class you want to say like sit in a class you want to say like sit in a class
which are you already did um and or uh which are you… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 04:
38 | first quiz next next class is just going
to be the things that we have learned to be the things that we have learned to be the things that we have learned
throughout this week. There will be no throug… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 05:
17 | distribution of the data so that we can
actually draw new samples from it. Yeah. actually draw new samples from it. Yeah. actually draw new samples from it. Yeah.
And we have looked at a couple of And… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 06:
06 | you try you try to learn an encoder so
that it can encode it into some latent that it can encode it into some latent that it can encode it into some latent
space. Uh and then like it is an space. Uh a… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 06:
47 | trying to make sure that this encoding trying to make sure that this encoding
this notion of of encoding is the same this notion of of encoding is the same this notion of of encoding is the same
for b… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 07:
33 | data. Okay. So this is what we can data. Okay. So this is what we can
design but what we do not have is uh design but what we do not have is uh design but what we do not have is uh
basically the the a… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 08:
21 | try to make sure that um the encoder and try to make sure that um the encoder and
decoder are induced by the same um like decoder are induced by the same um like decoder are induced by the same um lik… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 09:
09 | You multiply by something and divide it
by something and then you get a equal by something and then you get a equal by something and then you get a equal
sign, right? Uh and basically this thing sign,… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 10:
00 | if your function is con concave then the
the sign flips. So here the log function the sign flips. So here the log function the sign flips. So here the log function
is a concave function. So that's why… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 10:
56 | remember like this thing like we do not
have a sampler of x, right? So we just have a sampler of x, right? So we just have a sampler of x, right? So we just
we need a sampler over z that is like we ne… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 11:
36 | anyone take a guess of like what what
kind of parameterization that we're kind of parameterization that we're kind of parameterization that we're
going to use here? going to use here? going to use her… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 12:
28 | parameterize Q Z uh Z over uh Z given parameterize Q Z uh Z over uh Z given
accent also a gausian also gausian right uh so also a gausian also gausian right uh so
basically it just just you just choos… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 13:
15 | of it to be a diagonal gausian then we
can literally just write out the par the can literally just write out the par the can literally just write out the par the
parameters of the distribution like th… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 14:
05 | okay. Okay. So because we have this and
both of them are Gaussian. Uh so that both of them are Gaussian. Uh so that both of them are Gaussian. Uh so that
this KL actually has a closed form. So this KL… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 14:
48 | decoder?
Anyone want to take a guess? Maybe a gian mixture more because like Maybe a gian mixture more because like
gosh is more. >> Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. >> Yeah. Yeah. Yeah. Yeah. Yeah… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 15:
58 | and then uh
it's uh magically it's uh magically it's uh magically
something that that is basically L2. So something that that is basically L2. So something that that is basically L2. So
that's why it'… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 17:
04 | genes is like expressive. So basically
they have some like lazy genes. This is they have some like lazy genes. This is they have some like lazy genes. This is
kind of like that. So you you are kind of… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 17:
53 | him? Yeah. him? Yeah.
>> Oh, okay. Okay. That's okay. Go ahead. >> Oh, okay. Okay. That's okay. Go ahead. >> Oh, okay. Okay. That's okay. Go ahead.
>> Is there variance of the uh the Q >> Is there var… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 18:
44 | >> Two he explained to me. One is that u if
it is not a dis a distribution progress it is not a dis a distribution progress it is not a dis a distribution progress
take modeling it is a one to one map… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 19:
38 | Also, by the way, they're like this is
like VAE is not the only way to do like VAE is not the only way to do like VAE is not the only way to do
latent variable like and and actually latent variable li… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 20:
28 | calculate uh the elbow loss and then you calculate uh the elbow loss and then you
uh optimize it. This is how you train a uh optimize it. This is how you train a uh optimize it. This is how you train … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 21:
15 | talk about but like they they're also talk about but like they they're also
likelihood based and we can also do a likelihood based and we can also do a likelihood based and we can also do a
likelihood… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 22:
07 | into a more complicated one. So this is into a more complicated one. So this is
like sort of like the general underlying like sort of like the general underlying like sort of like the general underlyi… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 23:
23 | from the from from the audience.
H this column. Hello, guys. H Which one H this column. Hello, guys. H Which one H this column. Hello, guys. H Which one
do you prefer? I'll let you guys pick. do you p… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 24:
14 | >> Okay. So, like, uh, yeah. So, for Gans,
it's very difficult to train, right? it's very difficult to train, right? it's very difficult to train, right?
Basically, that's what you said. Basically, th… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 25:
10 | saying right like they cannot
>> you like basically is kind of a >> you like basically is kind of a >> you like basically is kind of a
continuous continuous continuous
>> um so for g for for GANs espe… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 25:
57 | >> Uh VAES are going to kind of take the >> Uh VAES are going to kind of take the
average of a distribution because average of a distribution because average of a distribution because
they're enforcin… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 27:
01 | be able to calculate stuff one by one be able to calculate stuff one by one
and for text that's okay because text is and for text that's okay because text is and for text that's okay because text is
s… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 27:
35 | break things up that everything is going break things up that everything is going
to be really really slow, right? So say to be really really slow, right? So say to be really really slow, right? So sa… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 28:
17 | from Reddit seven years ago are like why
are VA so blurry? Yeah, that's because are VA so blurry? Yeah, that's because are VA so blurry? Yeah, that's because
uh you know uh vaes are exactly right uh y… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 29:
05 | average uh data all the time and GANs
are extremely extremely difficult to are extremely extremely difficult to are extremely extremely difficult to
train. Uh, so even though it works train. Uh, so ev… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 29:
48 | something called mode collapse which something called mode collapse which
just means that like basically it's sort just means that like basically it's sort just means that like basically it's sort
of … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 30:
33 | you sample a noise from a noise
distribution like Gausian and then you distribution like Gausian and then you distribution like Gausian and then you
apply some sort of like magical training apply some… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 31:
14 | Do you think that's easier? Yes, it is Do you think that's easier? Yes, it is
easier. This is a healthier type of easier. This is a healthier type of easier. This is a healthier type of
magic. It's no… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 32:
01 | that we take from a complete noise and that we take from a complete noise and
then we gradually try to dn noiseise it then we gradually try to dn noiseise it then we gradually try to dn noiseise it
so… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 32:
39 | is this is all makes sense. All gonna
make sense. We we just just just channel make sense. We we just just just channel make sense. We we just just just channel
your inner gausian and everything's you… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 33:
23 | We're going to talk about it or or like
does it actually need to be scaled? does it actually need to be scaled? does it actually need to be scaled?
We're going to talk about it next class. We're going… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 34:
10 | distribution by their mean and their
variance right this is basic or or yeah variance right this is basic or or yeah variance right this is basic or or yeah
so this is what's happening so this is what… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 35:
04 | >> great question in general? How big is t? >> great question in general? How big is t?
The this later on the the model that The this later on the the model that The this later on the the model that
w… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 35:
50 | I shall hope so because this is
literally the first step that how we literally the first step that how we literally the first step that how we
develop the elbow from VAE as well, develop the elbow fro… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 36:
44 | joint from time zero to time t. So we're
just going to break things up into just going to break things up into just going to break things up into
things that we know. So first of all, we things that w… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 37:
30 | you can turn the join distribution into
a chain of the the into a chain of like a chain of the the into a chain of like a chain of the the into a chain of like
the your current given the previous the … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 38:
24 | basically we can get
yeah anyway we can turn we can turn this yeah anyway we can turn we can turn this yeah anyway we can turn we can turn this
Q here into a combination of the three Q here into a com… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 39:
22 | something is equal to the sum of log of
this thing minus this thing plus this this thing minus this thing plus this this thing minus this thing plus this
thing minus the next thing and then it's thing… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 40:
19 | Yeah. A plus a plus a reconstruction Yeah. A plus a plus a reconstruction
laws. laws. laws.
Do we do we find this familiar? No way we don't find this familiar. Do No way we don't find this familiar. D… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 41:
22 | doesn't work that well. Uh so is is
there any way that we can make this work there any way that we can make this work there any way that we can make this work
because that whole thing is just too beca… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 42:
14 | thing is Markoff, right? Which means
that at each step, it just adds a very that at each step, it just adds a very that at each step, it just adds a very
small amount of noise to the previous small am… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 43:
05 | you absolutely could now you have ddpm you absolutely could now you have ddpm
so if you if if there's another slide so if you if if there's another slide so if you if if there's another slide
that you… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 43:
50 | it so we just fix it right we just have it so we just fix it right we just have
a fixed noising schedule that each time a fixed noising schedule that each time a fixed noising schedule that each time
… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 44:
38 | right? So one gausian plus another
gausian plus t gausian is still a gausian plus t gausian is still a gausian plus t gausian is still a
gausian. So instead of like at in the gausian. So instead of li… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 45:
24 | both Q and P to be the same gausian. So both Q and P to be the same gausian. So
it's it's going to be perfect every it's it's going to be perfect every it's it's going to be perfect every
time. Doesn'… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 46:
16 | know in closed form the mean and the
variance of this particular uh uh variance of this particular uh uh variance of this particular uh uh
distribution that we're going to use. distribution that we're… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 47:
00 | this this variance based on the variance
that you choose for the forward process. that you choose for the forward process. that you choose for the forward process.
But basically what we can do is we c… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 47:
53 | this mean versus the mean that you this mean versus the mean that you
predict Any predict Any predict Any
question? Nope. Alrighty. Nope. Alrighty.
All right. Now we have everything. All right. Now we… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 48:
48 | weight you should give to each based on
your your your time step. So this is the your your your time step. So this is the your your your time step. So this is the
reparameter same reparameterization r… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 49:
48 | Why is this a good thing you may ask? Why is this a good thing you may ask?
This is so complicated. Why do we want This is so complicated. Why do we want This is so complicated. Why do we want
this? I… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 50:
54 | uh of your that that can represent your
prediction. And then here is why you can prediction. And then here is why you can prediction. And then here is why you can
directly predict the noise. All right… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 51:
50 | literally they simplify into a L2 loss
between the the noise that you're adding between the the noise that you're adding between the the noise that you're adding
to your sample and the predicted noise… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 52:
44 | because the the the noise you predicted because the the the noise you predicted
is literally how much noise should I d is literally how much noise should I d is literally how much noise should I d
noi… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 53:
27 | the the the the expression for the
predicted mean of that is given by XT. predicted mean of that is given by XT. predicted mean of that is given by XT.
So your previous time your your current So your … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 54:
20 | step. You're not trying to jump from to
x0 yet. Uh in order to jump to x0 x0 yet. Uh in order to jump to x0 x0 yet. Uh in order to jump to x0
there's actually a paper about it. And there's actually a … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 55:
00 | plus a gausian noise. So this is still
this this distribution. Okay, this this distribution. Okay, this this distribution. Okay,
any other questions? Cool. Erh anyhow anyhow so that's it for Cool. Erh… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 55:
53 | then you try to like basically see if
the noise that you predict is accurate the noise that you predict is accurate the noise that you predict is accurate
uh in comparison to the the ground truth uh i… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 56:
31 | And then like 10,000 steps later, you And then like 10,000 steps later, you
get a sample. Make sense? Make sense?
Cool. Amazing. And and the model is Cool. Amazing. And and the model is Cool. Amazing.… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 57:
25 | latent let's just say and then later
we're trying to map this latent this we're trying to map this latent this we're trying to map this latent this
gausian uh into the a reconstruction of gausian uh i… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 58:
18 | X
>> no >> no >> no
>> well well basically it doesn't depend on X in well basically it doesn't depend on X in
the sense that it's not like the sense that it's not like the sense that it's not like
it … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 59:
15 | >> This will be closer to this one like you >> This will be closer to this one like you
can do interpolation which has an can do interpolation which has an can do interpolation which has an
effect. ef… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 60:
02 | should be an autoenccoder which means should be an autoenccoder which means
that uh it it should have some that uh it it should have some that uh it it should have some
compression actually. So basica… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 60:
44 | this is not very clear anymore and it's
because um it's like the the noise and because um it's like the the noise and because um it's like the the noise and
the data are of the same dimension. So the … |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 61:
29 | called a unit. Okay. Uh so exactly what
it is is basically you have some input it is is basically you have some input it is is basically you have some input
like input image right and then you are lik… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 62:
16 | basically you are downsampling and
upsampling your uh your image so that upsampling your uh your image so that upsampling your uh your image so that
like basically you sort of covers both like basical… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 63:
10 | where input and outputs are about the where input and outputs are about the
same dimensionality same dimensionality same dimensionality
um spatially. Uh but basically later on um spatially. Uh but bas… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 63:
57 | because
uh we because we have this like rep like uh we because we have this like rep like uh we because we have this like rep like
this way to represent uh the the noisy this way to represent uh the t… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 64:
43 | literally just plug in this thing into
your mean and then you you should be your mean and then you you should be your mean and then you you should be
able to get a nicely formulated closed able to get… |
cmu-10799-diffusion-flow |
L3: CMU 10799 S26: Lecture 2 - Denoising Diffusion Models - Diffusion & Flow Matching | 65:
25 | we should now know what diffusion model
is. And in this class, we basically just is. And in this class, we basically just is. And in this class, we basically just
derive the diffusion model from from … |
cmu-10799-diffusion-flow |
L6: CMU 10799 S26: Lecture 7 - Guidance & Controllable Generation - Diffusion & Flow Matching | 37:
57 | so like B and if your model is really so like B and if your model is really
large the it's like a lot of memory large the it's like a lot of memory large the it's like a lot of memory
requirement righ… |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 17:
42 | consistency model he was at open eye and
then now he's apparently some big name then now he's apparently some big name then now he's apparently some big name
in meta super intelligence lab or in meta … |
cmu-10799-diffusion-flow |
L9: CMU 10799 S26: Lecture 10 - Distillation, Consistency Models & Flow Maps - Diffusion & Flow Matching | 18:
26 | DDPM. Yeah. Yeah. So, that guy is also a DDPM. Yeah. Yeah. So, that guy is also a
genius and he's now has his own startup. genius and he's now has his own startup. genius and he's now has his own star… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 00:
02 | Now that we have seen the probabilistic Now that we have seen the probabilistic
length of generative AI, let us actually length of generative AI, let us actually length of generative AI, let us actual… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 00:
50 | distribution space. Let's say this is distribution space. Let's say this is
noise and noise can live anywhere, noise and noise can live anywhere, noise and noise can live anywhere,
right? So we start … |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 01:
39 | and what good will this do to me if I and what good will this do to me if I
start from noise. Well, look at this start from noise. Well, look at this start from noise. Well, look at this
video, right?… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 02:
29 | this and we sequentially go on adding this and we sequentially go on adding
noise. We sequentially go on adding noise. We sequentially go on adding noise. We sequentially go on adding
noise until it b… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 03:
36 | I will train a machine learning model
to predict how much noise was added to predict how much noise was added to predict how much noise was added
to predict how much noise was added. Why to predict ho… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 04:
23 | step we have a model which predicts how
much noise has been added to each step. much noise has been added to each step. much noise has been added to each step.
So we have to train an ML model which So… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 05:
13 | So let's say if you have if you want to So let's say if you have if you want to
predict images which look like this predict images which look like this predict images which look like this
panda bear p… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 06:
01 | slowly. If you do this experiment at slowly. If you do this experiment at
home and if you stir this fluid very home and if you stir this fluid very home and if you stir this fluid very
fast and if you… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 07:
17 | diffusion dnoising and noising is also diffusion dnoising and noising is also
called as forward diffusion. So this called as forward diffusion. So this called as forward diffusion. So this
same intuit… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 07:
59 | So again to summarize the way diffusion So again to summarize the way diffusion
for images works is that we take a for images works is that we take a for images works is that we take a
random image fr… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 08:
53 | how to reduce the noise. how to reduce the noise.
By doing this repeatedly the model can By doing this repeatedly the model can By doing this repeatedly the model can
transform any noise to a sample t… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 09:
45 | That's the beauty of diffusion process.
So there are certain key characteristics So there are certain key characteristics So there are certain key characteristics
of diffusion. So there are key charac… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 10:
41 | here we use a machine learning model to here we use a machine learning model to
predict this noise. Um and to make it predict this noise. Um and to make it predict this noise. Um and to make it
more s… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 11:
28 | generate.
So these three are the key So these three are the key So these three are the key
characteristics of a diffusion model. characteristics of a diffusion model. characteristics of a diffusion mo… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 12:
37 | moving in a very uh straightforward
manner for image generation and people manner for image generation and people manner for image generation and people
were happy. There were companies which were hap… |
diffusion-lm-vizuara |
L5: Lecture 4: How Diffusion Models Work for Image Generation | 14:
14 | And we are just going to start looking And we are just going to start looking
at this field right now. I hope the at this field right now. I hope the at this field right now. I hope the
motivation for… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 00:
03 | Hello everyone. I'm very excited to be Hello everyone. I'm very excited to be
recording this video today because uh recording this video today because uh recording this video today because uh
I'm goin… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 01:
00 | focus on the technology behind
uh midjourney or stable diffusion but I uh midjourney or stable diffusion but I uh midjourney or stable diffusion but I
was so excited to try this out that I was so exci… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 01:
49 | related to the diffusion which I know
about I know about diffusion where let's about I know about diffusion where let's about I know about diffusion where let's
say I spray a perfume in one corner of … |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 02:
45 | these diffusion models have these diffusion models have
lasted for 10 to 15 years and every lasted for 10 to 15 years and every lasted for 10 to 15 years and every
development has used breakthroughs m… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 03:
43 | So this series has been made from that So this series has been made from that
point of view. I want to explain point of view. I want to explain point of view. I want to explain
diffusion models to you… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 04:
36 | I have structured this course and how I
plan to teach it. It'll be very similar plan to teach it. It'll be very similar plan to teach it. It'll be very similar
to Vijuara's other teaching material and… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 05:
43 | discuss is that of protein discuss is that of protein
structure prediction. structure prediction. structure prediction.
This has been a problem which has This has been a problem which has This has bee… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 06:
42 | seconds, 5 seconds, 10 seconds and now
you can generate up to 5 minute videos you can generate up to 5 minute videos you can generate up to 5 minute videos
using u generative AI and a lot of these usi… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 07:
42 | this advancement. Recently we are seeing this advancement. Recently we are seeing
a lot of foundational robotics models a lot of foundational robotics models a lot of foundational robotics models
bein… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 08:
43 | wonder is there something intrinsic
which is happening in diffusion that is which is happening in diffusion that is which is happening in diffusion that is
common in all these applications. But common… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 09:
42 | images and videos but also audio images and videos but also audio
synthesis. Now that we have established that Now that we have established that
diffusion models have a huge number of diffusion models… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 10:
55 | So you see there are these small videos So you see there are these small videos
15 minutes, 30 minutes, 12 minutes and 15 minutes, 30 minutes, 12 minutes and 15 minutes, 30 minutes, 12 minutes and
som… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 11:
48 | minutes maximum 30 minutes long. So very
little is known about how they work and little is known about how they work and little is known about how they work and
I think that diffusion models should be… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 12:
47 | Now the reason I'm coming out with this Now the reason I'm coming out with this
course right now is because I received a course right now is because I received a course right now is because I received… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 13:
42 | and one thing uh I realized after and one thing uh I realized after
reading this book and these thoughts reading this book and these thoughts reading this book and these thoughts
were also circling in… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 14:
31 | have divided the curriculum according to
this book. Uh and I I will not go very this book. Uh and I I will not go very this book. Uh and I I will not go very
theoretical in this section. So it might t… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 15:
20 | the first place? And what is the broader the first place? And what is the broader
objective we are trying to achieve here? objective we are trying to achieve here? objective we are trying to achieve h… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 16:
08 | lot in the last 10 years. As you can see
one of the first methods actually came one of the first methods actually came one of the first methods actually came
40 years back which was energy based 40 ye… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 17:
05 | we will cover all of these different we will cover all of these different
techniques individually. techniques individually. techniques individually.
So broadly speaking there are three So broadly spea… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 17:
59 | variational view then we will cover
score based view. Then we will cover score based view. Then we will cover score based view. Then we will cover
flow-based view and then after that we flow-based vie… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 18:
59 | finally in part three we are going to
understand how to sample from these understand how to sample from these understand how to sample from these
diffusion models. How to steer the diffusion models. H… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 19:
51 | actually attracted me to this field. I
want I I I wanted to study it in in in want I I I wanted to study it in in in want I I I wanted to study it in in in
more depth. more depth. more depth.
People w… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 20:
40 | reinforcement learning course just
finished a while back and uh after that finished a while back and uh after that finished a while back and uh after that
I have come up with this diffusion I have com… |
diffusion-principles-vizuara |
L11: The Principles of Diffusion Models - New Course Launch | 21:
32 | feeling you will not get by looking at
these concepts from a very surface level these concepts from a very surface level these concepts from a very surface level
to get that you need to understand the… |
diffusion-principles-vizuara |
L5: Lecture 6 - Denoising Score Matching | Principles of Diffusion Models | 32:
02 | really making sense. I have an excellent really making sense. I have an excellent
explanation to that which will come soon explanation to that which will come soon explanation to that which will come … |
diffusion-principles-vizuara |
L5: Lecture 6 - Denoising Score Matching | Principles of Diffusion Models | 39:
07 | you you you might have thought why
didn't I guess it before because it it didn't I guess it before because it it didn't I guess it before because it it
should be the vector which points from should be… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 34:
58 | trajectory?
Is is is it can I define it? Can I write Is is is it can I define it? Can I write Is is is it can I define it? Can I write
an equation which can tell me how do I an equation which can tell… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 38:
44 | So uh this is what we get if we actually
plot the score function. You can see plot the score function. You can see plot the score function. You can see
that the arrows are pointed towards the that the… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 00:
03 | Hello everyone. Hello everyone.
In this lecture, we will continue our In this lecture, we will continue our In this lecture, we will continue our
discussion on dnoising diffusion discussion on dnoisin… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 01:
10 | We will take an example a practical
example and we will exactly see how the example and we will exactly see how the example and we will exactly see how the
DDPM method is applied to that practical DDP… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 02:
07 | one corner of the room and then the
particles slowly diffuse and the smell particles slowly diffuse and the smell particles slowly diffuse and the smell
percolates in the room. percolates in the room.… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 03:
15 | input image into noise. input image into noise.
>> [snorts] >> [snorts]
>> This is not done through a single >> This is not done through a single >> This is not done through a single
transition but it… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 04:
18 | with, right? But uh when you actually
start doing the mathematics, this is not start doing the mathematics, this is not start doing the mathematics, this is not
very apparent at the beginning very app… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 05:
35 | execute the code and the forward and the
reverse transition process. I will reverse transition process. I will reverse transition process. I will
explain that in detail once we come to explain that in… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 06:
51 | And uh this is something which is you
need to install the git lfs to upload need to install the git lfs to upload need to install the git lfs to upload
your model checkpoints. Okay. So uh in this code… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 08:
13 | images as a single line as opposed to images as a single line as opposed to
converting them to a grid which is converting them to a grid which is converting them to a grid which is
happening in this f… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 09:
10 | true data distribution. We want to
predict the predict the predict the
data distribution which is P5 of X which data distribution which is P5 of X which data distribution which is P5 of X which
is as … |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 10:
28 | butterfly pictures.
And uh once you run this code And uh once you run this code And uh once you run this code
[snorts] [snorts]
in this code what we'll be doing is we in this code what we'll be doing … |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 11:
36 | we have divided uh
we have first of all got the data. I we have first of all got the data. I we have first of all got the data. I
don't think the train test split is done don't think the train test sp… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 12:
44 | the shape comes up like this the shape comes up like this
and here we are simply and here we are simply and here we are simply
resizing for u better viewing. So you resizing for u better viewing. So y… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 13:
51 | variance.
Now we had looked at this in quite some Now we had looked at this in quite some Now we had looked at this in quite some
detail to understand uh how these mean detail to understand uh how the… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 15:
13 | transitions. For example, x2 becomes transitions. For example, x2 becomes
alpha 2 x1 alpha 2 x1 alpha 2 x1
plus beta 2 epsilon. And you see here this epsilon this is And you see here this epsilon this… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 16:
59 | that alpha square + beta square is equal
to 1. So p + q = 1 to 1. So p + q = 1 to 1. So p + q = 1
and which which means that q = 1 - p. and which which means that q = 1 - p. and which which means that… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 18:
13 | scheduleuler does. Okay. So uh scheduleuler does. Okay. So uh
you see here what we are doing is from you see here what we are doing is from you see here what we are doing is from
diffusers we are impo… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 19:
37 | mean and the variance starting from the mean and the variance starting from the
first time step to anywhere in the first time step to anywhere in the first time step to anywhere in the
uh in in the fo… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 20:
43 | this noise tensor essentially means.
So imagine that you have uh this image So imagine that you have uh this image So imagine that you have uh this image
right right right
which is 32x 32 in this case… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 21:
48 | number of time steps important? because
you need to tell the you need to tell the you need to tell the
package how many steps you want to carry package how many steps you want to carry package how man… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 22:
56 | DDPM scheduleuler. You can write beta DDPM scheduleuler. You can write beta
start and beta end which means that you start and beta end which means that you start and beta end which means that you
can … |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 23:
46 | step. Now what do I mean by cumulative
mean? Well, cumulative means that if if mean? Well, cumulative means that if if mean? Well, cumulative means that if if
I have a mean of alpha 1 * x0 when I go I… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 24:
31 | mean changes es with time it it actually
reduces in magnitude and the cumulative reduces in magnitude and the cumulative reduces in magnitude and the cumulative
variance it it increases because as we … |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 25:
31 | Okay. So in fact it is not by3 I think Okay. So in fact it is not by3 I think
it's just 32x 32 because for every pixel it's just 32x 32 because for every pixel it's just 32x 32 because for every pixel… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 26:
18 | need to know how much each each pixel is
corrupted by and corrupted by and corrupted by and
that corruption level I'm identifying through through
uh this this noise value which I'm uh this this noise … |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 27:
21 | uh from this time step you are uh from this time step you are
transitioning so you're predicting what transitioning so you're predicting what transitioning so you're predicting what
is the image in th… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 28:
10 | Then I I again proceed ahead and I I
look at time step equal to two. And for look at time step equal to two. And for look at time step equal to two. And for
each of these time step, my image looks eac… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 29:
01 | passing both these quantities. passing both these quantities.
So the model has the input image. It So the model has the input image. It So the model has the input image. It
goes through several blocks… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 29:
49 | be the best and even in most of the
modern diffusion architectures you will modern diffusion architectures you will modern diffusion architectures you will
see the unit architecture very commonly see … |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 30:
42 | different output channels and the number different output channels and the number
of types in the down and the uplock. of types in the down and the uplock. of types in the down and the uplock.
This is… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 31:
28 | most important part of this lecture
because when we are looking at the because when we are looking at the because when we are looking at the
theory we don't really get to see the theory we don't reall… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 32:
19 | steps increases and the mean actually
goes down and because the sum of the goes down and because the sum of the goes down and because the sum of the
squares is equal to one. squares is equal to one. s… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 33:
29 | we do uh is in in this step. This is the we do uh is in in this step. This is the
main forward diffusion process. But I main forward diffusion process. But I main forward diffusion process. But I
will… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 34:
24 | process actually runs. What we say is process actually runs. What we say is
noise scheduleuler dot add noise. noise scheduleuler dot add noise. noise scheduleuler dot add noise.
So we are starting to … |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 35:
22 | in the next line I actually get the
model prediction. So I pass this these model prediction. So I pass this these model prediction. So I pass this these
noisy images noisy images noisy images
uh to my… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 36:
26 | predict how much noise has been added to predict how much noise has been added to
that image so that I can get back to my that image so that I can get back to my that image so that I can get back to m… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 37:
21 | subtracting an appropriate level of
noise from that image. So it is it it noise from that image. So it is it it noise from that image. So it is it it
becomes more robust to becomes more robust to beco… |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 38:
17 | prediction shape is like this 40x3x prediction shape is like this 40x3x
32x 32 and uh here 32x 32 and uh here 32x 32 and uh here
32x 32 is because it's the pixel size 32x 32 is because it's the pixel … |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 39:
28 | see the plots gradually varying. The
loss decreases with time which is loss decreases with time which is loss decreases with time which is
exactly what we expect. exactly what we expect. exactly what … |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 40:
18 | have a negative value and you go to the have a negative value and you go to the
previous sample. Then then you again previous sample. Then then you again previous sample. Then then you again
find the … |
diffusion-principles-vizuara |
L7: Lecture 4 - Diffusion Models Practical | DDPM | Principles of Diffusion Models | 41:
02 | slightly overwhelmed you, don't worry.
You can focus on this practical side of You can focus on this practical side of You can focus on this practical side of
the things more so that you get a more th… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 00:
05 | Hello everyone and welcome to the third Hello everyone and welcome to the third
lecture of this course principles of lecture of this course principles of lecture of this course principles of
diffusion… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 01:
10 | hidden factors or the factors which hidden factors or the factors which
affect the variation in the data. For affect the variation in the data. For affect the variation in the data. For
example, if we… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 02:
12 | transform them into a Latin space with
only two dimensions X and Y data. only two dimensions X and Y data. only two dimensions X and Y data.
The Latin space the intuition behind The Latin space the in… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 03:
15 | it's also called as a style variable it's also called as a style variable
because it captures the style which is because it captures the style which is because it captures the style which is
inherent … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 04:
14 | you get the image out of it. That is
this machine is called as the decoder. this machine is called as the decoder. this machine is called as the decoder.
So the decoder takes in the latent So the deco… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 05:
25 | when we map the word hello into the
areas of the Latin space, we don't just areas of the Latin space, we don't just areas of the Latin space, we don't just
map it to one single value, but we map map i… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 06:
28 | we saw the Latin space after the
training is completed. It looks training is completed. It looks training is completed. It looks
something like this. something like this. something like this.
Now Now … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 07:
29 | space. You have a decoder which maps it
back. How do you train the variation back. How do you train the variation back. How do you train the variation
autoenccoder? What's the process that autoenccode… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 08:
32 | distribution.
So it tries to move the distribution in So it tries to move the distribution in So it tries to move the distribution in
the latin space to a gshian the latin space to a gshian the latin … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 09:
31 | of the students of your class and let's of the students of your class and let's
look at this factor which is neatness. look at this factor which is neatness. look at this factor which is neatness.
Oka… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 10:
19 | distributions that you see in in reality
and that's what people try to capture and that's what people try to capture and that's what people try to capture
over here. over here. over here.
So mathemati… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 11:
19 | understand uh the two types of losses understand uh the two types of losses
which we are encountering here. Okay. So, uh the lower bound is called Okay. So, uh the lower bound is called
as elbow which… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 12:
27 | P of X is not possible to calculate. P of X is not possible to calculate.
It's intractable It's intractable It's intractable
because you need to calculate an because you need to calculate an because y… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 13:
24 | say that is the recipe written in
standard language. Does it follow a standard language. Does it follow a standard language. Does it follow a
distribution which is shown in yellow? distribution which … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 14:
34 | And this is something we can clearly see
from this image as well. Every single from this image as well. Every single from this image as well. Every single
output image appears blur compared to output … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 15:
33 | linked to the decoder which means that linked to the decoder which means that
we have to train two neural networks in we have to train two neural networks in we have to train two neural networks in
th… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 16:
35 | which you can see over here denoising which you can see over here denoising
diffusion probabilistic models diffusion probabilistic models diffusion probabilistic models
and this paper has over and thi… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 17:
42 | literature.
So diffusion models existed before as So diffusion models existed before as So diffusion models existed before as
well but people had not proven that they well but people had not proven th… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 18:
36 | authors have written. So for that
purpose I would really recommend all of purpose I would really recommend all of purpose I would really recommend all of
you to take a print out and keep it you to tak… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 19:
42 | other corner of the room, right? So it's other corner of the room, right? So it's
it's it's like the it's it's like the it's it's like the
particles which are uh particles which are uh particles which… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 20:
45 | of artificial intelligence.
It sounded It sounded It sounded
like ridiculous because how how can you like ridiculous because how how can you like ridiculous because how how can you
apply this to AI, r… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 21:
44 | AI and uh we are going to see how the AI and uh we are going to see how the
diffusion technique is used for diffusion technique is used for diffusion technique is used for
reproducing the original dat… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 22:
43 | That's what we are going to think about
because our primary intention is to because our primary intention is to because our primary intention is to
reproduce the original or the true data reproduce th… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 23:
56 | uniform and noisy removes all the uniform and noisy removes all the
structure in the data. structure in the data. structure in the data.
What if we think of our encoder as being What if we think of ou… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 24:
58 | disappear. When people look at the final
image, they have no idea that this came image, they have no idea that this came image, they have no idea that this came
from Batman. This could have this this … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 25:
58 | step. First we will add some noise. Then step. First we will add some noise. Then
we'll add another noise and then slowly we'll add another noise and then slowly we'll add another noise and then slowl… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 26:
59 | am holding this bottle in front of me. I
take an image of this and I want take an image of this and I want take an image of this and I want
something which can transform this image something which can… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 27:
52 | see all of these individual grids. They
are one one pixel each and every single are one one pixel each and every single are one one pixel each and every single
pixel holds some value. For example, pix… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 29:
18 | obviously the value is not going to be
0.5. The value is going to be different 0.5. The value is going to be different 0.5. The value is going to be different
than 0.5. It's either going to be than 0.… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 30:
03 | the same deviation of of beta 1. and the same deviation of of beta 1. and
then we'll again sample from it. then we'll again sample from it. then we'll again sample from it.
So we will continue to do t… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 31:
06 | this. You can see that it's not this. You can see that it's not
completely noise but it's becoming completely noise but it's becoming completely noise but it's becoming
noisier, right? It's it's slowl… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 32:
07 | gshian transition kernel and you do it
several times. several times. several times.
So this is the process which is called So this is the process which is called So this is the process which is called… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 33:
16 | Let's try to understand. Let's try to understand.
Remember that we are preserving the mean Remember that we are preserving the mean Remember that we are preserving the mean
value of the pixels. The me… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 34:
31 | from a gshian which with a mean which is from a gshian which with a mean which is
slightly scaled down from this mean. slightly scaled down from this mean. slightly scaled down from this mean.
So this… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 36:
03 | down the mean, we have made sure that down the mean, we have made sure that
the mean of every single pixel goes down the mean of every single pixel goes down the mean of every single pixel goes down
f… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 36:
59 | bottles into pure noise. bottles into pure noise.
And that happens for any data that you And that happens for any data that you And that happens for any data that you
feed into the encoder. feed into … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 38:
02 | from where any image in the world can be
generated. generated. generated.
So essentially we are looking at So essentially we are looking at So essentially we are looking at
something which is at the h… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 38:
51 | >> This is given by this. The first image
I'm going to denote as X0. The second I'm going to denote as X0. The second I'm going to denote as X0. The second
image I'll denote as X1 and so on. Now image… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 39:
48 | also called as noise schedule.
So beta is what adds noise to the image. So beta is what adds noise to the image. So beta is what adds noise to the image.
Uh and we have different betas because Uh and … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 40:
38 | uh find out values for your alphas?
And this is something which uh even I And this is something which uh even I And this is something which uh even I
was thinking for some time how are the was thinkin… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 41:
45 | such that alpha square + beta square
always becomes one. That is how you choose these alphas and That is how you choose these alphas and
bet alphas. After you first choose the bet alphas. After you fi… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 43:
21 | Okay. So in the first step we import our
libraries. We are going to use numpy and libraries. We are going to use numpy and libraries. We are going to use numpy and
uh mattplot lab for this example. So… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 44:
21 | after 100 different steps.
Maybe this number you might not even Maybe this number you might not even Maybe this number you might not even
require to be this high, but people require to be this high, b… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 45:
21 | our discussion as well. So let me run
this. this. this.
Okay. Now we do the u Okay. Now we do the u Okay. Now we do the u
step-by-step forward diffusion process. step-by-step forward diffusion process… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 46:
15 | So it's it's it's it's a simple trick So it's it's it's it's a simple trick
which is to say that you which is to say that you which is to say that you
uh keep the original value or rather you can say … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 47:
26 | before here the mu gets changed by before here the mu gets changed by
alpha* alpha* alpha*
alpha* the original pixel value and the alpha* the original pixel value and the alpha* the original pixel val… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 48:
24 | total of 64x 64 pixels and for every total of 64x 64 pixels and for every
single pixel we choose a random variable single pixel we choose a random variable single pixel we choose a random variable
uh … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 49:
11 | there it really helps if we go step by
step right you can't generate an image step right you can't generate an image step right you can't generate an image
directly from noise that is too much directl… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 49:
56 | structure disappears and it becomes
uniform. So it satisfies our original uniform. So it satisfies our original uniform. So it satisfies our original
criteria that if we want this to be criteria that … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 50:
55 | want to go directly from here to here. want to go directly from here to here.
So there is a way to uh write the mean So there is a way to uh write the mean So there is a way to uh write the mean
and t… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 51:
52 | transition. If I give this as the
output, then write down the effective output, then write down the effective output, then write down the effective
mean and the effective variance for this mean and th… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 52:
37 | disappear and the second tenant is the
structure should become uniform with structure should become uniform with structure should become uniform with
time. This is exactly what we did with time. This … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 53:
31 | that we are going to handle right now.
So the the first intuition probably So the the first intuition probably So the the first intuition probably
tells us that to go from noise to the tells us that t… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 54:
26 | this image to noise
is is is
maybe scaling the mean by alpha and maybe scaling the mean by alpha and maybe scaling the mean by alpha and
adding beta. Why don't we just upscale adding beta. Why don't w… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 55:
29 | the original data.
So this will become clear as as we go So this will become clear as as we go So this will become clear as as we go
along. Let's let's move along. along. Let's let's move along. along… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 56:
26 | right. So
you can see that this is not like a you can see that this is not like a you can see that this is not like a
one-time denoising step like we did for one-time denoising step like we did for on… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 57:
31 | And that probability should be very And that probability should be very
high, right? because we want our decoder high, right? because we want our decoder high, right? because we want our decoder
to as… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 58:
30 | So if we maximize that lower bound it So if we maximize that lower bound it
also means that we are maximized the also means that we are maximized the also means that we are maximized the
true objectiv… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 59:
34 | before, we are going to get pure noise
only. So there is no doubt that this is only. So there is no doubt that this is only. So there is no doubt that this is
going to be pure noise. So this is going … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 60:
26 | So this is saying that I want to So this is saying that I want to
minimize the kale divergence between minimize the kale divergence between
these two probabilities. these two probabilities. these two … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 61:
32 | what we are trying to predict how do we
actually go from x2 to x1 this is what actually go from x2 to x1 this is what actually go from x2 to x1 this is what
our neural network will predict and our our… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 62:
24 | true data. So if we achieve this, then
we are anyways going to achieve a we are anyways going to achieve a we are anyways going to achieve a
reconstruction which is that is good. reconstruction which … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 63:
11 | So let's let's take an example and if So let's let's take an example and if
you are bothered by the word posterior you are bothered by the word posterior you are bothered by the word posterior
don't w… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 64:
08 | rain fell on this paper rain fell on this paper
how did it look like? how did it look like? how did it look like?
And that's very hard to tell because And that's very hard to tell because And that's v… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 65:
00 | maybe if I just remove some noise it
will look something similar to meat. will look something similar to meat. will look something similar to meat.
This will look something similar to me This will loo… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 66:
00 | you go from X3 to X2?
That is the question. Okay, That is the question. Okay, That is the question. Okay,
now now now
you have access to x3 and you also have you have access to x3 and you also have yo… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 67:
05 | fine but if we are able to predict this
why can't we just why can't we just why can't we just
match this directly to this? If if we match this directly to this? If if we match this directly to this? I… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 68:
18 | by the way this is one path that you by the way this is one path that you
take from the noise to the data data. take from the noise to the data data. take from the noise to the data data.
This is anot… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 69:
14 | understand first how how this posterior
is is predicted. is is predicted. is is predicted.
It turns out that this this reverse It turns out that this this reverse It turns out that this this reverse
p… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 70:
29 | How much importance should I place on X3 How much importance should I place on X3
and how much importance should I place and how much importance should I place and how much importance should I place
o… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 71:
26 | process. And I think we do have a graph
which talks about it. Yeah. Yeah. Here. which talks about it. Yeah. Yeah. Here. which talks about it. Yeah. Yeah. Here.
So here you see that uh the weightage on… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 72:
28 | So, so you should read the time as this
this to this. So, here we have the this to this. So, here we have the this to this. So, here we have the
complete noisy image complete noisy image complete nois… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 73:
23 | as a3 which is a function of all these as a3 which is a function of all these
variances. variances. variances.
So basically you go from uh the original So basically you go from uh the original So basi… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 74:
47 | the original image to the final image in
100 steps. We have already seen how to 100 steps. We have already seen how to 100 steps. We have already seen how to
uh write down this forward diffusion uh wr… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 75:
48 | alphas and betas which is mentioned in
the paper. So I have deliberately stayed the paper. So I have deliberately stayed the paper. So I have deliberately stayed
away from it. I don't want to get too … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 76:
53 | to 49 48 47 and finally you move to to 49 48 47 and finally you move to
the original image. So, so it actually the original image. So, so it actually the original image. So, so it actually
works right… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 77:
45 | their codes on GitHub.
So this alpha actually represents alpha So this alpha actually represents alpha So this alpha actually represents alpha
square in our notation and this beta square in our notati… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 78:
37 | this beta is it it represents beta this beta is it it represents beta
square which is the variance in our square which is the variance in our square which is the variance in our
notation and alpha rep… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 79:
28 | the image from noise. So the original the image from noise. So the original
image will not be given to us. image will not be given to us. image will not be given to us.
So basically we want to uh pred… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 80:
32 | Okay. So now we have two gshian
distributions. First goshian distributions. First goshian distributions. First goshian
distribution we know the mean and distribution we know the mean and distribution … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 81:
28 | which we have predicted over here.
>> [snorts] >> [snorts]
>> So now it boils down to a simple uh >> So now it boils down to a simple uh >> So now it boils down to a simple uh
algebraic manipulation. … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 82:
23 | really know. And now if you apply this
formula mu of 5 - mu² formula mu of 5 - mu² formula mu of 5 - mu²
what you see is you want to basically what you see is you want to basically what you see is you… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 83:
38 | uh the reverse transition process will
be same as that of the true posterior be same as that of the true posterior be same as that of the true posterior
because then you have the same because then you… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 84:
32 | it can be written as a function of the it can be written as a function of the
original image original image original image
and noise. and noise. and noise.
Remember this was the homework which I Remem… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 85:
34 | uh how much noise we are basically
adding at each step. adding at each step. adding at each step.
Okay. So uh but for now let's let's keep Okay. So uh but for now let's let's keep Okay. So uh but for … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 86:
55 | So So this is what this means. So x0 can
in fact be written in as in in this way. in fact be written in as in in this way. in fact be written in as in in this way.
And in the paper what they do is sin… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 87:
53 | >> Now you see the uh simplicity of this. >> Now you see the uh simplicity of this.
What this essentially means is that What this essentially means is that What this essentially means is that
what I n… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 88:
51 | we have imparted at each transition.
Once we have this noise level, we can Once we have this noise level, we can Once we have this noise level, we can
actually calculate the the the images at every si… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 90:
09 | equation which which they get
uh you know and here you see we finally uh you know and here you see we finally uh you know and here you see we finally
have epsilon minus epsilon theta whole have epsilo… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 91:
17 | diffusion models and and why they work. diffusion models and and why they work.
So uh please take a print out and go So uh please take a print out and go So uh please take a print out and go
through a… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 92:
13 | encoder as a goshian transition kernel
with a specific mean and variance. And with a specific mean and variance. And with a specific mean and variance. And
we have different iterations which go we hav… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 93:
05 | first two terms are exactly same like we
saw in VAS. The regularization term is saw in VAS. The regularization term is saw in VAS. The regularization term is
not important here because we already not … |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 93:
54 | posterior in a very tractable way. And
we saw that it can be written as a we saw that it can be written as a we saw that it can be written as a
gshian with a mean and a variance. The gshian with a mea… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 94:
42 | more, we we found out that at the heart
of it, we want to make sure that the of it, we want to make sure that the of it, we want to make sure that the
original image true data which original image tru… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 95:
40 | in the forward diffusion process. in the forward diffusion process.
And this this makes a lot of sense. And this this makes a lot of sense. And this this makes a lot of sense.
That's why the name dino… |
diffusion-principles-vizuara |
L8: Lecture 3 - Introduction to Diffusion Models (DDPM) | Principles of Diffusion Models | 96:
26 | please let me know
and just just sit down and go through and just just sit down and go through and just just sit down and go through
this homework problem where you find the this homework problem wher… |