cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 00:
04 | So let's begin. Previously last class uh So let's begin. Previously last class uh
we have talked about what is diffusion we have talked about what is diffusion we have talked about what is diffusion
m… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 00:
57 | reserve sorry um all right so just a reserve sorry um all right so just a
reminder what is the forward process reminder what is the forward process reminder what is the forward process
basically ally … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 01:
44 | sums up together is another gausian and
you can just sum up a bunch of gausians. you can just sum up a bunch of gausians. you can just sum up a bunch of gausians.
So that's why in the forward process … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 02:
35 | is the diffusion for process. All right.
Now, uh how now that we uh we have the Now, uh how now that we uh we have the Now, uh how now that we uh we have the
forward process to train to construct forw… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 03:
24 | mean of the gausian uh and in addition
we can actually rewrite this mean this we can actually rewrite this mean this we can actually rewrite this mean this
prediction of mean into something that predi… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 04:
08 | last time there's some questions about
why should we do this reparameterization why should we do this reparameterization why should we do this reparameterization
well because doing this regression is … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 04:
54 | this uh before I do that does anyone
know in the audience what is normalizing know in the audience what is normalizing know in the audience what is normalizing
flow by any No way. Nobody know. No way.… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 05:
45 | to uh latent is the inverse of the
mapping from latent to data. So this is mapping from latent to data. So this is mapping from latent to data. So this is
sort of like normalizing flows. Um and sort o… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 06:
40 | if you have a auto reggressive models if you have a auto reggressive models
then you need to break things up by the then you need to break things up by the then you need to break things up by the
chai… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 07:
25 | And uh for normalizing flow just by
looking at this can anyone tell me what looking at this can anyone tell me what looking at this can anyone tell me what
potential problem it can it can have. Yes. Y… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 08:
23 | least. Um but then it as we have learned
last time it also has a problem of like last time it also has a problem of like last time it also has a problem of like
very unstable training and also it very… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 09:
24 | of images rather than actually having to of images rather than actually having to
comput the comput the comput the
>> yes that's right so you just need to >> yes that's right so you just need to >> ye… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 10:
13 | model so what you do is you sort of just
like follow the gra so say suppose you like follow the gra so say suppose you like follow the gra so say suppose you
have a gradient that gives you like u have… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 11:
00 | space right so basically what you can
imagine is that you can start from imagine is that you can start from imagine is that you can start from
anywhere on the data space so in this anywhere on the dat… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 11:
45 | respect to the data point of the log
likelihood so what is happening is that likelihood so what is happening is that likelihood so what is happening is that
like so and then again like what you can li… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 12:
31 | just like in diffusion right your source
distribution and your target distribution and your target distribution and your target
distribution should always be in the distribution should always be in th… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 13:
28 | gradient of uh of of P
X uh with respect to X. That's fine. But X uh with respect to X. That's fine. But X uh with respect to X. That's fine. But
later on we're going to see why because later on we're… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 14:
15 | is uh machine learning 101 right. So
very easy very easy uh uh uh very easy very easy uh uh uh very easy very easy uh uh uh
formulation. And this L2 here uh it has formulation. And this L2 here uh it … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 15:
08 | like auto reggressive models. So it can like auto reggressive models. So it can
sample or generate everything all at sample or generate everything all at sample or generate everything all at
once once… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 15:
47 | this L2 without this ground truth score.
Turns out you don't even need to use the Turns out you don't even need to use the Turns out you don't even need to use the
ground truth score in your loss grou… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 16:
35 | simplify uh the L2 the the original L2 simplify uh the L2 the the original L2
into two parts here. And then the first into two parts here. And then the first into two parts here. And then the first
pa… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 17:
27 | can have the gradient of uh log of p
with respect to x into you can break it with respect to x into you can break it with respect to x into you can break it
into this thing right uh and then these int… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 18:
13 | and then you can basically just get
something like this. So um yeah this is something like this. So um yeah this is something like this. So um yeah this is
a this is like we we probably wouldn't a thi… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 18:
58 | need to remember is that the second part
of the two things left. It's going to of the two things left. It's going to of the two things left. It's going to
come boils down to the negative of the come b… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 19:
41 | loss that we get, we don't really need a
ground true score. It's it's like ground true score. It's it's like ground true score. It's it's like
actually pretty pretty sick. So all you actually pretty p… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 20:
38 | function has the same dimensionality of function has the same dimensionality of
the of the data, right? Yeah. So, you the of the data, right? Yeah. So, you the of the data, right? Yeah. So, you
can ju… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 21:
37 | no one else. Do you have a Do you have a
Do you have a Do you have a Do you have a
>> Well, yes, but it's kind of related to >> Well, yes, but it's kind of related to >> Well, yes, but it's kind of re… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 22:
42 | Literally Gausian. Okay. So basically
what you do is uh and this is what we what you do is uh and this is what we what you do is uh and this is what we
call dnoising score matching. Why so get call dn… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 23:
25 | is you just need a sample data and then
perturb it a little bit and now it perturb it a little bit and now it perturb it a little bit and now it
becomes this like it's easier to becomes this like it's… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 24:
15 | literally this. Uh so yeah, for for literally this. Uh so yeah, for for
those of you who wonder why log density those of you who wonder why log density those of you who wonder why log density
easier b… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 25:
25 | truth it's a perturbed ground truth then truth it's a perturbed ground truth then
you may have a question but like let's you may have a question but like let's you may have a question but like let's
m… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 26:
04 | anyway it's fine. Um uh so what you do
is you first draw draw a sample from is you first draw draw a sample from is you first draw draw a sample from
your data space using some easy to your data space… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 26:
50 | So this is what could have happened if
you do laundry dynamic essentially. So you do laundry dynamic essentially. So you do laundry dynamic essentially. So
at the beginning, right, we just like at the… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 27:
44 | anyway you just train your model to
predict the score and then you use predict the score and then you use predict the score and then you use
dynamic to sample. Uh however does it dynamic to sample. Uh… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 29:
00 | survey. Let's survey. Uh we are starting
from this column again. from this column again. from this column again.
Hello friends. Uh Hello friends. Uh Hello friends. Uh
does anyone want to share their t… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 30:
06 | haven't been that I haven't been talked haven't been that I haven't been talked
to today. There's so many people in the to today. There's so many people in the to today. There's so many people in the
… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 31:
12 | is what we call the manifold
hypothesis. What does it mean is that hypothesis. What does it mean is that hypothesis. What does it mean is that
basically usually the data that we want basically usually… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 31:
59 | what here says like some area of the what here says like some area of the
data space will not have support. What data space will not have support. What data space will not have support. What
it means … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 32:
43 | manifold from some point from on the
manifold to some point that's not on the manifold to some point that's not on the manifold to some point that's not on the
manifold the score is going to blow up m… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 33:
28 | then the second thing. Yeah. Yeah. Yeah.
>> What is this the left plot even showing >> What is this the left plot even showing >> What is this the left plot even showing
either? Oh sorry the the the s… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 34:
10 | space or entire data space now has
support uh and then the score matching support uh and then the score matching support uh and then the score matching
actually behave really really nicely actually be… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 34:
57 | image is eventually boiled down to say
10 features or something in that case 10 features or something in that case 10 features or something in that case
the manifold is load eventually how is the mani… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 35:
45 | there's no probability and that's why there's no probability and that's why
the score is going to blow up and adding the score is going to blow up and adding the score is going to blow up and adding
s… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 36:
42 | is like they they they train on the is like they they they train on the
clean data. So it's like not I mean yes clean data. So it's like not I mean yes clean data. So it's like not I mean yes
obviousl… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 37:
28 | without any data right? So what's going
to happen here is that um so say this is to happen here is that um so say this is to happen here is that um so say this is
like your actual data score that your… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 38:
14 | really know where to move in the low
density region. So you're just going to density region. So you're just going to density region. So you're just going to
move sort of randomly and then you you move… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 39:
02 | high density from higher density right high density from higher density right
so like say this is your like intended so like say this is your like intended so like say this is your like intended
data … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 39:
48 | >> it it depends on like what exactly is >> it it depends on like what exactly is
this distribution say if this uh this distribution say if this uh this distribution say if this uh
distribution is a m… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 40:
34 | Okay.
All right. Any other question? Cool. Cool.
All right. So, we have all these All right. So, we have all these All right. So, we have all these
problems. problems. problems.
Is it problem? Is it p… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 41:
48 | thing doesn't exist anymore, right? And thing doesn't exist anymore, right? And
then now the perturbed like say you have then now the perturbed like say you have then now the perturbed like say you ha… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 42:
37 | large of a of a noise, then it you're
just learning a very noisy distribution. just learning a very noisy distribution. just learning a very noisy distribution.
That's not what we want. So how much Th… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 43:
33 | about which region that we are uh that about which region that we are uh that
we're s we should be sampling at, then we're s we should be sampling at, then we're s we should be sampling at, then
we're… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 44:
19 | sample from, you are going to just
gradually decrease the noise or the gradually decrease the noise or the gradually decrease the noise or the
exploration term that you're adding at exploration term t… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 45:
05 | have that next? uh we have the pseudo
code and you know the long dynam dynamic code and you know the long dynam dynamic code and you know the long dynam dynamic
left over. Yeah. But basically literall… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 45:
50 | And then at training time you do this
multi-level den noising score matching multi-level den noising score matching multi-level den noising score matching
because at each time you basically because at… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 47:
23 | Yeah, just just think about it. Uh,
basically the difference between DDPM, basically the difference between DDPM, basically the difference between DDPM,
which is what we learned last last time, which … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 48:
48 | that the the the score is sort of it has that the the the score is sort of it has
like another factor or it has another like another factor or it has another like another factor or it has another
coef… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 49:
49 | we have infinite numbers of noise level? we have infinite numbers of noise level?
What do you what do we think is going to What do you what do we think is going to What do you what do we think is goin… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 50:
47 | tailor estimation of of of the real
thing and you can actually write both of thing and you can actually write both of thing and you can actually write both of
them into this this format and uh if you … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 51:
42 | because you see how like basically if
you do xt plus delta t minus xt is going you do xt plus delta t minus xt is going you do xt plus delta t minus xt is going
to be this thing right. So it's going t… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 52:
28 | you just add some stoasticity. So this you just add some stoasticity. So this
is why it's called stochastic is why it's called stochastic is why it's called stochastic
differential equation. different… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 53:
28 | and we choose the for process right so
we know f and g uh and then it's we know f and g uh and then it's we know f and g uh and then it's
literally just gausian right so who literally just gausian rig… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 54:
21 | to use a oiler solver which is like to use a oiler solver which is like
probably simple simplest solver uh that probably simple simplest solver uh that probably simple simplest solver uh that
then you… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 55:
09 | apply the change in x to x and then we apply the change in x to x and then we
apply delta t to t apply delta t to t apply delta t to t
and that's it. and that's it. and that's it.
And then we iterate … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 56:
13 | written as this like this function this written as this like this function this
this uh so there are four process can be this uh so there are four process can be this uh so there are four process can … |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 57:
20 | the G as variance and in the VE
formulation we do not have any formulation we do not have any formulation we do not have any
adjustment of the X like whatever X you adjustment of the X like whatever X… |
cmu-10799-diffusion-flow |
L1: CMU 10799 S26: Lecture 4 - Score-based Models - Diffusion & Flow Matching | 58:
09 | did diffusion. This class we did did diffusion. This class we did
scorebased model and we also learned scorebased model and we also learned scorebased model and we also learned
that they're basically … |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 00:
04 | Hello everyone, welcome to the next Hello everyone, welcome to the next
lecture of the course principles of lecture of the course principles of lecture of the course principles of
diffusion models. di… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 01:
07 | calculate the magnetic field for all the
magnets and superimpose them together so magnets and superimpose them together so magnets and superimpose them together so
that we have a cumulative universal … |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 02:
13 | idea where this ball came from. idea where this ball came from.
But we have an idea where it came from. But we have an idea where it came from. But we have an idea where it came from.
And essentially … |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 03:
31 | Okay. And
the force required to pull the ball back the force required to pull the ball back the force required to pull the ball back
to the start is the score field for the to the start is the score f… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 04:
34 | Now this can also be represented as a
noise vector. Right? noise vector. Right? noise vector. Right?
[snorts] This is our flick. So [snorts] This is our flick. So [snorts] This is our flick. So
essent… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 05:
37 | And if this perturbed noise is very And if this perturbed noise is very
small, you could assume that whatever small, you could assume that whatever small, you could assume that whatever
score my I am … |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 06:
33 | exactly what we saw here. Minus noise
divided by sigma. divided by sigma. divided by sigma.
And the predicted score is what our And the predicted score is what our And the predicted score is what our
… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 07:
41 | as I said before uh the field of score
matching has evolved matching has evolved matching has evolved
through years. So in 2005 was the proof through years. So in 2005 was the proof through years. So … |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 08:
49 | that.
It's a very beautifully written paper It's a very beautifully written paper It's a very beautifully written paper
and in fact the textbook which I'm and in fact the textbook which I'm and in fac… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 09:
43 | image or data sample into noise and then
we predicted the reverse distribution we predicted the reverse distribution we predicted the reverse distribution
from the noise back to the data. from the noi… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 10:
59 | Now look at what is happening here. Your Now look at what is happening here. Your
space is huge. This is the your entire space is huge. This is the your entire space is huge. This is the your entire
s… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 12:
08 | 2005 which we explained and this is the
loss that we try to minimize. Okay. Now
this score is given by gradient of this score is given by gradient of this score is given by gradient of
log of p of x. … |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 13:
29 | completely fine. Everything is defined.
This the slope is defined right? You This the slope is defined right? You This the slope is defined right? You
know exactly how your data changes as know exactl… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 14:
27 | score vector will not be defined. score vector will not be defined.
And imagine that you're going with a And imagine that you're going with a And imagine that you're going with a
compass in your hand … |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 15:
28 | transformed from a low dimensional
manifold to a higher dimensional space. manifold to a higher dimensional space. manifold to a higher dimensional space.
It's taking up the space It's taking up the s… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 17:
00 | Remember in the previous formulation by Remember in the previous formulation by
Vincent, we just had one noise. Vincent, we just had one noise. Vincent, we just had one noise.
We we added one noise wi… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 18:
24 | you are flicking the magnet
not just once with one level, but you're not just once with one level, but you're not just once with one level, but you're
flicking it with multiple levels. flicking it wit… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 19:
42 | multiple levels of noise? multiple levels of noise?
We don't do this in diffusion also. In We don't do this in diffusion also. In We don't do this in diffusion also. In
diffusion there is one constant… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 20:
48 | is it a single neural network which is
trained on trained on trained on
uh for all these losses and if it's a uh for all these losses and if it's a uh for all these losses and if it's a
single neural … |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 22:
13 | Remember now the score does not just Remember now the score does not just
depend on X depend on X depend on X
but it depends on X comma sigma where but it depends on X comma sigma where but it depends… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 23:
27 | and land as close as possible to the and land as close as possible to the
data point. Now the interesting thing is that I have Now the interesting thing is that I have
a score function for a score fun… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 24:
58 | The only difference is that it is
performed sequentially performed sequentially performed sequentially
multiple times. Let us understand how we multiple times. Let us understand how we multiple times.… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 26:
21 | Okay. So you are purposely deliberately
inputting a very high noise level so inputting a very high noise level so inputting a very high noise level so
that that
uh you move to areas where you probably… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 27:
17 | here [snorts] the destination is not our
data but it's far away from our data data but it's far away from our data data but it's far away from our data
because we are adding so much noise. So because … |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 28:
10 | we are moving according to the lang
dynamics update rule. This time the dynamics update rule. This time the dynamics update rule. This time the
variance is a bit lower and um variance is a bit lower a… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 29:
07 | sample the entire data, right? Even if, sample the entire data, right? Even if,
let's say, you have handwritten digits let's say, you have handwritten digits let's say, you have handwritten digits
and… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 30:
00 | matching formula that it it it finally
tries to predict the noise itself and tries to predict the noise itself and tries to predict the noise itself and
that that is exactly what the diffusion that th… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 31:
01 | please go through this for people who please go through this for people who
like to see how both are connected and like to see how both are connected and like to see how both are connected and
it's it… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 32:
14 | uh and then
which is exactly what which is exactly what which is exactly what
we want to predict right we want to we want to predict right we want to we want to predict right we want to
understand tha… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 33:
12 | the reverse transition kernel is we are the reverse transition kernel is we are
trying to predict how much noise was trying to predict how much noise was trying to predict how much noise was
added in … |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 34:
09 | when we are inferring we do it n uh we when we are inferring we do it n uh we
do it l times as as we saw here there do it l times as as we saw here there do it l times as as we saw here there
are l di… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 34:
56 | this and you do it for some specific
time steps t so if the time step is time steps t so if the time step is time steps t so if the time step is
greater than t you just stop this greater than t you ju… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 35:
59 | So this is exactly how the anal So this is exactly how the anal
land dynamics loop looks like. land dynamics loop looks like. land dynamics loop looks like.
Let's take a look at a practical example Le… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 37:
01 | you
uh define your score network which is uh define your score network which is uh define your score network which is
typically a conditional unit. typically a conditional unit. typically a conditiona… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 38:
08 | And that is exactly what we see in the
practical U notebook as well. practical U notebook as well. practical U notebook as well.
[snorts] But this is done for different [snorts] But this is done for d… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 39:
15 | if I run this inference for this epox
where the loss is very low, you can see where the loss is very low, you can see where the loss is very low, you can see
it's almost 554, it's almost 554, it's alm… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 40:
06 | manifold
and by adding noise we are spreading it and by adding noise we are spreading it and by adding noise we are spreading it
so that it occupies the entire space. so that it occupies the entire sp… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 40:
48 | which have very low density of of the
data. And adding noise helps us to do data. And adding noise helps us to do data. And adding noise helps us to do
that. It helps us to have the gradient that. It … |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 41:
41 | strength of low strength also. And once strength of low strength also. And once
we train the network to understand all we train the network to understand all we train the network to understand all
the… |
diffusion-principles-vizuara |
L4: Lecture 7 - Noise Conditional Score Networks | Principles of Diffusion Models | 42:
36 | images, videos or audio. As an homework, images, videos or audio. As an homework,
I want you to play around with this I want you to play around with this I want you to play around with this
notebook. … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 00:
04 | Hello everyone and welcome to this next Hello everyone and welcome to this next
lecture in of the course principles of lecture in of the course principles of lecture in of the course principles of
dif… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 01:
15 | of X.
And our objective then becomes to And our objective then becomes to And our objective then becomes to
determine determine determine
the probability distribution P5 of X the probability distribut… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 02:
30 | The objective is to match the output
image with the input image as close as image with the input image as close as image with the input image as close as
possible. possible. possible.
And here we look… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 03:
46 | intelligence can be used to produce intelligence can be used to produce
images which look like they belong to images which look like they belong to images which look like they belong to
the underlying… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 04:
42 | it becomes uniform.
We took these key ideas and we We took these key ideas and we We took these key ideas and we
constructed a diffusion model which is constructed a diffusion model which is construct… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 05:
49 | So you inject noise at every transition.
So this is the first transition, second, So this is the first transition, second, So this is the first transition, second,
third etc. third etc. third etc.
Now… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 06:
45 | So our objective is to minimize the mean So our objective is to minimize the mean
square error between epsilon and epsilon square error between epsilon and epsilon square error between epsilon and eps… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 07:
41 | something which is common in every
single image on earth. And at a first single image on earth. And at a first single image on earth. And at a first
glance you're not able to answer it glance you're n… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 08:
36 | in the field of image generation through
generative AI. Models were extremely generative AI. Models were extremely generative AI. Models were extremely
important in creating a new wave in the importan… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 09:
45 | models in a single cohesive framework. models in a single cohesive framework.
So this is something which is very So this is something which is very So this is something which is very
unique and and ve… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 10:
48 | So in the energy based models our So in the energy based models our
objective is to predict P5 of X using objective is to predict P5 of X using objective is to predict P5 of X using
energy functions. … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 11:
57 | drop this ball in this curve.
What will happen is that the ball will What will happen is that the ball will What will happen is that the ball will
slowly settle down, slowly fall down and slowly settl… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 12:
57 | down and settle on the ground? The
reason is that at on on on the ground reason is that at on on on the ground reason is that at on on on the ground
the potential energy of the apple is is the potenti… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 13:
55 | energy function can be a proxy for how energy function can be a proxy for how
probable that particular data point or probable that particular data point or probable that particular data point or
sampl… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 14:
52 | course draw this schematically and we
can say that okay we want this energy or can say that okay we want this energy or can say that okay we want this energy or
or this probability curve to follow or … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 15:
51 | my function should satisfy my function should satisfy
and people use an exponential function and people use an exponential function and people use an exponential function
to relate the energy to the p… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 17:
03 | density as p5 of x. density as p5 of x.
Then from the above graph we can find Then from the above graph we can find Then from the above graph we can find
out a function which represents the out a func… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 17:
56 | system u every system finally tries to
go in a position with the minimum go in a position with the minimum go in a position with the minimum
possible energy and this is something possible energy and t… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 18:
46 | the whole idea behind energy based
models models
okay now let us discuss after this point okay now let us discuss after this point okay now let us discuss after this point
let's take an example suppos… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 20:
00 | add up to one? We need the probability
to add up to one because if we are to add up to one because if we are to add up to one because if we are
creating a lot of samples and we are creating a lot of s… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 20:
52 | so let's let's take that example again. so let's let's take that example again.
Uh let's say you have a dice. Okay. Uh Uh let's say you have a dice. Okay. Uh Uh let's say you have a dice. Okay. Uh
and… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 21:
42 | and the solution is that we can simply and the solution is that we can simply
normalize the probabilities by dividing normalize the probabilities by dividing normalize the probabilities by dividing
it… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 22:
48 | So for the discrete states it looked So for the discrete states it looked
like this summation of e ra to e5 of x like this summation of e ra to e5 of x like this summation of e ra to e5 of x
x goes fr… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 23:
46 | energy based models. In other words, how
do we find these energy functions which do we find these energy functions which do we find these energy functions which
correspond to each of these states? cor… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 24:
58 | push the good data up. push the good data up.
So I want to move this curve like this. So I want to move this curve like this. So I want to move this curve like this.
The left side I want to push down … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 25:
49 | higher probability. All of us understand higher probability. All of us understand
this. this. this.
Now we will again use the same Now we will again use the same Now we will again use the same
formula… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 26:
47 | Uh here E5 of X denotes the energy
function. Okay. So now if you break this down, Okay. So now if you break this down,
this is the first term that you get. uh this is the first term that you get. uh t… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 28:
00 | so p5 of x can be written as integral of
p5 of x given z into p of z dz. p5 of x given z into p of z dz. p5 of x given z into p of z dz.
And there we said that this integral is And there we said that … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 29:
00 | bypasses the partition function.
So you might be thinking that what is So you might be thinking that what is So you might be thinking that what is
this new terminology which I've this new terminology … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 29:
56 | And because of that we come up with a
alternative objective function which alternative objective function which alternative objective function which
uses something called as the score uses something c… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 31:
05 | appreciation for people who actually sit appreciation for people who actually sit
down, they do the mathematics and they down, they do the mathematics and they down, they do the mathematics and they
c… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 32:
17 | to occur. to occur.
Let's let let's take a sample example. Let's let let's take a sample example. Let's let let's take a sample example.
Let us say these black dots these are Let us say these black do… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 33:
18 | score function does. It tells you the score function does. It tells you the
direction to where the data is located. direction to where the data is located. direction to where the data is located.
So i… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 34:
09 | know where the data is where the score
is the maximum is the maximum is the maximum
we can start from any point and finally we can start from any point and finally we can start from any point and fina… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 34:
58 | trajectory?
Is is is it can I define it? Can I write Is is is it can I define it? Can I write Is is is it can I define it? Can I write
an equation which can tell me how do I an equation which can tell… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 35:
58 | Let's say I calculate the score for this Let's say I calculate the score for this
point. Where will this score point point. Where will this score point point. Where will this score point
towards to th… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 36:
53 | right or the left uh the value will
decline at x= to infinity or minus decline at x= to infinity or minus decline at x= to infinity or minus
infinity the value becomes zero. infinity the value becomes… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 37:
55 | Now this is just the first term. We have
to take a gradient of this also. to take a gradient of this also. to take a gradient of this also.
Gradient of a constant is zero and the Gradient of a constan… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 38:
44 | So uh this is what we get if we actually
plot the score function. You can see plot the score function. You can see plot the score function. You can see
that the arrows are pointed towards the that the… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 39:
41 | score-based framework but score-based framework but
score-based methods score-based methods score-based methods
is something which has come before is something which has come before is something which… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 40:
40 | challenges which we started out with. We
wanted something which can uh help us wanted something which can uh help us wanted something which can uh help us
offer a tractable alternative to offer a trac… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 42:
09 | Now what what happens is that when you
take this gradient uh the the first term take this gradient uh the the first term take this gradient uh the the first term
is minus grad of e5 of x which is simp… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 43:
06 | that's all.
So the partition function becomes zero So the partition function becomes zero So the partition function becomes zero
and is not included in the calculation and is not included in the calcu… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 44:
02 | makes sure that the score of the
predicted model matches the score of the predicted model matches the score of the predicted model matches the score of the
true probability distribution. true probabil… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 45:
33 | So ideally we want
a loss which looks like this. Now the a loss which looks like this. Now the a loss which looks like this. Now the
question again becomes how do we question again becomes how do we q… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 46:
22 | Let us assume that we have done the
training. We have matched this score training. We have matched this score training. We have matched this score
with the uh score for the true model for with the uh … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 47:
09 | data point and I know where to go I can
navigate my way and I can find samples navigate my way and I can find samples navigate my way and I can find samples
from the distribution Okay. And uh this fro… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 48:
18 | Here I can visually see that this is the
deepest valley and I can see the deepest valley and I can see the deepest valley and I can see the
treasure is hidden there. treasure is hidden there. treasure… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 49:
10 | valley and I start here. and I want to
reach the bottom of the valley. So I'll reach the bottom of the valley. So I'll reach the bottom of the valley. So I'll
make a step like this. I'll again make a … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 50:
12 | would get your next location.
But what is the slope here? How do we But what is the slope here? How do we But what is the slope here? How do we
find the slope? This is my energy find the slope? This i… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 51:
07 | exactly what we are doing here. We are exactly what we are doing here. We are
calculating the gradient of the energy calculating the gradient of the energy calculating the gradient of the energy
and u… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 51:
59 | this which is gradient of the energy
function. So the negative of this function. So the negative of this function. So the negative of this
actually is the score function itself. actually is the score … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 52:
52 | gradient is zero. So you are just
sitting there but there is another pit sitting there but there is another pit sitting there but there is another pit
which is even below which is even below which is … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 53:
40 | need to provide a shake which gives you
enough random energy to kick you out of enough random energy to kick you out of enough random energy to kick you out of
these small potholes so that you can the… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 54:
29 | we are sampling from a gshian we are sampling from a gshian
distribution with a mean of xi and a distribution with a mean of xi and a distribution with a mean of xi and a
standard deviation of beta. s… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 55:
30 | the total variance of this xt + 1 is
maybe a constant or it's just one that maybe a constant or it's just one that maybe a constant or it's just one that
is my hypothesis is my hypothesis is my hypoth… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 56:
32 | Now we what we do is we replace this Now we what we do is we replace this
grad of energy by s of x which we have grad of energy by s of x which we have grad of energy by s of x which we have
already s… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 57:
16 | trajectory let's say I start from here
with this update I'll go like this with this update I'll go like this with this update I'll go like this
and then I'll finally I I I need to come and then I'll f… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 57:
59 | the score function which takes you from
one location to the another and finally one location to the another and finally one location to the another and finally
you end up at a space or a region where … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 58:
55 | x is known to us. It looks something
like this. The probability of finding like this. The probability of finding like this. The probability of finding
the samples is maximum at these two the samples i… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 59:
55 | comments.
So first we define the setup which are So first we define the setup which are So first we define the setup which are
the two mountain setup as as we saw the two mountain setup as as we saw t… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 60:
40 | noise. We use the exact same equation
which we discuss in theory which we discuss in theory which we discuss in theory
and then we visualize the path. and then we visualize the path. and then we visua… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 61:
28 | It's it's called a drunk hiker.
It's like someone is drunk and walking It's like someone is drunk and walking It's like someone is drunk and walking
here and there and finally reaching the here and th… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 62:
18 | So I want all of you to run this tweak a
little things here and there have fun little things here and there have fun little things here and there have fun
with the notebook and really develop with the… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 63:
15 | since sampling with langu dynamics only
requires the score like we saw in this requires the score like we saw in this requires the score like we saw in this
formula it only requires the score we formu… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 64:
27 | and score matching fits this vector and score matching fits this vector
field by minimizing the mean square field by minimizing the mean square field by minimizing the mean square
error between the tr… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 65:
40 | know how to pronounce this name.
uh however they they came up with a uh however they they came up with a uh however they they came up with a
technique to technique to technique to
distill that objecti… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 66:
47 | a summation of this and a constant.
So basically minimizing this meant that So basically minimizing this meant that So basically minimizing this meant that
we want to minimize this and you see here we… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 68:
01 | like this. A1 A2 A3
A4 A5 A6 A4 A5 A6 A4 A5 A6
A7 A8 and A9. So the trace of this A7 A8 and A9. So the trace of this A7 A8 and A9. So the trace of this
matrix is simply the addition of a1 + a5 matrix … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 69:
11 | uh it is it is completely fine for our uh it is it is completely fine for our
purposes what we need to understand is purposes what we need to understand is purposes what we need to understand is
the i… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 70:
05 | Which makes a lot of sense, right? In Which makes a lot of sense, right? In
the previous example that we looked at the previous example that we looked at the previous example that we looked at
these t… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 71:
14 | where we are trying to make the data
samples appear as syncs. samples appear as syncs. samples appear as syncs.
which is what you see over here also. If which is what you see over here also. If which … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 72:
07 | more to this term.
So the this term drives So the this term drives So the this term drives
scores to zero in the higher probability scores to zero in the higher probability scores to zero in the highe… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 73:
01 | Now uh once we have understood this, Now uh once we have understood this,
this is where the actual training of the this is where the actual training of the this is where the actual training of the
uh … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 73:
57 | distribution. So in a way we will take a
look at the whole uh whole pipeline. look at the whole uh whole pipeline. look at the whole uh whole pipeline.
Okay, let's try to understand this. Okay, let's … |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 75:
18 | loss is made up of two terms. The first loss is made up of two terms. The first
is the trace of the Jacobian is the trace of the Jacobian is the trace of the Jacobian
which is what we discussed before… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 76:
10 | predictions for the x score and the y
score to create a vector. score to create a vector. score to create a vector.
Next we have a training loop which is a Next we have a training loop which is a Next… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 77:
00 | completely appear like a sync but there completely appear like a sync but there
are arrows coming in from the left and are arrows coming in from the left and are arrows coming in from the left and
the… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 77:
49 | we go to any other point. So we are we go to any other point. So we are
going to see trajectories like these. going to see trajectories like these. going to see trajectories like these.
And in this ex… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 78:
33 | perturve ourselves from the valleys so perturve ourselves from the valleys so
that we don't miss any other treasures that we don't miss any other treasures that we don't miss any other treasures
which… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 79:
19 | them yet. We will make the connection in
the next lecture. the next lecture. the next lecture.
So initially used to enable efficient So initially used to enable efficient So initially used to enable e… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 80:
15 | Scores are given by gradient of
logarithm of uh the probability. It logarithm of uh the probability. It logarithm of uh the probability. It
turns out that the scores appear as turns out that the score… |
diffusion-principles-vizuara |
L6: Lecture 5 - Energy Based Models | Score Function | Principles of Diffusion Models | 81:
02 | neural network to match the scores and neural network to match the scores and
to do that we use a score matching loss to do that we use a score matching loss to do that we use a score matching loss
wh… |